[PATCH v6 0/3] Add StarFive JH7110 PCIe drvier support

These PCIe series patches are based on the JH7110 RISC-V SoC and VisionFive V2 board.
The PCIe driver depends on gpio, pinctrl, clk and reset driver to do init. The PCIe dts configuation includes all these setting.
The PCIe drivers codes has been tested on the VisionFive V2 boards. The test devices includes M.2 NVMe SSD and Realtek 8169 Ethernet adapter.
previous patch version
v1: https://patchwork.ozlabs.org/project/uboot/cover/20230223105240.15180-1-mind... v2: https://patchwork.ozlabs.org/project/uboot/cover/20230308054833.95730-1-mind... v3: https://patchwork.ozlabs.org/project/uboot/cover/20230329100143.10724-1-mind... v4: https://patchwork.ozlabs.org/project/uboot/cover/20230411010209.76561-1-mind... v5: https://patchwork.ozlabs.org/project/uboot/cover/20230423105859.125764-1-min...
changes v6 patch 2 1. Change one VisionFive2 config file. patch 3 1. remove the pcie interrupt-controller. interrupts setting revert to origin.
setting. v5 patch 1 1. split PLDA controller driver codes to pcie_plda_common.c 2. correct the codes of record secondary number. patch 3 1. change the pcie dtsi config. make them the same with kernel.
v4 patch 1 1. Remove the IDS_REVISION_ID macros. 2. Replace sec_busno to first_busno in starfive_pcie 3. Remove starfive_pcie_off_conf function. 4. Replace "imply" to "depends on" in PCIe Kconfig. 5 .Check sec_busno in starfive_pcie_addr_valid.
v3 patch 1 1. remove the read vendor ID delay 2. remove starfive_pcie_hide_rc_bar function. do not hide host bridge BAR write. 3. Using PCIE_ECAM_OFFSET and PCI_CLASS_BRIDGE_PCI_NORMAL macros. 4. Add comments for bus and address limitation reason in function starfive_pcie_addr_valid 5. Change the multiple line comments in Line 373 6. Using gpio_request_by_name to get PCIe reset gpio,and using dm_gpio_set_value set GPIO value. patch 2 1. support PCIeboth 12a and 13b vf2 board. patch 3 1. reset dts change to reset-gpio.
v2 1. remove clock commit. The pcie clocks change has been includeded in [1]. 2. Using GENMASK marco1 in patch1. 3. remove the syscon dts node in patch3. The syscon dts dts node has been included in [1].
Mason Huo (3): starfive: pci: Add StarFive JH7110 pcie driver configs: starfive-jh7110: Add support for PCIe host driver riscv: dts: starfive: Enable PCIe host controller
.../dts/jh7110-starfive-visionfive-2.dtsi | 11 + arch/riscv/dts/jh7110.dtsi | 74 ++++ configs/starfive_visionfive2_defconfig | 10 + drivers/pci/Kconfig | 13 + drivers/pci/Makefile | 2 + drivers/pci/pcie_plda_common.c | 116 +++++++ drivers/pci/pcie_plda_common.h | 118 +++++++ drivers/pci/pcie_starfive_jh7110.c | 317 ++++++++++++++++++ 8 files changed, 661 insertions(+) create mode 100644 drivers/pci/pcie_plda_common.c create mode 100644 drivers/pci/pcie_plda_common.h create mode 100644 drivers/pci/pcie_starfive_jh7110.c
base-commit: c9c2c95d4cd27fe0cd41fe13a863899d268f973c

From: Mason Huo mason.huo@starfivetech.com
Add pcie driver for StarFive JH7110, Also add PLDA PCIe controller common driver functions.
Several devices are tested: a) M.2 NVMe SSD b) Realtek 8169 Ethernet adapter.
Signed-off-by: Mason Huo mason.huo@starfivetech.com Signed-off-by: Minda Chen minda.chen@starfivetech.com Acked-by: Pali Rohár pali@kernel.org --- drivers/pci/Kconfig | 13 ++ drivers/pci/Makefile | 2 + drivers/pci/pcie_plda_common.c | 116 +++++++++++ drivers/pci/pcie_plda_common.h | 118 +++++++++++ drivers/pci/pcie_starfive_jh7110.c | 317 +++++++++++++++++++++++++++++ 5 files changed, 566 insertions(+) create mode 100644 drivers/pci/pcie_plda_common.c create mode 100644 drivers/pci/pcie_plda_common.h create mode 100644 drivers/pci/pcie_starfive_jh7110.c
diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index ef328d2652..f2c5f0be63 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -374,4 +374,17 @@ config PCIE_UNIPHIER Say Y here if you want to enable PCIe controller support on UniPhier SoCs.
+config PCIE_PLDA_COMMON + bool + +config PCIE_STARFIVE_JH7110 + bool "Enable Starfive JH7110 PCIe driver" + select PCIE_PLDA_COMMON + imply STARFIVE_JH7110 + imply CLK_JH7110 + imply RESET_JH7110 + help + Say Y here if you want to enable PLDA XpressRich PCIe controller + support on StarFive JH7110 SoC. + endif diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile index 49506e7ba5..5a6974f013 100644 --- a/drivers/pci/Makefile +++ b/drivers/pci/Makefile @@ -49,3 +49,5 @@ obj-$(CONFIG_PCI_OCTEONTX) += pci_octeontx.o obj-$(CONFIG_PCIE_OCTEON) += pcie_octeon.o obj-$(CONFIG_PCIE_DW_SIFIVE) += pcie_dw_sifive.o obj-$(CONFIG_PCIE_UNIPHIER) += pcie_uniphier.o +obj-$(CONFIG_PCIE_PLDA_COMMON) += pcie_plda_common.o +obj-$(CONFIG_PCIE_STARFIVE_JH7110) += pcie_starfive_jh7110.o diff --git a/drivers/pci/pcie_plda_common.c b/drivers/pci/pcie_plda_common.c new file mode 100644 index 0000000000..005b92616a --- /dev/null +++ b/drivers/pci/pcie_plda_common.c @@ -0,0 +1,116 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * PLDA XpressRich PCIe host controller common functions. + * + * Copyright (C) 2023 StarFive Technology Co., Ltd. + * + */ + +#include <common.h> +#include <clk.h> +#include <dm.h> +#include <pci.h> +#include <pci_ids.h> +#include <asm/global_data.h> +#include <asm/io.h> +#include <dm/device_compat.h> +#include <linux/delay.h> +#include <linux/iopoll.h> +#include "pcie_plda_common.h" + +static bool plda_pcie_addr_valid(struct pcie_plda *plda, pci_dev_t bdf) +{ + /* + * Single device limitation. + * PCIe controller contain HW issue that secondary bus of + * host bridge emumerate duplicate devices. + * Only can access device 0 in secondary bus. + */ + if (PCI_BUS(bdf) == plda->sec_busno && PCI_DEV(bdf) > 0) + return false; + + return true; +} + +static int plda_pcie_conf_address(const struct udevice *udev, pci_dev_t bdf, + uint offset, void **paddr) +{ + struct pcie_plda *priv = dev_get_priv(udev); + int where = PCIE_ECAM_OFFSET(PCI_BUS(bdf), PCI_DEV(bdf), + PCI_FUNC(bdf), offset); + + if (!plda_pcie_addr_valid(priv, bdf)) + return -ENODEV; + + *paddr = (void *)(priv->cfg_base + where); + return 0; +} + +int plda_pcie_config_read(const struct udevice *udev, pci_dev_t bdf, + uint offset, ulong *valuep, + enum pci_size_t size) +{ + return pci_generic_mmap_read_config(udev, plda_pcie_conf_address, + bdf, offset, valuep, size); +} + +int plda_pcie_config_write(struct udevice *udev, pci_dev_t bdf, + uint offset, ulong value, + enum pci_size_t size) +{ + struct pcie_plda *priv = dev_get_priv(udev); + int ret; + + ret = pci_generic_mmap_write_config(udev, plda_pcie_conf_address, + bdf, offset, value, size); + + /* record secondary bus number */ + if (!ret && PCI_BUS(bdf) == dev_seq(udev) && + PCI_DEV(bdf) == 0 && PCI_FUNC(bdf) == 0 && + (offset == PCI_SECONDARY_BUS || + (offset == PCI_PRIMARY_BUS && size != PCI_SIZE_8))) { + priv->sec_busno = + ((offset == PCI_PRIMARY_BUS) ? (value >> 8) : value) & 0xff; + debug("Secondary bus number was changed to %d\n", + priv->sec_busno); + } + return ret; +} + +int plda_pcie_set_atr_entry(struct pcie_plda *plda, phys_addr_t src_addr, + phys_addr_t trsl_addr, phys_size_t window_size, + int trsl_param) +{ + void __iomem *base = + plda->reg_base + XR3PCI_ATR_AXI4_SLV0; + + /* Support AXI4 Slave 0 Address Translation Tables 0-7. */ + if (plda->atr_table_num >= XR3PCI_ATR_MAX_TABLE_NUM) { + dev_err(plda->dev, "ATR table number %d exceeds max num\n", + plda->atr_table_num); + return -EINVAL; + } + base += XR3PCI_ATR_TABLE_OFFSET * plda->atr_table_num; + plda->atr_table_num++; + + /* + * X3PCI_ATR_SRC_ADDR_LOW: + * - bit 0: enable entry, + * - bits 1-6: ATR window size: total size in bytes: 2^(ATR_WSIZE + 1) + * - bits 7-11: reserved + * - bits 12-31: start of source address + */ + writel((lower_32_bits(src_addr) & XR3PCI_ATR_SRC_ADDR_MASK) | + (fls(window_size) - 1) << XR3PCI_ATR_SRC_WIN_SIZE_SHIFT | 1, + base + XR3PCI_ATR_SRC_ADDR_LOW); + writel(upper_32_bits(src_addr), base + XR3PCI_ATR_SRC_ADDR_HIGH); + writel((lower_32_bits(trsl_addr) & XR3PCI_ATR_TRSL_ADDR_MASK), + base + XR3PCI_ATR_TRSL_ADDR_LOW); + writel(upper_32_bits(trsl_addr), base + XR3PCI_ATR_TRSL_ADDR_HIGH); + writel(trsl_param, base + XR3PCI_ATR_TRSL_PARAM); + + dev_dbg(plda->dev, "ATR entry: 0x%010llx %s 0x%010llx [0x%010llx] (param: 0x%06x)\n", + src_addr, (trsl_param & XR3PCI_ATR_TRSL_DIR) ? "<-" : "->", + trsl_addr, (u64)window_size, trsl_param); + return 0; +} diff --git a/drivers/pci/pcie_plda_common.h b/drivers/pci/pcie_plda_common.h new file mode 100644 index 0000000000..409949f534 --- /dev/null +++ b/drivers/pci/pcie_plda_common.h @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright (C) 2023 StarFive Technology Co., Ltd. + * Author: Minda Chen minda.chen@starfivetech.com + * + */ + +#ifndef PCIE_PLDA_COMMON_H +#define PCIE_PLDA_COMMON_H + +#define GEN_SETTINGS 0x80 +#define PCIE_PCI_IDS 0x9C +#define PCIE_WINROM 0xFC +#define PMSG_SUPPORT_RX 0x3F0 +#define PCI_MISC 0xB4 + +#define PLDA_EP_ENABLE 0 +#define PLDA_RP_ENABLE 1 + +#define IDS_CLASS_CODE_SHIFT 8 + +#define PREF_MEM_WIN_64_SUPPORT BIT(3) +#define PMSG_LTR_SUPPORT BIT(2) +#define PLDA_FUNCTION_DIS BIT(15) +#define PLDA_FUNC_NUM 4 +#define PLDA_PHY_FUNC_SHIFT 9 + +#define XR3PCI_ATR_AXI4_SLV0 0x800 +#define XR3PCI_ATR_SRC_ADDR_LOW 0x0 +#define XR3PCI_ATR_SRC_ADDR_HIGH 0x4 +#define XR3PCI_ATR_TRSL_ADDR_LOW 0x8 +#define XR3PCI_ATR_TRSL_ADDR_HIGH 0xc +#define XR3PCI_ATR_TRSL_PARAM 0x10 +#define XR3PCI_ATR_TABLE_OFFSET 0x20 +#define XR3PCI_ATR_MAX_TABLE_NUM 8 + +#define XR3PCI_ATR_SRC_WIN_SIZE_SHIFT 1 +#define XR3PCI_ATR_SRC_ADDR_MASK GENMASK(31, 12) +#define XR3PCI_ATR_TRSL_ADDR_MASK GENMASK(31, 12) +#define XR3PCI_ATR_TRSL_DIR BIT(22) +/* IDs used in the XR3PCI_ATR_TRSL_PARAM */ +#define XR3PCI_ATR_TRSLID_PCIE_MEMORY 0x0 +#define XR3PCI_ATR_TRSLID_PCIE_CONFIG 0x1 + +/** + * struct pcie_plda - PLDA PCIe controller state + * + * @reg_base: The base address of controller register space + * @cfg_base: The base address of configuration space + * @cfg_size: The size of configuration space + * @sec_busno: Secondary bus number. + * @atr_table_num: Total ATR table numbers. + */ +struct pcie_plda { + struct udevice *dev; + void __iomem *reg_base; + void __iomem *cfg_base; + phys_size_t cfg_size; + int sec_busno; + int atr_table_num; +}; + +int plda_pcie_config_read(const struct udevice *udev, pci_dev_t bdf, + uint offset, ulong *valuep, + enum pci_size_t size); +int plda_pcie_config_write(struct udevice *udev, pci_dev_t bdf, + uint offset, ulong value, + enum pci_size_t size); +int plda_pcie_set_atr_entry(struct pcie_plda *plda, phys_addr_t src_addr, + phys_addr_t trsl_addr, phys_size_t window_size, + int trsl_param); + +static inline void plda_pcie_enable_root_port(struct pcie_plda *plda) +{ + u32 value; + + value = readl(plda->reg_base + GEN_SETTINGS); + value |= PLDA_RP_ENABLE; + writel(value, plda->reg_base + GEN_SETTINGS); +} + +static inline void plda_pcie_set_standard_class(struct pcie_plda *plda) +{ + u32 value; + + value = readl(plda->reg_base + PCIE_PCI_IDS); + value &= 0xff; + value |= (PCI_CLASS_BRIDGE_PCI_NORMAL << IDS_CLASS_CODE_SHIFT); + writel(value, plda->reg_base + PCIE_PCI_IDS); +} + +static inline void plda_pcie_set_pref_win_64bit(struct pcie_plda *plda) +{ + u32 value; + + value = readl(plda->reg_base + PCIE_WINROM); + value |= PREF_MEM_WIN_64_SUPPORT; + writel(value, plda->reg_base + PCIE_WINROM); +} + +static inline void plda_pcie_disable_ltr(struct pcie_plda *plda) +{ + u32 value; + + value = readl(plda->reg_base + PMSG_SUPPORT_RX); + value &= ~PMSG_LTR_SUPPORT; + writel(value, plda->reg_base + PMSG_SUPPORT_RX); +} + +static inline void plda_pcie_disable_func(struct pcie_plda *plda) +{ + u32 value; + + value = readl(plda->reg_base + PCI_MISC); + value |= PLDA_FUNCTION_DIS; + writel(value, plda->reg_base + PCI_MISC); +} +#endif diff --git a/drivers/pci/pcie_starfive_jh7110.c b/drivers/pci/pcie_starfive_jh7110.c new file mode 100644 index 0000000000..903a544d37 --- /dev/null +++ b/drivers/pci/pcie_starfive_jh7110.c @@ -0,0 +1,317 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * StarFive PLDA PCIe host controller driver + * + * Copyright (C) 2023 StarFive Technology Co., Ltd. + * Author: Mason Huo mason.huo@starfivetech.com + * + */ + +#include <common.h> +#include <clk.h> +#include <dm.h> +#include <pci.h> +#include <pci_ids.h> +#include <power-domain.h> +#include <regmap.h> +#include <reset.h> +#include <syscon.h> +#include <asm/global_data.h> +#include <asm/io.h> +#include <asm-generic/gpio.h> +#include <dm/device_compat.h> +#include <dm/pinctrl.h> +#include <linux/delay.h> +#include <linux/iopoll.h> +#include "pcie_plda_common.h" + +/* system control */ +#define STG_SYSCON_K_RP_NEP_MASK BIT(8) +#define STG_SYSCON_AXI4_SLVL_ARFUNC_MASK GENMASK(22, 8) +#define STG_SYSCON_AXI4_SLVL_ARFUNC_SHIFT 8 +#define STG_SYSCON_AXI4_SLVL_AWFUNC_MASK GENMASK(14, 0) +#define STG_SYSCON_CLKREQ_MASK BIT(22) +#define STG_SYSCON_CKREF_SRC_SHIFT 18 +#define STG_SYSCON_CKREF_SRC_MASK GENMASK(19, 18) + +DECLARE_GLOBAL_DATA_PTR; + +struct starfive_pcie { + struct pcie_plda plda; + struct clk_bulk clks; + struct reset_ctl_bulk rsts; + struct gpio_desc reset_gpio; + struct regmap *regmap; + u32 stg_arfun; + u32 stg_awfun; + u32 stg_rp_nep; +}; + +static int starfive_pcie_atr_init(struct starfive_pcie *priv) +{ + struct udevice *ctlr = pci_get_controller(priv->plda.dev); + struct pci_controller *hose = dev_get_uclass_priv(ctlr); + int i, ret; + + /* + * As the two host bridges in JH7110 soc have the same default + * address translation table, this cause the second root port can't + * access it's host bridge config space correctly. + * To workaround, config the ATR of host bridge config space by SW. + */ + + ret = plda_pcie_set_atr_entry(&priv->plda, + (phys_addr_t)priv->plda.cfg_base, 0, + priv->plda.cfg_size, + XR3PCI_ATR_TRSLID_PCIE_CONFIG); + if (ret) + return ret; + + for (i = 0; i < hose->region_count; i++) { + if (hose->regions[i].flags == PCI_REGION_SYS_MEMORY) + continue; + + /* Only support identity mappings. */ + if (hose->regions[i].bus_start != + hose->regions[i].phys_start) + return -EINVAL; + + ret = plda_pcie_set_atr_entry(&priv->plda, + hose->regions[i].phys_start, + hose->regions[i].bus_start, + hose->regions[i].size, + XR3PCI_ATR_TRSLID_PCIE_MEMORY); + if (ret) + return ret; + } + + return 0; +} + +static int starfive_pcie_get_syscon(struct udevice *dev) +{ + struct starfive_pcie *priv = dev_get_priv(dev); + struct udevice *syscon; + struct ofnode_phandle_args syscfg_phandle; + u32 cells[4]; + int ret; + + /* get corresponding syscon phandle */ + ret = dev_read_phandle_with_args(dev, "starfive,stg-syscon", NULL, 0, 0, + &syscfg_phandle); + + if (ret < 0) { + dev_err(dev, "Can't get syscfg phandle: %d\n", ret); + return ret; + } + + ret = uclass_get_device_by_ofnode(UCLASS_SYSCON, syscfg_phandle.node, + &syscon); + if (ret) { + dev_err(dev, "Unable to find syscon device (%d)\n", ret); + return ret; + } + + priv->regmap = syscon_get_regmap(syscon); + if (!priv->regmap) { + dev_err(dev, "Unable to find regmap\n"); + return -ENODEV; + } + + /* get syscon register offset */ + ret = dev_read_u32_array(dev, "starfive,stg-syscon", + cells, ARRAY_SIZE(cells)); + if (ret) { + dev_err(dev, "Get syscon register err %d\n", ret); + return -EINVAL; + } + + dev_dbg(dev, "Get syscon values: %x, %x, %x\n", + cells[1], cells[2], cells[3]); + priv->stg_arfun = cells[1]; + priv->stg_awfun = cells[2]; + priv->stg_rp_nep = cells[3]; + + return 0; +} + +static int starfive_pcie_parse_dt(struct udevice *dev) +{ + struct starfive_pcie *priv = dev_get_priv(dev); + int ret; + + priv->plda.reg_base = (void *)dev_read_addr_name(dev, "reg"); + if (priv->plda.reg_base == (void __iomem *)FDT_ADDR_T_NONE) { + dev_err(dev, "Missing required reg address range\n"); + return -EINVAL; + } + + priv->plda.cfg_base = + (void *)dev_read_addr_size_name(dev, + "config", + &priv->plda.cfg_size); + if (priv->plda.cfg_base == (void __iomem *)FDT_ADDR_T_NONE) { + dev_err(dev, "Missing required config address range"); + return -EINVAL; + } + + ret = starfive_pcie_get_syscon(dev); + if (ret) { + dev_err(dev, "Can't get syscon: %d\n", ret); + return ret; + } + + ret = reset_get_bulk(dev, &priv->rsts); + if (ret) { + dev_err(dev, "Can't get reset: %d\n", ret); + return ret; + } + + ret = clk_get_bulk(dev, &priv->clks); + if (ret) { + dev_err(dev, "Can't get clock: %d\n", ret); + return ret; + } + + ret = gpio_request_by_name(dev, "reset-gpios", 0, &priv->reset_gpio, + GPIOD_IS_OUT); + if (ret) { + dev_err(dev, "Can't get reset-gpio: %d\n", ret); + return ret; + } + + if (!dm_gpio_is_valid(&priv->reset_gpio)) { + dev_err(dev, "reset-gpio is not valid\n"); + return -EINVAL; + } + return 0; +} + +static int starfive_pcie_init_port(struct udevice *dev) +{ + int ret, i; + struct starfive_pcie *priv = dev_get_priv(dev); + struct pcie_plda *plda = &priv->plda; + + ret = clk_enable_bulk(&priv->clks); + if (ret) { + dev_err(dev, "Failed to enable clks (ret=%d)\n", ret); + return ret; + } + + ret = reset_deassert_bulk(&priv->rsts); + if (ret) { + dev_err(dev, "Failed to deassert resets (ret=%d)\n", ret); + goto err_deassert_clk; + } + + dm_gpio_set_value(&priv->reset_gpio, 1); + /* Disable physical functions except #0 */ + for (i = 1; i < PLDA_FUNC_NUM; i++) { + regmap_update_bits(priv->regmap, + priv->stg_arfun, + STG_SYSCON_AXI4_SLVL_ARFUNC_MASK, + (i << PLDA_PHY_FUNC_SHIFT) << + STG_SYSCON_AXI4_SLVL_ARFUNC_SHIFT); + regmap_update_bits(priv->regmap, + priv->stg_awfun, + STG_SYSCON_AXI4_SLVL_AWFUNC_MASK, + i << PLDA_PHY_FUNC_SHIFT); + + plda_pcie_disable_func(plda); + } + + /* Disable physical functions */ + regmap_update_bits(priv->regmap, + priv->stg_arfun, + STG_SYSCON_AXI4_SLVL_ARFUNC_MASK, + 0); + regmap_update_bits(priv->regmap, + priv->stg_awfun, + STG_SYSCON_AXI4_SLVL_AWFUNC_MASK, + 0); + + plda_pcie_enable_root_port(plda); + + /* PCIe PCI Standard Configuration Identification Settings. */ + plda_pcie_set_standard_class(plda); + + /* + * The LTR message forwarding of PCIe Message Reception was set by core + * as default, but the forward id & addr are also need to be reset. + * If we do not disable LTR message forwarding here, or set a legal + * forwarding address, the kernel will get stuck after this driver probe. + * To workaround, disable the LTR message forwarding support on + * PCIe Message Reception. + */ + plda_pcie_disable_ltr(plda); + + /* Prefetchable memory window 64-bit addressing support */ + plda_pcie_set_pref_win_64bit(plda); + starfive_pcie_atr_init(priv); + + dm_gpio_set_value(&priv->reset_gpio, 0); + /* Ensure that PERST in default at least 300 ms */ + mdelay(300); + + return 0; + +err_deassert_clk: + clk_disable_bulk(&priv->clks); + return ret; +} + +static int starfive_pcie_probe(struct udevice *dev) +{ + struct starfive_pcie *priv = dev_get_priv(dev); + int ret; + + priv->plda.atr_table_num = 0; + priv->plda.dev = dev; + + ret = starfive_pcie_parse_dt(dev); + if (ret) + return ret; + + regmap_update_bits(priv->regmap, + priv->stg_rp_nep, + STG_SYSCON_K_RP_NEP_MASK, + STG_SYSCON_K_RP_NEP_MASK); + + regmap_update_bits(priv->regmap, + priv->stg_awfun, + STG_SYSCON_CKREF_SRC_MASK, + 2 << STG_SYSCON_CKREF_SRC_SHIFT); + + regmap_update_bits(priv->regmap, + priv->stg_awfun, + STG_SYSCON_CLKREQ_MASK, + STG_SYSCON_CLKREQ_MASK); + + ret = starfive_pcie_init_port(dev); + if (ret) + return ret; + + dev_err(dev, "Starfive PCIe bus probed.\n"); + + return 0; +} + +static const struct dm_pci_ops starfive_pcie_ops = { + .read_config = plda_pcie_config_read, + .write_config = plda_pcie_config_write, +}; + +static const struct udevice_id starfive_pcie_ids[] = { + { .compatible = "starfive,jh7110-pcie" }, + { } +}; + +U_BOOT_DRIVER(starfive_pcie_drv) = { + .name = "starfive_7110_pcie", + .id = UCLASS_PCI, + .of_match = starfive_pcie_ids, + .ops = &starfive_pcie_ops, + .probe = starfive_pcie_probe, + .priv_auto = sizeof(struct starfive_pcie), +};

From: Mason Huo mason.huo@starfivetech.com
also add the nvme driver and rtl8169 support.
Signed-off-by: Mason Huo mason.huo@starfivetech.com Signed-off-by: Minda Chen minda.chen@starfivetech.com --- configs/starfive_visionfive2_defconfig | 10 ++++++++++ 1 file changed, 10 insertions(+)
diff --git a/configs/starfive_visionfive2_defconfig b/configs/starfive_visionfive2_defconfig index 550d0ff3ab..ea798919be 100644 --- a/configs/starfive_visionfive2_defconfig +++ b/configs/starfive_visionfive2_defconfig @@ -16,6 +16,7 @@ CONFIG_SPL=y CONFIG_SPL_SPI_FLASH_SUPPORT=y CONFIG_SPL_SPI=y CONFIG_SYS_LOAD_ADDR=0x82000000 +CONFIG_SYS_PCI_64BIT=y CONFIG_TARGET_STARFIVE_VISIONFIVE2=y CONFIG_SPL_OPENSBI_LOAD_ADDR=0x40000000 CONFIG_ARCH_RV64I=y @@ -49,9 +50,12 @@ CONFIG_SPL_SPI_LOAD=y CONFIG_SYS_CBSIZE=256 CONFIG_SYS_PBSIZE=276 CONFIG_SYS_BOOTM_LEN=0x4000000 +CONFIG_CMD_PCI=y CONFIG_CMD_MEMINFO=y CONFIG_CMD_TFTPPUT=y CONFIG_SYS_RELOC_GD_ENV_ADDR=y +CONFIG_REGMAP=y +CONFIG_SYSCON=y CONFIG_SPL_CLK_COMPOSITE_CCF=y CONFIG_CLK_COMPOSITE_CCF=y CONFIG_SPL_CLK_JH7110=y @@ -65,6 +69,12 @@ CONFIG_SPI_FLASH_EON=y CONFIG_SPI_FLASH_GIGADEVICE=y CONFIG_SPI_FLASH_ISSI=y CONFIG_SPI_FLASH_MACRONIX=y +CONFIG_RTL8169=y +CONFIG_NVME_PCI=y +CONFIG_PCI=y +CONFIG_DM_PCI_COMPAT=y +CONFIG_PCI_REGION_MULTI_ENTRY=y +CONFIG_PCIE_STARFIVE_JH7110=y CONFIG_PINCTRL=y CONFIG_PINCONF=y CONFIG_SPL_PINCTRL=y

From: Mason Huo mason.huo@starfivetech.com
Enable and add pinctrl configuration for PCIe host controller. Also add JH7110 stg syscon configuration.
Signed-off-by: Mason Huo mason.huo@starfivetech.com Signed-off-by: Minda Chen minda.chen@starfivetech.com --- .../dts/jh7110-starfive-visionfive-2.dtsi | 11 +++ arch/riscv/dts/jh7110.dtsi | 74 +++++++++++++++++++ 2 files changed, 85 insertions(+)
diff --git a/arch/riscv/dts/jh7110-starfive-visionfive-2.dtsi b/arch/riscv/dts/jh7110-starfive-visionfive-2.dtsi index c6b6dfa940..12245576ac 100644 --- a/arch/riscv/dts/jh7110-starfive-visionfive-2.dtsi +++ b/arch/riscv/dts/jh7110-starfive-visionfive-2.dtsi @@ -7,6 +7,7 @@
#include "jh7110.dtsi" #include <dt-bindings/pinctrl/pinctrl-starfive-jh7110.h> +#include <dt-bindings/gpio/gpio.h> / { aliases { serial0 = &uart0; @@ -300,6 +301,16 @@ }; };
+&pcie0 { + reset-gpios = <&sysgpio 26 GPIO_ACTIVE_LOW>; + status = "disabled"; +}; + +&pcie1 { + reset-gpios = <&sysgpio 28 GPIO_ACTIVE_LOW>; + status = "okay"; +}; + &syscrg { assigned-clocks = <&syscrg JH7110_SYSCLK_CPU_ROOT>, <&syscrg JH7110_SYSCLK_BUS_ROOT>, diff --git a/arch/riscv/dts/jh7110.dtsi b/arch/riscv/dts/jh7110.dtsi index bd60879615..c7b071e743 100644 --- a/arch/riscv/dts/jh7110.dtsi +++ b/arch/riscv/dts/jh7110.dtsi @@ -569,5 +569,79 @@ gpio-controller; #gpio-cells = <2>; }; + + pcie0: pcie@2b000000 { + compatible = "starfive,jh7110-pcie"; + reg = <0x0 0x2b000000 0x0 0x1000000 + 0x9 0x40000000 0x0 0x10000000>; + reg-names = "reg", "config"; + #address-cells = <3>; + #size-cells = <2>; + #interrupt-cells = <1>; + ranges = <0x82000000 0x0 0x30000000 0x0 0x30000000 0x0 0x08000000>, + <0xc3000000 0x9 0x00000000 0x9 0x00000000 0x0 0x40000000>; + interrupts = <56>; + interrupt-parent = <&plic>; + interrupt-map-mask = <0x0 0x0 0x0 0x7>; + interrupt-map = <0x0 0x0 0x0 0x1 &plic 0x1>, + <0x0 0x0 0x0 0x2 &plic 0x2>, + <0x0 0x0 0x0 0x3 &plic 0x3>, + <0x0 0x0 0x0 0x4 &plic 0x4>; + msi-parent = <&plic>; + device_type = "pci"; + starfive,stg-syscon = <&stg_syscon 0xc0 0xc4 0x130 0x1b8>; + bus-range = <0x0 0xff>; + clocks = <&syscrg JH7110_SYSCLK_NOC_BUS_STG_AXI>, + <&stgcrg JH7110_STGCLK_PCIE0_TL>, + <&stgcrg JH7110_STGCLK_PCIE0_AXI>, + <&stgcrg JH7110_STGCLK_PCIE0_APB>; + clock-names = "noc", "tl", "axi", "apb"; + resets = <&stgcrg JH7110_STGRST_PCIE0_MST0>, + <&stgcrg JH7110_STGRST_PCIE0_SLV0>, + <&stgcrg JH7110_STGRST_PCIE0_SLV>, + <&stgcrg JH7110_STGRST_PCIE0_BRG>, + <&stgcrg JH7110_STGRST_PCIE0_CORE>, + <&stgcrg JH7110_STGRST_PCIE0_APB>; + reset-names = "mst0", "slv0", "slv", "brg", + "core", "apb"; + status = "disabled"; + }; + + pcie1: pcie@2c000000 { + compatible = "starfive,jh7110-pcie"; + reg = <0x0 0x2c000000 0x0 0x1000000 + 0x9 0xc0000000 0x0 0x10000000>; + reg-names = "reg", "config"; + #address-cells = <3>; + #size-cells = <2>; + #interrupt-cells = <1>; + ranges = <0x82000000 0x0 0x38000000 0x0 0x38000000 0x0 0x08000000>, + <0xc3000000 0x9 0x80000000 0x9 0x80000000 0x0 0x40000000>; + interrupts = <57>; + interrupt-parent = <&plic>; + interrupt-map-mask = <0x0 0x0 0x0 0x7>; + interrupt-map = <0x0 0x0 0x0 0x1 &plic 0x1>, + <0x0 0x0 0x0 0x2 &plic 0x2>, + <0x0 0x0 0x0 0x3 &plic 0x3>, + <0x0 0x0 0x0 0x4 &plic 0x4>; + msi-parent = <&plic>; + device_type = "pci"; + starfive,stg-syscon = <&stg_syscon 0x270 0x274 0x2e0 0x368>; + bus-range = <0x0 0xff>; + clocks = <&syscrg JH7110_SYSCLK_NOC_BUS_STG_AXI>, + <&stgcrg JH7110_STGCLK_PCIE1_TL>, + <&stgcrg JH7110_STGCLK_PCIE1_AXI>, + <&stgcrg JH7110_STGCLK_PCIE1_APB>; + clock-names = "noc", "tl", "axi", "apb"; + resets = <&stgcrg JH7110_STGRST_PCIE1_MST0>, + <&stgcrg JH7110_STGRST_PCIE1_SLV0>, + <&stgcrg JH7110_STGRST_PCIE1_SLV>, + <&stgcrg JH7110_STGRST_PCIE1_BRG>, + <&stgcrg JH7110_STGRST_PCIE1_CORE>, + <&stgcrg JH7110_STGRST_PCIE1_APB>; + reset-names = "mst0", "slv0", "slv", "brg", + "core", "apb"; + status = "disabled"; + }; }; };

Hi Rick and Leo Could you review these series patches? Thanks. Since patch 1 was acked by Pali, Does it mean patch 1 is accepted?
On 2023/5/9 16:26, Minda Chen wrote:
These PCIe series patches are based on the JH7110 RISC-V SoC and VisionFive V2 board.
The PCIe driver depends on gpio, pinctrl, clk and reset driver to do init. The PCIe dts configuation includes all these setting.
The PCIe drivers codes has been tested on the VisionFive V2 boards. The test devices includes M.2 NVMe SSD and Realtek 8169 Ethernet adapter.
previous patch version
v1: https://patchwork.ozlabs.org/project/uboot/cover/20230223105240.15180-1-mind... v2: https://patchwork.ozlabs.org/project/uboot/cover/20230308054833.95730-1-mind... v3: https://patchwork.ozlabs.org/project/uboot/cover/20230329100143.10724-1-mind... v4: https://patchwork.ozlabs.org/project/uboot/cover/20230411010209.76561-1-mind... v5: https://patchwork.ozlabs.org/project/uboot/cover/20230423105859.125764-1-min...
changes v6 patch 2 1. Change one VisionFive2 config file. patch 3 1. remove the pcie interrupt-controller. interrupts setting revert to origin.
setting. v5 patch 1 1. split PLDA controller driver codes to pcie_plda_common.c 2. correct the codes of record secondary number. patch 3 1. change the pcie dtsi config. make them the same with kernel.
v4 patch 1 1. Remove the IDS_REVISION_ID macros. 2. Replace sec_busno to first_busno in starfive_pcie 3. Remove starfive_pcie_off_conf function. 4. Replace "imply" to "depends on" in PCIe Kconfig. 5 .Check sec_busno in starfive_pcie_addr_valid.
v3 patch 1 1. remove the read vendor ID delay 2. remove starfive_pcie_hide_rc_bar function. do not hide host bridge BAR write. 3. Using PCIE_ECAM_OFFSET and PCI_CLASS_BRIDGE_PCI_NORMAL macros. 4. Add comments for bus and address limitation reason in function starfive_pcie_addr_valid 5. Change the multiple line comments in Line 373 6. Using gpio_request_by_name to get PCIe reset gpio,and using dm_gpio_set_value set GPIO value. patch 2 1. support PCIeboth 12a and 13b vf2 board. patch 3 1. reset dts change to reset-gpio.
v2
- remove clock commit. The pcie clocks change has been includeded in [1].
- Using GENMASK marco1 in patch1.
- remove the syscon dts node in patch3. The syscon dts dts node has been included in [1].
Mason Huo (3): starfive: pci: Add StarFive JH7110 pcie driver configs: starfive-jh7110: Add support for PCIe host driver riscv: dts: starfive: Enable PCIe host controller
.../dts/jh7110-starfive-visionfive-2.dtsi | 11 + arch/riscv/dts/jh7110.dtsi | 74 ++++ configs/starfive_visionfive2_defconfig | 10 + drivers/pci/Kconfig | 13 + drivers/pci/Makefile | 2 + drivers/pci/pcie_plda_common.c | 116 +++++++ drivers/pci/pcie_plda_common.h | 118 +++++++ drivers/pci/pcie_starfive_jh7110.c | 317 ++++++++++++++++++ 8 files changed, 661 insertions(+) create mode 100644 drivers/pci/pcie_plda_common.c create mode 100644 drivers/pci/pcie_plda_common.h create mode 100644 drivers/pci/pcie_starfive_jh7110.c
base-commit: c9c2c95d4cd27fe0cd41fe13a863899d268f973c

Hi Minda,
On Thu, Jun 01, 2023 at 09:07:14AM +0800, Minda Chen wrote:
Hi Rick and Leo Could you review these series patches? Thanks. Since patch 1 was acked by Pali, Does it mean patch 1 is accepted?
I think acked-by means that the patch is considered good enough to be merged.
Aside from that, your patchset caused CI error by enabling the RTL8169 driver. Could you take a look at it ? (https://source.denx.de/u-boot/custodians/u-boot-riscv/-/jobs/638021) It's mostly compilation warnings that are being treated as errors. The patchset looks good to me, I could merge the patchset as soon as the CI errors are fixed!
Thanks & best regards, Leo
On 2023/5/9 16:26, Minda Chen wrote:
These PCIe series patches are based on the JH7110 RISC-V SoC and VisionFive V2 board.
The PCIe driver depends on gpio, pinctrl, clk and reset driver to do init. The PCIe dts configuation includes all these setting.
The PCIe drivers codes has been tested on the VisionFive V2 boards. The test devices includes M.2 NVMe SSD and Realtek 8169 Ethernet adapter.
previous patch version
v1: https://patchwork.ozlabs.org/project/uboot/cover/20230223105240.15180-1-mind... v2: https://patchwork.ozlabs.org/project/uboot/cover/20230308054833.95730-1-mind... v3: https://patchwork.ozlabs.org/project/uboot/cover/20230329100143.10724-1-mind... v4: https://patchwork.ozlabs.org/project/uboot/cover/20230411010209.76561-1-mind... v5: https://patchwork.ozlabs.org/project/uboot/cover/20230423105859.125764-1-min...
changes v6 patch 2 1. Change one VisionFive2 config file. patch 3 1. remove the pcie interrupt-controller. interrupts setting revert to origin.
setting. v5 patch 1 1. split PLDA controller driver codes to pcie_plda_common.c 2. correct the codes of record secondary number. patch 3 1. change the pcie dtsi config. make them the same with kernel.
v4 patch 1 1. Remove the IDS_REVISION_ID macros. 2. Replace sec_busno to first_busno in starfive_pcie 3. Remove starfive_pcie_off_conf function. 4. Replace "imply" to "depends on" in PCIe Kconfig. 5 .Check sec_busno in starfive_pcie_addr_valid.
v3 patch 1 1. remove the read vendor ID delay 2. remove starfive_pcie_hide_rc_bar function. do not hide host bridge BAR write. 3. Using PCIE_ECAM_OFFSET and PCI_CLASS_BRIDGE_PCI_NORMAL macros. 4. Add comments for bus and address limitation reason in function starfive_pcie_addr_valid 5. Change the multiple line comments in Line 373 6. Using gpio_request_by_name to get PCIe reset gpio,and using dm_gpio_set_value set GPIO value. patch 2 1. support PCIeboth 12a and 13b vf2 board. patch 3 1. reset dts change to reset-gpio.
v2
- remove clock commit. The pcie clocks change has been includeded in [1].
- Using GENMASK marco1 in patch1.
- remove the syscon dts node in patch3. The syscon dts dts node has been included in [1].
Mason Huo (3): starfive: pci: Add StarFive JH7110 pcie driver configs: starfive-jh7110: Add support for PCIe host driver riscv: dts: starfive: Enable PCIe host controller
.../dts/jh7110-starfive-visionfive-2.dtsi | 11 + arch/riscv/dts/jh7110.dtsi | 74 ++++ configs/starfive_visionfive2_defconfig | 10 + drivers/pci/Kconfig | 13 + drivers/pci/Makefile | 2 + drivers/pci/pcie_plda_common.c | 116 +++++++ drivers/pci/pcie_plda_common.h | 118 +++++++ drivers/pci/pcie_starfive_jh7110.c | 317 ++++++++++++++++++ 8 files changed, 661 insertions(+) create mode 100644 drivers/pci/pcie_plda_common.c create mode 100644 drivers/pci/pcie_plda_common.h create mode 100644 drivers/pci/pcie_starfive_jh7110.c
base-commit: c9c2c95d4cd27fe0cd41fe13a863899d268f973c

On 2023/6/16 16:42, Leo Liang wrote:
Hi Minda,
On Thu, Jun 01, 2023 at 09:07:14AM +0800, Minda Chen wrote:
Hi Rick and Leo Could you review these series patches? Thanks. Since patch 1 was acked by Pali, Does it mean patch 1 is accepted?
I think acked-by means that the patch is considered good enough to be merged.
Aside from that, your patchset caused CI error by enabling the RTL8169 driver. Could you take a look at it ? (https://source.denx.de/u-boot/custodians/u-boot-riscv/-/jobs/638021) It's mostly compilation warnings that are being treated as errors. The patchset looks good to me, I could merge the patchset as soon as the CI errors are fixed!
Thanks & best regards, Leo
Thanks. I will commit another patch to fix the warning.
On 2023/5/9 16:26, Minda Chen wrote:
These PCIe series patches are based on the JH7110 RISC-V SoC and VisionFive V2 board.
The PCIe driver depends on gpio, pinctrl, clk and reset driver to do init. The PCIe dts configuation includes all these setting.
The PCIe drivers codes has been tested on the VisionFive V2 boards. The test devices includes M.2 NVMe SSD and Realtek 8169 Ethernet adapter.
previous patch version
v1: https://patchwork.ozlabs.org/project/uboot/cover/20230223105240.15180-1-mind... v2: https://patchwork.ozlabs.org/project/uboot/cover/20230308054833.95730-1-mind... v3: https://patchwork.ozlabs.org/project/uboot/cover/20230329100143.10724-1-mind... v4: https://patchwork.ozlabs.org/project/uboot/cover/20230411010209.76561-1-mind... v5: https://patchwork.ozlabs.org/project/uboot/cover/20230423105859.125764-1-min...
changes v6 patch 2 1. Change one VisionFive2 config file. patch 3 1. remove the pcie interrupt-controller. interrupts setting revert to origin.
setting. v5 patch 1 1. split PLDA controller driver codes to pcie_plda_common.c 2. correct the codes of record secondary number. patch 3 1. change the pcie dtsi config. make them the same with kernel.
v4 patch 1 1. Remove the IDS_REVISION_ID macros. 2. Replace sec_busno to first_busno in starfive_pcie 3. Remove starfive_pcie_off_conf function. 4. Replace "imply" to "depends on" in PCIe Kconfig. 5 .Check sec_busno in starfive_pcie_addr_valid.
v3 patch 1 1. remove the read vendor ID delay 2. remove starfive_pcie_hide_rc_bar function. do not hide host bridge BAR write. 3. Using PCIE_ECAM_OFFSET and PCI_CLASS_BRIDGE_PCI_NORMAL macros. 4. Add comments for bus and address limitation reason in function starfive_pcie_addr_valid 5. Change the multiple line comments in Line 373 6. Using gpio_request_by_name to get PCIe reset gpio,and using dm_gpio_set_value set GPIO value. patch 2 1. support PCIeboth 12a and 13b vf2 board. patch 3 1. reset dts change to reset-gpio.
v2
- remove clock commit. The pcie clocks change has been includeded in [1].
- Using GENMASK marco1 in patch1.
- remove the syscon dts node in patch3. The syscon dts dts node has been included in [1].
Mason Huo (3): starfive: pci: Add StarFive JH7110 pcie driver configs: starfive-jh7110: Add support for PCIe host driver riscv: dts: starfive: Enable PCIe host controller
.../dts/jh7110-starfive-visionfive-2.dtsi | 11 + arch/riscv/dts/jh7110.dtsi | 74 ++++ configs/starfive_visionfive2_defconfig | 10 + drivers/pci/Kconfig | 13 + drivers/pci/Makefile | 2 + drivers/pci/pcie_plda_common.c | 116 +++++++ drivers/pci/pcie_plda_common.h | 118 +++++++ drivers/pci/pcie_starfive_jh7110.c | 317 ++++++++++++++++++ 8 files changed, 661 insertions(+) create mode 100644 drivers/pci/pcie_plda_common.c create mode 100644 drivers/pci/pcie_plda_common.h create mode 100644 drivers/pci/pcie_starfive_jh7110.c
base-commit: c9c2c95d4cd27fe0cd41fe13a863899d268f973c
participants (2)
-
Leo Liang
-
Minda Chen