[RFC PATCH v4 0/2] arch: riscv: cpu: Add callback to init each core
Add a callback interface, riscv_hart_early_init() to perform riscv CPU hart-dependent configuration for each hart. riscv_hart_early_init() is placed after stack of harts are initialized and before main boot hart is picked up. Several conditions should be aware of or avoided are listed:
- cannot access gd At the moment, gd hasn't initialized yet.
- all operations in riscv_hart_early_init() should only affect core itself For example, the operations for board level should be moved to mach_xxx() or board_init_xxx() functions instead.
- Common resource need protection if multicore Since all harts might enter riscv_hart_early_init() in parallel, common resource need a mechanism to handle race condition.
- A dummy implementation is added in ./arch/riscv/cpu/cpu.c The implementation is declared with '__weak' and can be overridden.
Changelogs: v4 - Revising the comments for riscv_hart_early_init() in [1/2] - Remove unnecessary braces and add reviewed-by in [2/2]
Green Wan (2): arch: riscv: cpu: Add callback to init each core board: sifive: unmatched: clear feature disable CSR
arch/riscv/cpu/cpu.c | 15 +++++++++++++++ arch/riscv/cpu/start.S | 14 ++++++++++++++ board/sifive/unmatched/spl.c | 15 +++++++++++++++ 3 files changed, 44 insertions(+)
Add a callback riscv_hart_early_init() to ./arch/riscv/cpu/start.S to allow different riscv hart perform setup code for each hart as early as possible. Since all the harts enter the callback, they must be able to run the same setup.
Signed-off-by: Green Wan green.wan@sifive.com --- arch/riscv/cpu/cpu.c | 15 +++++++++++++++ arch/riscv/cpu/start.S | 14 ++++++++++++++ 2 files changed, 29 insertions(+)
diff --git a/arch/riscv/cpu/cpu.c b/arch/riscv/cpu/cpu.c index 85592f5bee..1652e51137 100644 --- a/arch/riscv/cpu/cpu.c +++ b/arch/riscv/cpu/cpu.c @@ -140,3 +140,18 @@ int arch_early_init_r(void) { return riscv_cpu_probe(); } + +/** + * riscv_hart_early_init() - A dummy function called by + * ./arch/riscv/cpu/start.S to allow to disable/enable features of each core. + * For example, turn on or off the functional block of CPU harts. + * + * In a multi-core system, this function must not access shared resources. + * + * Any access to such resources would probably be better done with + * available_harts_lock held. However, I doubt that any such access will be + * necessary. + */ +__weak void riscv_hart_early_init(void) +{ +} diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S index 8589509e01..ab73008f23 100644 --- a/arch/riscv/cpu/start.S +++ b/arch/riscv/cpu/start.S @@ -117,6 +117,20 @@ call_board_init_f_0: mv sp, a0 #endif
+#if CONFIG_IS_ENABLED(RISCV_MMODE) + /* + * Jump to riscv_hart_early_init() to perform init for each core. Not + * expect to access gd since gd is not initialized. All operations in the + * function should affect core itself only. In multi-core system, any access + * to common resource or registers outside core should be avoided or need a + * protection for multicore. + * + * A dummy implementation is provided in ./arch/riscv/cpu/cpu.c. + */ +call_riscv_hart_early_init: + jal riscv_hart_early_init +#endif + #ifndef CONFIG_XIP /* * Pick hart to initialize global data and run U-Boot. The other harts
On 3/30/21 1:26 AM, Green Wan wrote:
Add a callback riscv_hart_early_init() to ./arch/riscv/cpu/start.S to allow different riscv hart perform setup code for each hart as early as possible. Since all the harts enter the callback, they must be able to run the same setup.
Signed-off-by: Green Wan green.wan@sifive.com
arch/riscv/cpu/cpu.c | 15 +++++++++++++++ arch/riscv/cpu/start.S | 14 ++++++++++++++ 2 files changed, 29 insertions(+)
diff --git a/arch/riscv/cpu/cpu.c b/arch/riscv/cpu/cpu.c index 85592f5bee..1652e51137 100644 --- a/arch/riscv/cpu/cpu.c +++ b/arch/riscv/cpu/cpu.c @@ -140,3 +140,18 @@ int arch_early_init_r(void) { return riscv_cpu_probe(); }
+/**
- riscv_hart_early_init() - A dummy function called by
- ./arch/riscv/cpu/start.S to allow to disable/enable features of each core.
- For example, turn on or off the functional block of CPU harts.
- In a multi-core system, this function must not access shared resources.
- Any access to such resources would probably be better done with
- available_harts_lock held. However, I doubt that any such access will be
- necessary.
- */
+__weak void riscv_hart_early_init(void) +{ +} diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S index 8589509e01..ab73008f23 100644 --- a/arch/riscv/cpu/start.S +++ b/arch/riscv/cpu/start.S @@ -117,6 +117,20 @@ call_board_init_f_0: mv sp, a0 #endif
+#if CONFIG_IS_ENABLED(RISCV_MMODE)
- /*
* Jump to riscv_hart_early_init() to perform init for each core. Not* expect to access gd since gd is not initialized. All operations in the* function should affect core itself only. In multi-core system, any access* to common resource or registers outside core should be avoided or need a* protection for multicore.** A dummy implementation is provided in ./arch/riscv/cpu/cpu.c.*/+call_riscv_hart_early_init:
- jal riscv_hart_early_init
+#endif
I wonder if we could move the calls to icache_enable and dcache_enable into this function. Though this would have the consequence of enabling caches on all harts for CPUs which previously only enabled them for the boot hart. I think ax25 is the only CPU which currently does this. Bin, would this be an issue?
--Sean
#ifndef CONFIG_XIP /* * Pick hart to initialize global data and run U-Boot. The other harts
On Sat, Apr 3, 2021 at 6:53 AM Sean Anderson seanga2@gmail.com wrote:
On 3/30/21 1:26 AM, Green Wan wrote:
Add a callback riscv_hart_early_init() to ./arch/riscv/cpu/start.S to allow different riscv hart perform setup code for each hart as early as possible. Since all the harts enter the callback, they must be able to run the same setup.
Signed-off-by: Green Wan green.wan@sifive.com
arch/riscv/cpu/cpu.c | 15 +++++++++++++++ arch/riscv/cpu/start.S | 14 ++++++++++++++ 2 files changed, 29 insertions(+)
diff --git a/arch/riscv/cpu/cpu.c b/arch/riscv/cpu/cpu.c index 85592f5bee..1652e51137 100644 --- a/arch/riscv/cpu/cpu.c +++ b/arch/riscv/cpu/cpu.c @@ -140,3 +140,18 @@ int arch_early_init_r(void) { return riscv_cpu_probe(); }
+/**
- riscv_hart_early_init() - A dummy function called by
- ./arch/riscv/cpu/start.S to allow to disable/enable features of each core.
- For example, turn on or off the functional block of CPU harts.
- In a multi-core system, this function must not access shared resources.
- Any access to such resources would probably be better done with
- available_harts_lock held. However, I doubt that any such access will be
- necessary.
- */
+__weak void riscv_hart_early_init(void) +{ +} diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S index 8589509e01..ab73008f23 100644 --- a/arch/riscv/cpu/start.S +++ b/arch/riscv/cpu/start.S @@ -117,6 +117,20 @@ call_board_init_f_0: mv sp, a0 #endif
+#if CONFIG_IS_ENABLED(RISCV_MMODE)
/** Jump to riscv_hart_early_init() to perform init for each core. Not* expect to access gd since gd is not initialized. All operations in the* function should affect core itself only. In multi-core system, any access* to common resource or registers outside core should be avoided or need a* protection for multicore.** A dummy implementation is provided in ./arch/riscv/cpu/cpu.c.*/+call_riscv_hart_early_init:
jal riscv_hart_early_init+#endif
I wonder if we could move the calls to icache_enable and dcache_enable into this function. Though this would have the consequence of enabling caches on all harts for CPUs which previously only enabled them for the boot hart. I think ax25 is the only CPU which currently does this. Bin, would this be an issue?
Good catch. I believe AX25 cache support is currently somehow broken?
I think Rick has to comment on this as he added this via commit:
commit 52923c6db7f00e0197ec894c8c1bb8a7681974bb Author: Rick Chen rick@andestech.com Date: Wed Nov 7 09:34:06 2018 +0800
riscv: cache: Implement i/dcache [status, enable, disable]
AndeStar RISC-V(V5) provide mcache_ctl register which can configure I/D cache as enabled or disabled.
This CSR will be encapsulated by CONFIG_RISCV_NDS. If you want to configure cache on AndeStar V5 AE350 platform. YOu can enable [*] AndeStar V5 ISA support by make menuconfig.
This approach also provide the expansion when the vender specific features are going to join in.
Signed-off-by: Rick Chen rick@andestech.com Cc: Greentime Hu greentime@andestech.com
The original commit has i/d cache enabled on all harts. But it was later moved to boot hart due to SMP support.
However on the cleanup side, it looks only the boot hart calls i/d cache disable?
See arch/riscv/cpu/ax25/cpu.c:: cleanup_before_linux()
Regards, Bin
Clear feature disable CSR to turn on all features of hart. The detail is specified at section, 'SiFive Feature Disable CSR', in user manual
https://sifive.cdn.prismic.io/sifive/aee0dd4c-d156-496e-a6c4-db0cf54bbe68_si...
Signed-off-by: Green Wan green.wan@sifive.com Reviewed-by: Sean Anderson seanga2@gmail.com --- board/sifive/unmatched/spl.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
diff --git a/board/sifive/unmatched/spl.c b/board/sifive/unmatched/spl.c index 5e1333b09a..2af3069b55 100644 --- a/board/sifive/unmatched/spl.c +++ b/board/sifive/unmatched/spl.c @@ -12,6 +12,7 @@ #include <log.h> #include <linux/delay.h> #include <linux/io.h> +#include <asm/csr.h> #include <asm/gpio.h> #include <asm/arch/gpio.h> #include <asm/arch/spl.h> @@ -22,6 +23,20 @@ #define MODE_SELECT_SD 0xb #define MODE_SELECT_MASK GENMASK(3, 0)
+#define CSR_U74_FEATURE_DISABLE 0x7c1 + +void riscv_hart_early_init(void) +{ + /* + * Feature Disable CSR + * + * Clear feature disable CSR to '0' to turn on all features for + * each core. This operation must be in m-mode. + */ + if (CONFIG_IS_ENABLED(RISCV_MMODE)) + csr_write(CSR_U74_FEATURE_DISABLE, 0); +} + int spl_board_init_f(void) { int ret;
On Tue, Mar 30, 2021 at 1:27 PM Green Wan green.wan@sifive.com wrote:
Clear feature disable CSR to turn on all features of hart. The detail is specified at section, 'SiFive Feature Disable CSR', in user manual
https://sifive.cdn.prismic.io/sifive/aee0dd4c-d156-496e-a6c4-db0cf54bbe68_si...
Signed-off-by: Green Wan green.wan@sifive.com Reviewed-by: Sean Anderson seanga2@gmail.com
board/sifive/unmatched/spl.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
diff --git a/board/sifive/unmatched/spl.c b/board/sifive/unmatched/spl.c index 5e1333b09a..2af3069b55 100644 --- a/board/sifive/unmatched/spl.c +++ b/board/sifive/unmatched/spl.c @@ -12,6 +12,7 @@ #include <log.h> #include <linux/delay.h> #include <linux/io.h> +#include <asm/csr.h> #include <asm/gpio.h> #include <asm/arch/gpio.h> #include <asm/arch/spl.h> @@ -22,6 +23,20 @@ #define MODE_SELECT_SD 0xb #define MODE_SELECT_MASK GENMASK(3, 0)
+#define CSR_U74_FEATURE_DISABLE 0x7c1
+void riscv_hart_early_init(void) +{
/** Feature Disable CSR** Clear feature disable CSR to '0' to turn on all features for* each core. This operation must be in m-mode.
nits: M-mode
*/if (CONFIG_IS_ENABLED(RISCV_MMODE))csr_write(CSR_U74_FEATURE_DISABLE, 0);+}
Otherwise, Reviewed-by: Bin Meng bmeng.cn@gmail.com
k, thanks, will fix it.
On Tue, Apr 6, 2021 at 5:17 PM Bin Meng bmeng.cn@gmail.com wrote:
On Tue, Mar 30, 2021 at 1:27 PM Green Wan green.wan@sifive.com wrote:
Clear feature disable CSR to turn on all features of hart. The detail is specified at section, 'SiFive Feature Disable CSR', in user manual
https://sifive.cdn.prismic.io/sifive/aee0dd4c-d156-496e-a6c4-db0cf54bbe68_si...
Signed-off-by: Green Wan green.wan@sifive.com Reviewed-by: Sean Anderson seanga2@gmail.com
board/sifive/unmatched/spl.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
diff --git a/board/sifive/unmatched/spl.c b/board/sifive/unmatched/spl.c index 5e1333b09a..2af3069b55 100644 --- a/board/sifive/unmatched/spl.c +++ b/board/sifive/unmatched/spl.c @@ -12,6 +12,7 @@ #include <log.h> #include <linux/delay.h> #include <linux/io.h> +#include <asm/csr.h> #include <asm/gpio.h> #include <asm/arch/gpio.h> #include <asm/arch/spl.h> @@ -22,6 +23,20 @@ #define MODE_SELECT_SD 0xb #define MODE_SELECT_MASK GENMASK(3, 0)
+#define CSR_U74_FEATURE_DISABLE 0x7c1
+void riscv_hart_early_init(void) +{
/** Feature Disable CSR** Clear feature disable CSR to '0' to turn on all features for* each core. This operation must be in m-mode.nits: M-mode
*/if (CONFIG_IS_ENABLED(RISCV_MMODE))csr_write(CSR_U74_FEATURE_DISABLE, 0);+}
Otherwise, Reviewed-by: Bin Meng bmeng.cn@gmail.com
participants (3)
-
Bin Meng -
Green Wan -
Sean Anderson