[U-Boot] [PATCH v6 00/15] Add PSCI support for Jetson TK1/Tegra124 + CNTFRQ fix
Changes in v6: - rebased over master - included Thierry's SMMU enabling patch - moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
Jan
CC: Ian Campbell ijc@hellion.org.uk CC: Marc Zyngier marc.zyngier@arm.com CC: Thierry Reding treding@nvidia.com
Ian Campbell (3): tegra124: Add more registers to struct mc_ctlr tegra124: Reserve secure RAM using MC_SECURITY_CFG{0, 1}_0 jetson-tk1: Add PSCI configuration options and reserve secure code
Jan Kiszka (11): sun7i: Remove duplicate call to psci_arch_init ARM: Factor out common psci_get_cpu_id ARM: Factor out reusable psci_cpu_off_common ARM: Factor out reusable psci_cpu_entry ARM: Factor out reusable psci_get_cpu_stack_top ARM: Put target PC for PSCI CPU_ON on per-CPU stack virt-dt: Allow reservation of secure region when in a RAM carveout tegra: Make tegra_powergate_power_on public tegra: Add ap_pm_init hook tegra124: Add PSCI support for Tegra124 tegra: Set CNTFRQ for secondary CPUs
Thierry Reding (1): ARM: tegra: Enable SMMU when going non-secure
arch/arm/cpu/armv7/psci.S | 121 ++++++++++++++++++++++++++++ arch/arm/cpu/armv7/sunxi/psci.S | 112 ++++--------------------- arch/arm/cpu/armv7/virt-dt.c | 29 +++++++ arch/arm/cpu/armv7/virt-v7.c | 5 ++ arch/arm/include/asm/arch-tegra/ap.h | 5 ++ arch/arm/include/asm/arch-tegra/powergate.h | 1 + arch/arm/include/asm/arch-tegra124/flow.h | 6 ++ arch/arm/include/asm/arch-tegra124/mc.h | 37 ++++++++- arch/arm/include/asm/armv7.h | 1 + arch/arm/include/asm/system.h | 1 + arch/arm/lib/bootm-fdt.c | 6 ++ arch/arm/mach-tegra/Makefile | 4 + arch/arm/mach-tegra/ap.c | 55 +++++++++++++ arch/arm/mach-tegra/powergate.c | 2 +- arch/arm/mach-tegra/psci.S | 114 ++++++++++++++++++++++++++ arch/arm/mach-tegra/tegra124/Kconfig | 2 + arch/arm/mach-tegra/tegra124/Makefile | 4 + arch/arm/mach-tegra/tegra124/ap.c | 55 +++++++++++++ board/nvidia/common/board.c | 3 + include/configs/jetson-tk1.h | 5 ++ 20 files changed, 470 insertions(+), 98 deletions(-) create mode 100644 arch/arm/mach-tegra/psci.S create mode 100644 arch/arm/mach-tegra/tegra124/ap.c
This is already invoked a few cycles later in monitor mode by _secure_monitor (_sunxi_cpu_entry calls _do_nonsec_entry which triggers _secure_monitor via smc #0). Drop it here, it serves no purpose.
CC: Marc Zyngier marc.zyngier@arm.com Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk Tested-by: Hans de Goede hdegoede@redhat.com --- arch/arm/cpu/armv7/sunxi/psci.S | 1 - 1 file changed, 1 deletion(-)
diff --git a/arch/arm/cpu/armv7/sunxi/psci.S b/arch/arm/cpu/armv7/sunxi/psci.S index e0a524e..07b2d76 100644 --- a/arch/arm/cpu/armv7/sunxi/psci.S +++ b/arch/arm/cpu/armv7/sunxi/psci.S @@ -254,7 +254,6 @@ _sunxi_cpu_entry: isb
bl _nonsec_init - bl psci_arch_init
adr r0, _target_pc ldr r0, [r0]
Will be required for obtaining the ID of the current CPU in shared PSCI functions. The default implementation requires a dense ID space and only supports a single cluster. Therefore, the functions can be overloaded in cases where these assumptions do not hold.
CC: Marc Zyngier marc.zyngier@arm.com Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk Acked-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/cpu/armv7/psci.S | 8 ++++++++ arch/arm/cpu/armv7/sunxi/psci.S | 12 +++++++----- 2 files changed, 15 insertions(+), 5 deletions(-)
diff --git a/arch/arm/cpu/armv7/psci.S b/arch/arm/cpu/armv7/psci.S index bf11a34..12ad09b 100644 --- a/arch/arm/cpu/armv7/psci.S +++ b/arch/arm/cpu/armv7/psci.S @@ -99,4 +99,12 @@ _smc_psci: pop {r4-r7, lr} movs pc, lr @ Return to the kernel
+@ Requires dense and single-cluster CPU ID space +ENTRY(psci_get_cpu_id) + mrc p15, 0, r0, c0, c0, 5 /* read MPIDR */ + and r0, r0, #0xff /* return CPU ID in cluster */ + bx lr +ENDPROC(psci_get_cpu_id) +.weak psci_get_cpu_id + .popsection diff --git a/arch/arm/cpu/armv7/sunxi/psci.S b/arch/arm/cpu/armv7/sunxi/psci.S index 07b2d76..bcc419d 100644 --- a/arch/arm/cpu/armv7/sunxi/psci.S +++ b/arch/arm/cpu/armv7/sunxi/psci.S @@ -19,6 +19,7 @@
#include <config.h> #include <asm/gic.h> +#include <asm/macro.h> #include <asm/psci.h> #include <asm/arch/cpu.h>
@@ -290,6 +291,8 @@ psci_cpu_off:
.globl psci_arch_init psci_arch_init: + mov r6, lr + movw r4, #(GICD_BASE & 0xffff) movt r4, #(GICD_BASE >> 16)
@@ -315,18 +318,17 @@ psci_arch_init: mcr p15, 0, r5, c1, c1, 0 @ Write SCR isb
- mrc p15, 0, r4, c0, c0, 5 @ MPIDR - and r4, r4, #3 @ cpu number in cluster + bl psci_get_cpu_id mov r5, #0x400 @ 1kB of stack per CPU - mul r4, r4, r5 + mul r0, r0, r5
adr r5, text_end @ end of text add r5, r5, #0x2000 @ Skip two pages lsr r5, r5, #12 @ Align to start of page lsl r5, r5, #12 - sub sp, r5, r4 @ here's our stack! + sub sp, r5, r0 @ here's our stack!
- bx lr + bx r6
text_end: .popsection
Move parts of sunxi's psci_cpu_off into psci_cpu_off_common, namely cache disabling and flushing, clrex and the disabling of SMP for the dying CPU. These steps are apparently generic for ARMv7 and will be reused for Tegra124 support.
As the way of disabled SMP is not architectural, though commonly done via ACLTR, the related function can be overloaded.
CC: Marc Zyngier marc.zyngier@arm.com Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/cpu/armv7/psci.S | 77 +++++++++++++++++++++++++++++++++++++++++ arch/arm/cpu/armv7/sunxi/psci.S | 63 +-------------------------------- 2 files changed, 78 insertions(+), 62 deletions(-)
diff --git a/arch/arm/cpu/armv7/psci.S b/arch/arm/cpu/armv7/psci.S index 12ad09b..cdcdccd 100644 --- a/arch/arm/cpu/armv7/psci.S +++ b/arch/arm/cpu/armv7/psci.S @@ -107,4 +107,81 @@ ENTRY(psci_get_cpu_id) ENDPROC(psci_get_cpu_id) .weak psci_get_cpu_id
+/* Imported from Linux kernel */ +LENTRY(v7_flush_dcache_all) + dmb @ ensure ordering with previous memory accesses + mrc p15, 1, r0, c0, c0, 1 @ read clidr + ands r3, r0, #0x7000000 @ extract loc from clidr + mov r3, r3, lsr #23 @ left align loc bit field + beq finished @ if loc is 0, then no need to clean + mov r10, #0 @ start clean at cache level 0 +flush_levels: + add r2, r10, r10, lsr #1 @ work out 3x current cache level + mov r1, r0, lsr r2 @ extract cache type bits from clidr + and r1, r1, #7 @ mask of the bits for current cache only + cmp r1, #2 @ see what cache we have at this level + blt skip @ skip if no cache, or just i-cache + mrs r9, cpsr @ make cssr&csidr read atomic + mcr p15, 2, r10, c0, c0, 0 @ select current cache level in cssr + isb @ isb to sych the new cssr&csidr + mrc p15, 1, r1, c0, c0, 0 @ read the new csidr + msr cpsr_c, r9 + and r2, r1, #7 @ extract the length of the cache lines + add r2, r2, #4 @ add 4 (line length offset) + ldr r4, =0x3ff + ands r4, r4, r1, lsr #3 @ find maximum number on the way size + clz r5, r4 @ find bit position of way size increment + ldr r7, =0x7fff + ands r7, r7, r1, lsr #13 @ extract max number of the index size +loop1: + mov r9, r7 @ create working copy of max index +loop2: + orr r11, r10, r4, lsl r5 @ factor way and cache number into r11 + orr r11, r11, r9, lsl r2 @ factor index number into r11 + mcr p15, 0, r11, c7, c14, 2 @ clean & invalidate by set/way + subs r9, r9, #1 @ decrement the index + bge loop2 + subs r4, r4, #1 @ decrement the way + bge loop1 +skip: + add r10, r10, #2 @ increment cache number + cmp r3, r10 + bgt flush_levels +finished: + mov r10, #0 @ swith back to cache level 0 + mcr p15, 2, r10, c0, c0, 0 @ select current cache level in cssr + dsb st + isb + bx lr +ENDPROC(v7_flush_dcache_all) + +ENTRY(psci_disable_smp) + mrc p15, 0, r0, c1, c0, 1 @ ACTLR + bic r0, r0, #(1 << 6) @ Clear SMP bit + mcr p15, 0, r0, c1, c0, 1 @ ACTLR + isb + dsb + bx lr +ENDPROC(psci_disable_smp) +.weak psci_disable_smp + +ENTRY(psci_cpu_off_common) + push {lr} + + mrc p15, 0, r0, c1, c0, 0 @ SCTLR + bic r0, r0, #(1 << 2) @ Clear C bit + mcr p15, 0, r0, c1, c0, 0 @ SCTLR + isb + dsb + + bl v7_flush_dcache_all + + clrex @ Why??? + + bl psci_disable_smp + + pop {lr} + bx lr +ENDPROC(psci_cpu_off_common) + .popsection diff --git a/arch/arm/cpu/armv7/sunxi/psci.S b/arch/arm/cpu/armv7/sunxi/psci.S index bcc419d..05d047b 100644 --- a/arch/arm/cpu/armv7/sunxi/psci.S +++ b/arch/arm/cpu/armv7/sunxi/psci.S @@ -200,53 +200,6 @@ psci_cpu_on: _target_pc: .word 0
-/* Imported from Linux kernel */ -v7_flush_dcache_all: - dmb @ ensure ordering with previous memory accesses - mrc p15, 1, r0, c0, c0, 1 @ read clidr - ands r3, r0, #0x7000000 @ extract loc from clidr - mov r3, r3, lsr #23 @ left align loc bit field - beq finished @ if loc is 0, then no need to clean - mov r10, #0 @ start clean at cache level 0 -flush_levels: - add r2, r10, r10, lsr #1 @ work out 3x current cache level - mov r1, r0, lsr r2 @ extract cache type bits from clidr - and r1, r1, #7 @ mask of the bits for current cache only - cmp r1, #2 @ see what cache we have at this level - blt skip @ skip if no cache, or just i-cache - mrs r9, cpsr @ make cssr&csidr read atomic - mcr p15, 2, r10, c0, c0, 0 @ select current cache level in cssr - isb @ isb to sych the new cssr&csidr - mrc p15, 1, r1, c0, c0, 0 @ read the new csidr - msr cpsr_c, r9 - and r2, r1, #7 @ extract the length of the cache lines - add r2, r2, #4 @ add 4 (line length offset) - ldr r4, =0x3ff - ands r4, r4, r1, lsr #3 @ find maximum number on the way size - clz r5, r4 @ find bit position of way size increment - ldr r7, =0x7fff - ands r7, r7, r1, lsr #13 @ extract max number of the index size -loop1: - mov r9, r7 @ create working copy of max index -loop2: - orr r11, r10, r4, lsl r5 @ factor way and cache number into r11 - orr r11, r11, r9, lsl r2 @ factor index number into r11 - mcr p15, 0, r11, c7, c14, 2 @ clean & invalidate by set/way - subs r9, r9, #1 @ decrement the index - bge loop2 - subs r4, r4, #1 @ decrement the way - bge loop1 -skip: - add r10, r10, #2 @ increment cache number - cmp r3, r10 - bgt flush_levels -finished: - mov r10, #0 @ swith back to cache level 0 - mcr p15, 2, r10, c0, c0, 0 @ select current cache level in cssr - dsb st - isb - bx lr - _sunxi_cpu_entry: @ Set SMP bit mrc p15, 0, r0, c1, c0, 1 @@ -262,21 +215,7 @@ _sunxi_cpu_entry:
.globl psci_cpu_off psci_cpu_off: - mrc p15, 0, r0, c1, c0, 0 @ SCTLR - bic r0, r0, #(1 << 2) @ Clear C bit - mcr p15, 0, r0, c1, c0, 0 @ SCTLR - isb - dsb - - bl v7_flush_dcache_all - - clrex @ Why??? - - mrc p15, 0, r0, c1, c0, 1 @ ACTLR - bic r0, r0, #(1 << 6) @ Clear SMP bit - mcr p15, 0, r0, c1, c0, 1 @ ACTLR - isb - dsb + bl psci_cpu_off_common
@ Ask CPU0 to pull the rug... movw r0, #(GICD_BASE & 0xffff)
_sunxi_cpu_entry can be converted completely into a reusable psci_cpu_entry. Tegra124 will use it as well.
As with psci_disable_smp, also the enabling is designed to be overloaded in cased SMP is not controlled via ACTLR.
CC: Marc Zyngier marc.zyngier@arm.com Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/cpu/armv7/psci.S | 23 +++++++++++++++++++++++ arch/arm/cpu/armv7/sunxi/psci.S | 20 ++------------------ 2 files changed, 25 insertions(+), 18 deletions(-)
diff --git a/arch/arm/cpu/armv7/psci.S b/arch/arm/cpu/armv7/psci.S index cdcdccd..7d89b43 100644 --- a/arch/arm/cpu/armv7/psci.S +++ b/arch/arm/cpu/armv7/psci.S @@ -165,6 +165,15 @@ ENTRY(psci_disable_smp) ENDPROC(psci_disable_smp) .weak psci_disable_smp
+ENTRY(psci_enable_smp) + mrc p15, 0, r0, c1, c0, 1 @ ACTLR + orr r0, r0, #(1 << 6) @ Set SMP bit + mcr p15, 0, r0, c1, c0, 1 @ ACTLR + isb + bx lr +ENDPROC(psci_enable_smp) +.weak psci_enable_smp + ENTRY(psci_cpu_off_common) push {lr}
@@ -184,4 +193,18 @@ ENTRY(psci_cpu_off_common) bx lr ENDPROC(psci_cpu_off_common)
+ENTRY(psci_cpu_entry) + bl psci_enable_smp + + bl _nonsec_init + + adr r0, _psci_target_pc + ldr r0, [r0] + b _do_nonsec_entry +ENDPROC(psci_cpu_entry) + +.globl _psci_target_pc +_psci_target_pc: + .word 0 + .popsection diff --git a/arch/arm/cpu/armv7/sunxi/psci.S b/arch/arm/cpu/armv7/sunxi/psci.S index 05d047b..90dcff1 100644 --- a/arch/arm/cpu/armv7/sunxi/psci.S +++ b/arch/arm/cpu/armv7/sunxi/psci.S @@ -139,7 +139,7 @@ out: mcr p15, 0, r7, c1, c1, 0 @ r2 = target PC .globl psci_cpu_on psci_cpu_on: - adr r0, _target_pc + ldr r0, =_psci_target_pc str r2, [r0] dsb
@@ -151,7 +151,7 @@ psci_cpu_on: mov r4, #1 lsl r4, r4, r1
- adr r6, _sunxi_cpu_entry + ldr r6, =psci_cpu_entry str r6, [r0, #0x1a4] @ PRIVATE_REG (boot vector)
@ Assert reset on target CPU @@ -197,22 +197,6 @@ psci_cpu_on: mov r0, #ARM_PSCI_RET_SUCCESS @ Return PSCI_RET_SUCCESS mov pc, lr
-_target_pc: - .word 0 - -_sunxi_cpu_entry: - @ Set SMP bit - mrc p15, 0, r0, c1, c0, 1 - orr r0, r0, #0x40 - mcr p15, 0, r0, c1, c0, 1 - isb - - bl _nonsec_init - - adr r0, _target_pc - ldr r0, [r0] - b _do_nonsec_entry - .globl psci_cpu_off psci_cpu_off: bl psci_cpu_off_common
This algorithm will be useful on Tegra as well, plus we will need it for making _psci_target_pc per-CPU.
CC: Marc Zyngier marc.zyngier@arm.com Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/cpu/armv7/psci.S | 14 ++++++++++++++ arch/arm/cpu/armv7/sunxi/psci.S | 15 +++++---------- 2 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/arch/arm/cpu/armv7/psci.S b/arch/arm/cpu/armv7/psci.S index 7d89b43..18d85c4 100644 --- a/arch/arm/cpu/armv7/psci.S +++ b/arch/arm/cpu/armv7/psci.S @@ -193,6 +193,20 @@ ENTRY(psci_cpu_off_common) bx lr ENDPROC(psci_cpu_off_common)
+@ expects CPU ID in r0 and returns stack top in r0 +ENTRY(psci_get_cpu_stack_top) + mov r5, #0x400 @ 1kB of stack per CPU + mul r0, r0, r5 + + ldr r5, =psci_text_end @ end of monitor text + add r5, r5, #0x2000 @ Skip two pages + lsr r5, r5, #12 @ Align to start of page + lsl r5, r5, #12 + sub r0, r5, r0 @ here's our stack! + + bx lr +ENDPROC(psci_get_cpu_stack_top) + ENTRY(psci_cpu_entry) bl psci_enable_smp
diff --git a/arch/arm/cpu/armv7/sunxi/psci.S b/arch/arm/cpu/armv7/sunxi/psci.S index 90dcff1..dd583b2 100644 --- a/arch/arm/cpu/armv7/sunxi/psci.S +++ b/arch/arm/cpu/armv7/sunxi/psci.S @@ -241,17 +241,12 @@ psci_arch_init: mcr p15, 0, r5, c1, c1, 0 @ Write SCR isb
- bl psci_get_cpu_id - mov r5, #0x400 @ 1kB of stack per CPU - mul r0, r0, r5 - - adr r5, text_end @ end of text - add r5, r5, #0x2000 @ Skip two pages - lsr r5, r5, #12 @ Align to start of page - lsl r5, r5, #12 - sub sp, r5, r0 @ here's our stack! + bl psci_get_cpu_id @ CPU ID => r0 + bl psci_get_cpu_stack_top @ stack top => r0 + mov sp, r0
bx r6
-text_end: + .globl psci_text_end +psci_text_end: .popsection
Use a per-CPU variable for saving the target PC during CPU_ON operations. This allows us to run this service independently on targets that have more than 2 cores and also core-local power control.
CC: Marc Zyngier marc.zyngier@arm.com Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/cpu/armv7/psci.S | 11 +++++------ arch/arm/cpu/armv7/sunxi/psci.S | 9 ++++++--- 2 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/arch/arm/cpu/armv7/psci.S b/arch/arm/cpu/armv7/psci.S index 18d85c4..87c0c0b 100644 --- a/arch/arm/cpu/armv7/psci.S +++ b/arch/arm/cpu/armv7/psci.S @@ -17,6 +17,7 @@
#include <config.h> #include <linux/linkage.h> +#include <asm/macro.h> #include <asm/psci.h>
.pushsection ._secure.text, "ax" @@ -202,6 +203,7 @@ ENTRY(psci_get_cpu_stack_top) add r5, r5, #0x2000 @ Skip two pages lsr r5, r5, #12 @ Align to start of page lsl r5, r5, #12 + sub r5, r5, #4 @ reserve 1 word for target PC sub r0, r5, r0 @ here's our stack!
bx lr @@ -212,13 +214,10 @@ ENTRY(psci_cpu_entry)
bl _nonsec_init
- adr r0, _psci_target_pc - ldr r0, [r0] + bl psci_get_cpu_id @ CPU ID => r0 + bl psci_get_cpu_stack_top @ stack top => r0 + ldr r0, [r0] @ target PC at stack top b _do_nonsec_entry ENDPROC(psci_cpu_entry)
-.globl _psci_target_pc -_psci_target_pc: - .word 0 - .popsection diff --git a/arch/arm/cpu/armv7/sunxi/psci.S b/arch/arm/cpu/armv7/sunxi/psci.S index dd583b2..7ec0500 100644 --- a/arch/arm/cpu/armv7/sunxi/psci.S +++ b/arch/arm/cpu/armv7/sunxi/psci.S @@ -139,8 +139,11 @@ out: mcr p15, 0, r7, c1, c1, 0 @ r2 = target PC .globl psci_cpu_on psci_cpu_on: - ldr r0, =_psci_target_pc - str r2, [r0] + push {lr} + + mov r0, r1 + bl psci_get_cpu_stack_top @ get stack top of target CPU + str r2, [r0] @ store target PC at stack top dsb
movw r0, #(SUN7I_CPUCFG_BASE & 0xffff) @@ -195,7 +198,7 @@ psci_cpu_on: str r6, [r0, #0x1e4]
mov r0, #ARM_PSCI_RET_SUCCESS @ Return PSCI_RET_SUCCESS - mov pc, lr + pop {pc}
.globl psci_cpu_off psci_cpu_off:
From: Ian Campbell ijc@hellion.org.uk
I will need mc_security_cfg0/1 in a future patch and I added the rest while debugging, so thought I might as well commit them.
Signed-off-by: Ian Campbell ijc@hellion.org.uk Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/include/asm/arch-tegra124/mc.h | 35 +++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-)
diff --git a/arch/arm/include/asm/arch-tegra124/mc.h b/arch/arm/include/asm/arch-tegra124/mc.h index d526dfe..5557732 100644 --- a/arch/arm/include/asm/arch-tegra124/mc.h +++ b/arch/arm/include/asm/arch-tegra124/mc.h @@ -35,9 +35,40 @@ struct mc_ctlr { u32 mc_emem_adr_cfg; /* offset 0x54 */ u32 mc_emem_adr_cfg_dev0; /* offset 0x58 */ u32 mc_emem_adr_cfg_dev1; /* offset 0x5C */ - u32 reserved3[12]; /* offset 0x60 - 0x8C */ + u32 reserved3[4]; /* offset 0x60 - 0x6C */ + u32 mc_security_cfg0; /* offset 0x70 */ + u32 mc_security_cfg1; /* offset 0x74 */ + u32 reserved4[6]; /* offset 0x7C - 0x8C */ u32 mc_emem_arb_reserved[28]; /* offset 0x90 - 0xFC */ - u32 reserved4[338]; /* offset 0x100 - 0x644 */ + u32 reserved5[74]; /* offset 0x100 - 0x224 */ + u32 mc_smmu_translation_enable_0; /* offset 0x228 */ + u32 mc_smmu_translation_enable_1; /* offset 0x22C */ + u32 mc_smmu_translation_enable_2; /* offset 0x230 */ + u32 mc_smmu_translation_enable_3; /* offset 0x234 */ + u32 mc_smmu_afi_asid; /* offset 0x238 */ + u32 mc_smmu_avpc_asid; /* offset 0x23C */ + u32 mc_smmu_dc_asid; /* offset 0x240 */ + u32 mc_smmu_dcb_asid; /* offset 0x244 */ + u32 reserved6[2]; /* offset 0x248 - 0x24C */ + u32 mc_smmu_hc_asid; /* offset 0x250 */ + u32 mc_smmu_hda_asid; /* offset 0x254 */ + u32 mc_smmu_isp2_asid; /* offset 0x258 */ + u32 reserved7[2]; /* offset 0x25C - 0x260 */ + u32 mc_smmu_msenc_asid; /* offset 0x264 */ + u32 mc_smmu_nv_asid; /* offset 0x268 */ + u32 mc_smmu_nv2_asid; /* offset 0x26C */ + u32 mc_smmu_ppcs_asid; /* offset 0x270 */ + u32 mc_smmu_sata_asid; /* offset 0x274 */ + u32 reserved8[1]; /* offset 0x278 */ + u32 mc_smmu_vde_asid; /* offset 0x27C */ + u32 mc_smmu_vi_asid; /* offset 0x280 */ + u32 mc_smmu_vic_asid; /* offset 0x284 */ + u32 mc_smmu_xusb_host_asid; /* offset 0x288 */ + u32 mc_smmu_xusb_dev_asid; /* offset 0x28C */ + u32 reserved9[1]; /* offset 0x290 */ + u32 mc_smmu_tsec_asid; /* offset 0x294 */ + u32 mc_smmu_ppcs1_asid; /* offset 0x298 */ + u32 reserved10[235]; /* offset 0x29C - 0x644 */ u32 mc_video_protect_bom; /* offset 0x648 */ u32 mc_video_protect_size_mb; /* offset 0x64c */ u32 mc_video_protect_reg_ctrl; /* offset 0x650 */
In this case the secure code lives in RAM, and hence the memory node in the device tree needs to be adjusted. This avoids that the OS will map and possibly access the reservation.
Add support for setting CONFIG_ARMV7_SECURE_RESERVE_SIZE to carve out such a region. We only support cutting off memory from the beginning or the end of a RAM bank as we do not want to increase their number (which would happen if punching a hole) for simplicity reasons
This will be used in a subsequent patch for Jetson-TK1.
Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/cpu/armv7/virt-dt.c | 29 +++++++++++++++++++++++++++++ arch/arm/include/asm/armv7.h | 1 + arch/arm/lib/bootm-fdt.c | 6 ++++++ 3 files changed, 36 insertions(+)
diff --git a/arch/arm/cpu/armv7/virt-dt.c b/arch/arm/cpu/armv7/virt-dt.c index 9408e33..6b86018 100644 --- a/arch/arm/cpu/armv7/virt-dt.c +++ b/arch/arm/cpu/armv7/virt-dt.c @@ -16,6 +16,7 @@ */
#include <common.h> +#include <errno.h> #include <stdio_dev.h> #include <linux/ctype.h> #include <linux/types.h> @@ -88,6 +89,34 @@ static int fdt_psci(void *fdt) return 0; }
+int armv7_apply_memory_carveout(u64 *start, u64 *size) +{ +#ifdef CONFIG_ARMV7_SECURE_RESERVE_SIZE + if (*start + *size < CONFIG_ARMV7_SECURE_BASE || + *start >= (u64)CONFIG_ARMV7_SECURE_BASE + + CONFIG_ARMV7_SECURE_RESERVE_SIZE) + return 0; + + /* carveout must be at the beginning or the end of the bank */ + if (*start == CONFIG_ARMV7_SECURE_BASE || + *start + *size == (u64)CONFIG_ARMV7_SECURE_BASE + + CONFIG_ARMV7_SECURE_RESERVE_SIZE) { + if (*size < CONFIG_ARMV7_SECURE_RESERVE_SIZE) { + debug("Secure monitor larger than RAM bank!?\n"); + return -EINVAL; + } + *size -= CONFIG_ARMV7_SECURE_RESERVE_SIZE; + if (*start == CONFIG_ARMV7_SECURE_BASE) + *start += CONFIG_ARMV7_SECURE_RESERVE_SIZE; + return 0; + } + debug("Secure monitor not located at beginning or end of RAM bank\n"); + return -EINVAL; +#else /* !CONFIG_ARMV7_SECURE_RESERVE_SIZE */ + return 0; +#endif +} + int psci_update_dt(void *fdt) { #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT) diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h index 58d8b16..77adb24 100644 --- a/arch/arm/include/asm/armv7.h +++ b/arch/arm/include/asm/armv7.h @@ -124,6 +124,7 @@ void v7_outer_cache_inval_range(u32 start, u32 end); #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
int armv7_init_nonsec(void); +int armv7_apply_memory_carveout(u64 *start, u64 *size); bool armv7_boot_nonsec(void);
/* defined in assembly file */ diff --git a/arch/arm/lib/bootm-fdt.c b/arch/arm/lib/bootm-fdt.c index 665a3bc..2b9cf3c 100644 --- a/arch/arm/lib/bootm-fdt.c +++ b/arch/arm/lib/bootm-fdt.c @@ -17,6 +17,7 @@
#include <common.h> #include <fdt_support.h> +#include <asm/armv7.h> #include <asm/psci.h>
DECLARE_GLOBAL_DATA_PTR; @@ -31,6 +32,11 @@ int arch_fixup_fdt(void *blob) for (bank = 0; bank < CONFIG_NR_DRAM_BANKS; bank++) { start[bank] = bd->bi_dram[bank].start; size[bank] = bd->bi_dram[bank].size; +#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT) + ret = armv7_apply_memory_carveout(&start[bank], &size[bank]); + if (ret) + return ret; +#endif }
ret = fdt_fixup_memory_banks(blob, start, size, CONFIG_NR_DRAM_BANKS);
Will be used for unpowergating CPUs.
Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/include/asm/arch-tegra/powergate.h | 1 + arch/arm/mach-tegra/powergate.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/arm/include/asm/arch-tegra/powergate.h b/arch/arm/include/asm/arch-tegra/powergate.h index 130b58b..2e491f1 100644 --- a/arch/arm/include/asm/arch-tegra/powergate.h +++ b/arch/arm/include/asm/arch-tegra/powergate.h @@ -33,6 +33,7 @@ enum tegra_powergate {
int tegra_powergate_sequence_power_up(enum tegra_powergate id, enum periph_id periph); +int tegra_powergate_power_on(enum tegra_powergate id); int tegra_powergate_power_off(enum tegra_powergate id);
#endif diff --git a/arch/arm/mach-tegra/powergate.c b/arch/arm/mach-tegra/powergate.c index 439cff3..6331cd4 100644 --- a/arch/arm/mach-tegra/powergate.c +++ b/arch/arm/mach-tegra/powergate.c @@ -44,7 +44,7 @@ static int tegra_powergate_set(enum tegra_powergate id, bool state) return -ETIMEDOUT; }
-static int tegra_powergate_power_on(enum tegra_powergate id) +int tegra_powergate_power_on(enum tegra_powergate id) { return tegra_powergate_set(id, true); }
This function will be used to initialize CPU power management for Tegra SOCs. For now it does nothing.
Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/include/asm/arch-tegra/ap.h | 5 +++++ board/nvidia/common/board.c | 3 +++ 2 files changed, 8 insertions(+)
diff --git a/arch/arm/include/asm/arch-tegra/ap.h b/arch/arm/include/asm/arch-tegra/ap.h index ca40e4e..c2b16cf 100644 --- a/arch/arm/include/asm/arch-tegra/ap.h +++ b/arch/arm/include/asm/arch-tegra/ap.h @@ -63,6 +63,11 @@ int tegra_get_chip(void); */ int tegra_get_sku_info(void);
+/** + * Initialize power management for application processors + */ +void ap_pm_init(void); + /* Do any chip-specific cache config */ void config_cache(void);
diff --git a/board/nvidia/common/board.c b/board/nvidia/common/board.c index 018dddb..c3d7789 100644 --- a/board/nvidia/common/board.c +++ b/board/nvidia/common/board.c @@ -57,6 +57,7 @@ const struct tegra_sysinfo sysinfo = { CONFIG_TEGRA_BOARD_STRING };
+__weak void ap_pm_init(void) {} __weak void pinmux_init(void) {} __weak void pin_mux_usb(void) {} __weak void pin_mux_spi(void) {} @@ -97,6 +98,8 @@ int board_init(void) clock_init(); clock_verify();
+ ap_pm_init(); + #ifdef CONFIG_TEGRA_SPI pin_mux_spi(); #endif
This is based on Thierry Reding's work and uses Ian Campell's preparatory patches. It comes with full support for CPU_ON/OFF PSCI services. The algorithm used in this version for turning CPUs on and off was proposed by Peter De Schrijver and Thierry Reding in http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/210881. It consists of first enabling CPU1..3 via the PMC, just to powergate them again with the help of the Flow Controller. Once the Flow Controller is in place, we can leave the PMC alone while processing CPU_ON and CPU_OFF PSCI requests.
Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/include/asm/arch-tegra124/flow.h | 6 ++ arch/arm/mach-tegra/Makefile | 4 ++ arch/arm/mach-tegra/psci.S | 101 ++++++++++++++++++++++++++++++ arch/arm/mach-tegra/tegra124/Makefile | 4 ++ arch/arm/mach-tegra/tegra124/ap.c | 55 ++++++++++++++++ 5 files changed, 170 insertions(+) create mode 100644 arch/arm/mach-tegra/psci.S create mode 100644 arch/arm/mach-tegra/tegra124/ap.c
diff --git a/arch/arm/include/asm/arch-tegra124/flow.h b/arch/arm/include/asm/arch-tegra124/flow.h index 0db1881..d6f515f 100644 --- a/arch/arm/include/asm/arch-tegra124/flow.h +++ b/arch/arm/include/asm/arch-tegra124/flow.h @@ -37,4 +37,10 @@ struct flow_ctlr { /* FLOW_CTLR_CLUSTER_CONTROL_0 0x2c */ #define ACTIVE_LP (1 << 0)
+/* CPUn_CSR_0 */ +#define CSR_ENABLE (1 << 0) +#define CSR_IMMEDIATE_WAKE (1 << 3) +#define CSR_WAIT_WFI_SHIFT 8 +#define CSR_PWR_OFF_STS (1 << 16) + #endif /* _TEGRA124_FLOW_H_ */ diff --git a/arch/arm/mach-tegra/Makefile b/arch/arm/mach-tegra/Makefile index 04cef0a..0779086 100644 --- a/arch/arm/mach-tegra/Makefile +++ b/arch/arm/mach-tegra/Makefile @@ -25,6 +25,10 @@ obj-y += xusb-padctl.o obj-$(CONFIG_DISPLAY_CPUINFO) += sys_info.o obj-$(CONFIG_TEGRA124) += vpr.o
+ifndef CONFIG_SPL_BUILD +obj-$(CONFIG_ARMV7_PSCI) += psci.o +endif + obj-$(CONFIG_TEGRA20) += tegra20/ obj-$(CONFIG_TEGRA30) += tegra30/ obj-$(CONFIG_TEGRA114) += tegra114/ diff --git a/arch/arm/mach-tegra/psci.S b/arch/arm/mach-tegra/psci.S new file mode 100644 index 0000000..e4733e6 --- /dev/null +++ b/arch/arm/mach-tegra/psci.S @@ -0,0 +1,101 @@ +/* + * Copyright (C) 2014, NVIDIA + * Copyright (C) 2015, Siemens AG + * + * Authors: + * Thierry Reding treding@nvidia.com + * Jan Kiszka jan.kiszka@siemens.com + * + * SPDX-License-Identifier: GPL-2.0+ + */ + +#include <linux/linkage.h> +#include <asm/macro.h> +#include <asm/psci.h> + + .pushsection ._secure.text, "ax" + .arch_extension sec + +#define TEGRA_SB_CSR_0 0x6000c200 +#define NS_RST_VEC_WR_DIS (1 << 1) + +#define TEGRA_RESET_EXCEPTION_VECTOR 0x6000f100 + +#define TEGRA_FLOW_CTRL_BASE 0x60007000 +#define FLOW_CTRL_CPU_CSR 0x08 +#define CSR_ENABLE (1 << 0) +#define CSR_IMMEDIATE_WAKE (1 << 3) +#define CSR_WAIT_WFI_SHIFT 8 +#define FLOW_CTRL_CPU1_CSR 0x18 + +@ converts CPU ID into FLOW_CTRL_CPUn_CSR offset +.macro get_csr_reg cpu, ofs, tmp + cmp \cpu, #0 @ CPU0? + lsl \tmp, \cpu, #3 @ multiple by 8 (register offset CPU1-3) + moveq \ofs, #FLOW_CTRL_CPU_CSR + addne \ofs, \tmp, #FLOW_CTRL_CPU1_CSR - 8 +.endm + +ENTRY(psci_arch_init) + mov r6, lr + + mrc p15, 0, r5, c1, c1, 0 @ Read SCR + bic r5, r5, #1 @ Secure mode + mcr p15, 0, r5, c1, c1, 0 @ Write SCR + isb + + @ lock reset vector for non-secure + ldr r4, =TEGRA_SB_CSR_0 + ldr r5, [r4] + orr r5, r5, #NS_RST_VEC_WR_DIS + str r5, [r4] + + bl psci_get_cpu_id @ CPU ID => r0 + bl psci_get_cpu_stack_top @ stack top => r0 + mov sp, r0 + + bx r6 +ENDPROC(psci_arch_init) + +ENTRY(psci_cpu_off) + bl psci_cpu_off_common + + bl psci_get_cpu_id @ CPU ID => r0 + + get_csr_reg r0, r2, r3 + + ldr r6, =TEGRA_FLOW_CTRL_BASE + mov r5, #(CSR_ENABLE) + mov r4, #(1 << CSR_WAIT_WFI_SHIFT) + add r5, r4, lsl r0 + str r5, [r6, r2] + +_loop: wfi + b _loop +ENDPROC(psci_cpu_off) + +ENTRY(psci_cpu_on) + push {lr} + + mov r0, r1 + bl psci_get_cpu_stack_top @ get stack top of target CPU + str r2, [r0] @ store target PC at stack top + dsb + + ldr r6, =TEGRA_RESET_EXCEPTION_VECTOR + ldr r5, =psci_cpu_entry + str r5, [r6] + + get_csr_reg r1, r2, r3 + + ldr r6, =TEGRA_FLOW_CTRL_BASE + mov r5, #(CSR_IMMEDIATE_WAKE | CSR_ENABLE) + str r5, [r6, r2] + + mov r0, #ARM_PSCI_RET_SUCCESS @ Return PSCI_RET_SUCCESS + pop {pc} +ENDPROC(psci_cpu_on) + + .globl psci_text_end +psci_text_end: + .popsection diff --git a/arch/arm/mach-tegra/tegra124/Makefile b/arch/arm/mach-tegra/tegra124/Makefile index ef2da29..d19ddb3 100644 --- a/arch/arm/mach-tegra/tegra124/Makefile +++ b/arch/arm/mach-tegra/tegra124/Makefile @@ -11,3 +11,7 @@ obj-y += clock.o obj-y += funcmux.o obj-y += pinmux.o obj-y += xusb-padctl.o + +ifndef CONFIG_SPL_BUILD +obj-$(CONFIG_ARMV7_PSCI) += ap.o +endif diff --git a/arch/arm/mach-tegra/tegra124/ap.c b/arch/arm/mach-tegra/tegra124/ap.c new file mode 100644 index 0000000..d729c16 --- /dev/null +++ b/arch/arm/mach-tegra/tegra124/ap.c @@ -0,0 +1,55 @@ +/* + * (C) Copyright 2015, Siemens AG + * Author: Jan Kiszka jan.kiszka@siemens.com + * + * SPDX-License-Identifier: GPL-2.0+ + */ + +#include <common.h> +#include <asm/io.h> +#include <asm/arch/flow.h> +#include <asm/arch/powergate.h> +#include <asm/arch-tegra/ap.h> +#include <asm/arch-tegra/pmc.h> + +static void park_cpu(void) +{ + while (1) + asm volatile("wfi"); +} + +void ap_pm_init(void) +{ + struct flow_ctlr *flow = (struct flow_ctlr *)NV_PA_FLOW_BASE; + + writel((u32)park_cpu, EXCEP_VECTOR_CPU_RESET_VECTOR); + + /* + * The naturally expected order of putting these CPUs under Flow + * Controller regime would be + * - configure the Flow Controller + * - power up the CPUs + * - wait for the CPUs to hit wfi and be powered down again + * + * However, this doesn't work in practice. We rather need to power them + * up first and park them in wfi. While they are waiting there, we can + * indeed program the Flow Controller to powergate them on wfi, which + * will then happen immediately as they are already in that state. + */ + tegra_powergate_power_on(TEGRA_POWERGATE_CPU1); + tegra_powergate_power_on(TEGRA_POWERGATE_CPU2); + tegra_powergate_power_on(TEGRA_POWERGATE_CPU3); + + writel((2 << CSR_WAIT_WFI_SHIFT) | CSR_ENABLE, &flow->cpu1_csr); + writel((4 << CSR_WAIT_WFI_SHIFT) | CSR_ENABLE, &flow->cpu2_csr); + writel((8 << CSR_WAIT_WFI_SHIFT) | CSR_ENABLE, &flow->cpu3_csr); + + writel(EVENT_MODE_STOP, &flow->halt_cpu1_events); + writel(EVENT_MODE_STOP, &flow->halt_cpu2_events); + writel(EVENT_MODE_STOP, &flow->halt_cpu3_events); + + while (!(readl(&flow->cpu1_csr) & CSR_PWR_OFF_STS) || + !(readl(&flow->cpu2_csr) & CSR_PWR_OFF_STS) || + !(readl(&flow->cpu3_csr) & CSR_PWR_OFF_STS)) + /* wait */; +}
From: Ian Campbell ijc@hellion.org.uk
These registers can be used to prevent non-secure world from accessing a megabyte aligned region of RAM, use them to protect the u-boot secure monitor code.
At first I tried to do this from s_init(), however this inexplicably causes u-boot's networking (e.g. DHCP) to fail, while networking under Linux was fine.
So instead I have added a new weak arch function protect_secure_section() called from relocate_secure_section() and reserved the region there. This is better overall since it defers the reservation until after the sec vs. non-sec decision (which can be influenced by an envvar) has been made when booting the os.
Signed-off-by: Ian Campbell ijc@hellion.org.uk [Jan: tiny style adjustment] Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/cpu/armv7/virt-v7.c | 5 +++++ arch/arm/include/asm/system.h | 1 + arch/arm/mach-tegra/ap.c | 15 +++++++++++++++ 3 files changed, 21 insertions(+)
diff --git a/arch/arm/cpu/armv7/virt-v7.c b/arch/arm/cpu/armv7/virt-v7.c index 4cb8806..4fbf26b 100644 --- a/arch/arm/cpu/armv7/virt-v7.c +++ b/arch/arm/cpu/armv7/virt-v7.c @@ -46,6 +46,10 @@ static unsigned long get_gicd_base_address(void) #endif }
+/* Define a specific version of this function to enable any available + * hardware protections for the reserved region */ +void __weak protect_secure_section(void) {} + static void relocate_secure_section(void) { #ifdef CONFIG_ARMV7_SECURE_BASE @@ -54,6 +58,7 @@ static void relocate_secure_section(void) memcpy((void *)CONFIG_ARMV7_SECURE_BASE, __secure_start, sz); flush_dcache_range(CONFIG_ARMV7_SECURE_BASE, CONFIG_ARMV7_SECURE_BASE + sz + 1); + protect_secure_section(); invalidate_icache_all(); #endif } diff --git a/arch/arm/include/asm/system.h b/arch/arm/include/asm/system.h index 2a5bed2..d6dfddd 100644 --- a/arch/arm/include/asm/system.h +++ b/arch/arm/include/asm/system.h @@ -77,6 +77,7 @@ void armv8_switch_to_el1(void); void gic_init(void); void gic_send_sgi(unsigned long sgino); void wait_for_wakeup(void); +void protect_secure_region(void); void smp_kick_all_cpus(void);
void flush_l3_cache(void); diff --git a/arch/arm/mach-tegra/ap.c b/arch/arm/mach-tegra/ap.c index a17dfd1..869a2ed 100644 --- a/arch/arm/mach-tegra/ap.c +++ b/arch/arm/mach-tegra/ap.c @@ -10,6 +10,7 @@ #include <common.h> #include <asm/io.h> #include <asm/arch/gp_padctrl.h> +#include <asm/arch/mc.h> #include <asm/arch-tegra/ap.h> #include <asm/arch-tegra/clock.h> #include <asm/arch-tegra/fuse.h> @@ -154,6 +155,20 @@ static void init_pmc_scratch(void) writel(odmdata, &pmc->pmc_scratch20); }
+#ifdef CONFIG_ARMV7_SECURE_RESERVE_SIZE +void protect_secure_section(void) +{ + struct mc_ctlr *mc = (struct mc_ctlr *)NV_PA_MC_BASE; + + /* Must be MB aligned */ + BUILD_BUG_ON(CONFIG_ARMV7_SECURE_BASE & 0xFFFFF); + BUILD_BUG_ON(CONFIG_ARMV7_SECURE_RESERVE_SIZE & 0xFFFFF); + + writel(CONFIG_ARMV7_SECURE_BASE, &mc->mc_security_cfg0); + writel(CONFIG_ARMV7_SECURE_RESERVE_SIZE >> 20, &mc->mc_security_cfg1); +} +#endif + void s_init(void) { /* Init PMC scratch memory */
We only set CNTFRQ in arch_timer_init for the boot CPU. But this has to happen for all cores.
Fixing this resolves problems of KVM with emulating the generic timer/counter.
Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/mach-tegra/psci.S | 13 +++++++++++++ 1 file changed, 13 insertions(+)
diff --git a/arch/arm/mach-tegra/psci.S b/arch/arm/mach-tegra/psci.S index e4733e6..b836da1 100644 --- a/arch/arm/mach-tegra/psci.S +++ b/arch/arm/mach-tegra/psci.S @@ -51,12 +51,25 @@ ENTRY(psci_arch_init) str r5, [r4]
bl psci_get_cpu_id @ CPU ID => r0 + + adr r5, _sys_clock_freq + cmp r0, #0 + + mrceq p15, 0, r7, c14, c0, 0 @ read CNTFRQ from CPU0 + streq r7, [r5] + + ldrne r7, [r5] + mcrne p15, 0, r7, c14, c0, 0 @ write CNTFRQ to CPU1..3 + bl psci_get_cpu_stack_top @ stack top => r0 mov sp, r0
bx r6 ENDPROC(psci_arch_init)
+_sys_clock_freq: + .word 0 + ENTRY(psci_cpu_off) bl psci_cpu_off_common
From: Thierry Reding treding@nvidia.com
Make sure to enable the SMMU when booting the kernel in non-secure mode. This is necessary because some of the SMMU registers are restricted to TrustZone-secured requestors, hence the kernel wouldn't be able to turn the SMMU on. At the same time, enable translation for all memory clients for the same reasons. The kernel will still be able to control SMMU IOVA translation using the per-SWGROUP enable bits.
Signed-off-by: Thierry Reding treding@nvidia.com Signed-off-by: Jan Kiszka jan.kiszka@siemens.com --- arch/arm/include/asm/arch-tegra124/mc.h | 2 ++ arch/arm/mach-tegra/ap.c | 40 +++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+)
diff --git a/arch/arm/include/asm/arch-tegra124/mc.h b/arch/arm/include/asm/arch-tegra124/mc.h index 5557732..37998a4 100644 --- a/arch/arm/include/asm/arch-tegra124/mc.h +++ b/arch/arm/include/asm/arch-tegra124/mc.h @@ -74,6 +74,8 @@ struct mc_ctlr { u32 mc_video_protect_reg_ctrl; /* offset 0x650 */ };
+#define TEGRA_MC_SMMU_CONFIG_ENABLE (1 << 0) + #define TEGRA_MC_VIDEO_PROTECT_REG_WRITE_ACCESS_ENABLED (0 << 0) #define TEGRA_MC_VIDEO_PROTECT_REG_WRITE_ACCESS_DISABLED (1 << 0)
diff --git a/arch/arm/mach-tegra/ap.c b/arch/arm/mach-tegra/ap.c index 869a2ed..0b94e8a 100644 --- a/arch/arm/mach-tegra/ap.c +++ b/arch/arm/mach-tegra/ap.c @@ -169,6 +169,43 @@ void protect_secure_section(void) } #endif
+#if defined(CONFIG_ARMV7_NONSEC) +static void smmu_flush(struct mc_ctlr *mc) +{ + (void)readl(&mc->mc_smmu_config); +} + +static void smmu_enable(void) +{ + struct mc_ctlr *mc = (struct mc_ctlr *)NV_PA_MC_BASE; + u32 value; + + /* + * Enable translation for all clients since access to this register + * is restricted to TrustZone-secured requestors. The kernel will use + * the per-SWGROUP enable bits to enable or disable translations. + */ + writel(0xffffffff, &mc->mc_smmu_translation_enable_0); + writel(0xffffffff, &mc->mc_smmu_translation_enable_1); + writel(0xffffffff, &mc->mc_smmu_translation_enable_2); + writel(0xffffffff, &mc->mc_smmu_translation_enable_3); + + /* + * Enable SMMU globally since access to this register is restricted + * to TrustZone-secured requestors. + */ + value = readl(&mc->mc_smmu_config); + value |= TEGRA_MC_SMMU_CONFIG_ENABLE; + writel(value, &mc->mc_smmu_config); + + smmu_flush(mc); +} +#else +static void smmu_enable(void) +{ +} +#endif + void s_init(void) { /* Init PMC scratch memory */ @@ -179,6 +216,9 @@ void s_init(void) /* init the cache */ config_cache();
+ /* enable SMMU */ + smmu_enable(); + /* init vpr */ config_vpr(); }
From: Ian Campbell ijc@hellion.org.uk
The secure world code is relocated to the MB just below the top of 4G, we reserve it in the FDT (by setting CONFIG_ARMV7_SECURE_RESERVE_SIZE) but it is not protected in h/w. See next patch.
Signed-off-by: Ian Campbell ijc@hellion.org.uk Signed-off-by: Jan Kiszka jan.kiszka@siemens.com Reviewed-by: Tom Rini trini@konsulko.com Reviewed-by: Thierry Reding treding@nvidia.com Tested-by: Thierry Reding treding@nvidia.com Tested-by: Ian Campbell ijc@hellion.org.uk --- arch/arm/mach-tegra/tegra124/Kconfig | 2 ++ include/configs/jetson-tk1.h | 5 +++++ 2 files changed, 7 insertions(+)
diff --git a/arch/arm/mach-tegra/tegra124/Kconfig b/arch/arm/mach-tegra/tegra124/Kconfig index 88f627c..5114299 100644 --- a/arch/arm/mach-tegra/tegra124/Kconfig +++ b/arch/arm/mach-tegra/tegra124/Kconfig @@ -5,6 +5,8 @@ choice
config TARGET_JETSON_TK1 bool "NVIDIA Tegra124 Jetson TK1 board" + select CPU_V7_HAS_NONSEC if !SPL_BUILD + select CPU_V7_HAS_VIRT if !SPL_BUILD
config TARGET_NYAN_BIG bool "Google/NVIDIA Nyan-big Chrombook" diff --git a/include/configs/jetson-tk1.h b/include/configs/jetson-tk1.h index 8c016b7..aeafbd5 100644 --- a/include/configs/jetson-tk1.h +++ b/include/configs/jetson-tk1.h @@ -79,4 +79,9 @@ #include "tegra-common-usb-gadget.h" #include "tegra-common-post.h"
+#define CONFIG_ARMV7_PSCI 1 +/* Reserve top 1M for secure RAM */ +#define CONFIG_ARMV7_SECURE_BASE 0xfff00000 +#define CONFIG_ARMV7_SECURE_RESERVE_SIZE 0x00100000 + #endif /* __CONFIG_H */
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
Changes in v6:
- rebased over master
- included Thierry's SMMU enabling patch
- moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
Changes in v6:
- rebased over master
- included Thierry's SMMU enabling patch
- moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
Changes in v6:
- rebased over master
- included Thierry's SMMU enabling patch
- moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
So we can either leave out the last patch for now and wait until the cpu-idle fix is upstream or tell the people picking upstream u-boot to turn either virt support in u-boot on or CPU_IDLE in the kernel off (just tell me where to put such a note).
Jan
On 04/14/2015 08:12 AM, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
Changes in v6:
- rebased over master
- included Thierry's SMMU enabling patch
- moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
So we can either leave out the last patch for now and wait until the cpu-idle fix is upstream or tell the people picking upstream u-boot to turn either virt support in u-boot on or CPU_IDLE in the kernel off (just tell me where to put such a note).
We should definitely not enable support by default if it requires kernel changes. If someone wants to use the feature, they should edit the source to enable it.
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
Changes in v6:
- rebased over master
- included Thierry's SMMU enabling patch
- moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
Ian.
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
Changes in v6:
- rebased over master
- included Thierry's SMMU enabling patch
- moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
Can this be done selectively for the TK1 only?
Also, Thierry, what is the time frame for the upstream fix? Is it lined up for 4.1?
Jan
On Tue, 2015-04-14 at 16:33 +0200, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
Changes in v6:
- rebased over master
- included Thierry's SMMU enabling patch
- moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
Can this be done selectively for the TK1 only?
Well, you might need to add the mechanism for inserting it into the default bootenv, but I don't think that should be too hard at either the tegra or more generic levels.
Ian.
On 2015-04-14 16:40, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:33 +0200, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
> Changes in v6: > - rebased over master > - included Thierry's SMMU enabling patch > - moved activation patch at the end so that it can be held back > > This version can also be found at > https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
Can this be done selectively for the TK1 only?
Well, you might need to add the mechanism for inserting it into the default bootenv, but I don't think that should be too hard at either the tegra or more generic levels.
Maybe this already works:
diff --git a/include/configs/tegra-common-post.h b/include/configs/tegra-common-post.h index c3ad8be..c1f6c24 100644 --- a/include/configs/tegra-common-post.h +++ b/include/configs/tegra-common-post.h @@ -56,6 +56,7 @@ "fdt_high=ffffffff\0" \ "initrd_high=ffffffff\0" \ BOOTENV \ + "bootm_boot_mode=sec\0" \ BOARD_EXTRA_ENV_SETTINGS
#if defined(CONFIG_TEGRA20_SFLASH) || defined(CONFIG_TEGRA20_SLINK) || defined(CONFIG_TEGRA114_SPI)
Will test later.
Jan
On 04/14/2015 08:45 AM, Jan Kiszka wrote:
On 2015-04-14 16:40, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:33 +0200, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote: > On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote: > >> Changes in v6: >> - rebased over master >> - included Thierry's SMMU enabling patch >> - moved activation patch at the end so that it can be held back >> >> This version can also be found at >> https://github.com/siemens/u-boot/tree/jetson-tk1-v6. > > So what level of coordination do we need on applying this series so that > kernels (both old and new) can continue to function? And perhaps README > updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
Can this be done selectively for the TK1 only?
Well, you might need to add the mechanism for inserting it into the default bootenv, but I don't think that should be too hard at either the tegra or more generic levels.
Maybe this already works:
diff --git a/include/configs/tegra-common-post.h b/include/configs/tegra-common-post.h index c3ad8be..c1f6c24 100644 --- a/include/configs/tegra-common-post.h +++ b/include/configs/tegra-common-post.h @@ -56,6 +56,7 @@ "fdt_high=ffffffff\0" \ "initrd_high=ffffffff\0" \ BOOTENV \
"bootm_boot_mode=sec\0" \ BOARD_EXTRA_ENV_SETTINGS
#if defined(CONFIG_TEGRA20_SFLASH) || defined(CONFIG_TEGRA20_SLINK) || defined(CONFIG_TEGRA114_SPI)
That will add the variable for all boards, even when the feature isn't enabled.
You should either:
* Modify jetson-tk1.h to add that value to BOARD_EXTRA_ENV_SETTINGS.
* Better: Modify tegra-common-post.h to do something like:
#ifdef whatever_enables_the_psci_feature_etc #define PSCI_EXTRA_ENV_SETTINGS "bootm_boot_mode=sec\0 #else #define PSCI_EXTRA_ENV_SETTINGS #endif
... and add PSCI_EXTRA_ENV_SETTINGS into the value of CONFIG_EXTRA_ENV_SETTINGS.
On 2015-04-14 16:50, Stephen Warren wrote:
On 04/14/2015 08:45 AM, Jan Kiszka wrote:
On 2015-04-14 16:40, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:33 +0200, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote: > On 04/14/2015 07:46 AM, Tom Rini wrote: >> On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote: >> >>> Changes in v6: >>> - rebased over master >>> - included Thierry's SMMU enabling patch >>> - moved activation patch at the end so that it can be held back >>> >>> This version can also be found at >>> https://github.com/siemens/u-boot/tree/jetson-tk1-v6. >> >> So what level of coordination do we need on applying this series >> so that >> kernels (both old and new) can continue to function? And >> perhaps README >> updates or similar? Thanks! > > Hopefully this series doesn't change anything by default, and simply > allows people to turn on support for booting kernels in > non-secure mode > if they want to? If so, there shouldn't be any co-ordination > required. > If it changes the default behaviour, co-ordination is probably > required, > and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
Can this be done selectively for the TK1 only?
Well, you might need to add the mechanism for inserting it into the default bootenv, but I don't think that should be too hard at either the tegra or more generic levels.
Maybe this already works:
diff --git a/include/configs/tegra-common-post.h b/include/configs/tegra-common-post.h index c3ad8be..c1f6c24 100644 --- a/include/configs/tegra-common-post.h +++ b/include/configs/tegra-common-post.h @@ -56,6 +56,7 @@ "fdt_high=ffffffff\0" \ "initrd_high=ffffffff\0" \ BOOTENV \
"bootm_boot_mode=sec\0" \ BOARD_EXTRA_ENV_SETTINGS
#if defined(CONFIG_TEGRA20_SFLASH) || defined(CONFIG_TEGRA20_SLINK)
|| defined(CONFIG_TEGRA114_SPI)
That will add the variable for all boards, even when the feature isn't enabled.
You should either:
- Modify jetson-tk1.h to add that value to BOARD_EXTRA_ENV_SETTINGS.
That issue should affect more boards than the TK1.
- Better: Modify tegra-common-post.h to do something like:
#ifdef whatever_enables_the_psci_feature_etc #define PSCI_EXTRA_ENV_SETTINGS "bootm_boot_mode=sec\0 #else #define PSCI_EXTRA_ENV_SETTINGS #endif
... and add PSCI_EXTRA_ENV_SETTINGS into the value of CONFIG_EXTRA_ENV_SETTINGS.
Yep, makes sense.
Thanks, Jan
On 04/14/2015 08:30 AM, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
Changes in v6:
- rebased over master
- included Thierry's SMMU enabling patch
- moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
Yes, if there's already an environment variable that controls the behaviour, that'd be even better.
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
Changes in v6:
- rebased over master
- included Thierry's SMMU enabling patch
- moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
This turned out to not work as expected: booting in secure mode seems to prevent that Linux can bring up CPUs 1-3. Not sure if this is to be expected or a bug, but I will now take a different route:
diff --git a/arch/arm/cpu/armv7/Kconfig b/arch/arm/cpu/armv7/Kconfig index 61e7c82..e5d4dcc 100644 --- a/arch/arm/cpu/armv7/Kconfig +++ b/arch/arm/cpu/armv7/Kconfig @@ -27,7 +27,7 @@ config ARMV7_BOOT_SEC_DEFAULT config ARMV7_VIRT boolean "Enable support for hardware virtualization" if EXPERT depends on CPU_V7_HAS_VIRT && ARMV7_NONSEC - default y + default y if !TEGRA ---help--- Say Y here to boot in hypervisor (HYP) mode when booting non-secure.
A patch to be inserted before the last one (and to be reverted once Linux is fixed) will follow.
Jan
On 04/17/2015 12:47 AM, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
Changes in v6:
- rebased over master
- included Thierry's SMMU enabling patch
- moved activation patch at the end so that it can be held back
This version can also be found at https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
This turned out to not work as expected: booting in secure mode seems to prevent that Linux can bring up CPUs 1-3. Not sure if this is to be expected or a bug, but I will now take a different route:
That was the whole point of the environment variable suggestion; the environment variable would default to off so nobody got new behaviour, but anyone who wanted to boot in secure mode could simply set the environment variable and get it. That way, nobody who doesn't want the feature needs to co-ordinate U-Boot and kernel updates. Why doesn't that work?
diff --git a/arch/arm/cpu/armv7/Kconfig b/arch/arm/cpu/armv7/Kconfig index 61e7c82..e5d4dcc 100644 --- a/arch/arm/cpu/armv7/Kconfig +++ b/arch/arm/cpu/armv7/Kconfig @@ -27,7 +27,7 @@ config ARMV7_BOOT_SEC_DEFAULT config ARMV7_VIRT boolean "Enable support for hardware virtualization" if EXPERT depends on CPU_V7_HAS_VIRT && ARMV7_NONSEC
- default y
- default y if !TEGRA ---help--- Say Y here to boot in hypervisor (HYP) mode when booting non-secure.
A patch to be inserted before the last one (and to be reverted once Linux is fixed) will follow.
Jan
On 2015-04-17 15:57, Stephen Warren wrote:
On 04/17/2015 12:47 AM, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote:
On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote:
> Changes in v6: > - rebased over master > - included Thierry's SMMU enabling patch > - moved activation patch at the end so that it can be held back > > This version can also be found at > https://github.com/siemens/u-boot/tree/jetson-tk1-v6.
So what level of coordination do we need on applying this series so that kernels (both old and new) can continue to function? And perhaps README updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
This turned out to not work as expected: booting in secure mode seems to prevent that Linux can bring up CPUs 1-3. Not sure if this is to be expected or a bug, but I will now take a different route:
That was the whole point of the environment variable suggestion; the environment variable would default to off so nobody got new behaviour, but anyone who wanted to boot in secure mode could simply set the environment variable and get it. That way, nobody who doesn't want the feature needs to co-ordinate U-Boot and kernel updates. Why doesn't that work?
Because it breaks SMP on Linux, namely the boot of secondary cores. Don't ask me why, I didn't debug the details. But you can probably reproduce by specifying bootm_boot_mode=sec with current U-boot and recent upstream kernels.
Jan
On 04/17/2015 08:02 AM, Jan Kiszka wrote:
On 2015-04-17 15:57, Stephen Warren wrote:
On 04/17/2015 12:47 AM, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote:
On 04/14/2015 07:46 AM, Tom Rini wrote: > On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote: > >> Changes in v6: >> - rebased over master >> - included Thierry's SMMU enabling patch >> - moved activation patch at the end so that it can be held back >> >> This version can also be found at >> https://github.com/siemens/u-boot/tree/jetson-tk1-v6. > > So what level of coordination do we need on applying this series > so that > kernels (both old and new) can continue to function? And perhaps > README > updates or similar? Thanks!
Hopefully this series doesn't change anything by default, and simply allows people to turn on support for booting kernels in non-secure mode if they want to? If so, there shouldn't be any co-ordination required. If it changes the default behaviour, co-ordination is probably required, and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
This turned out to not work as expected: booting in secure mode seems to prevent that Linux can bring up CPUs 1-3. Not sure if this is to be expected or a bug, but I will now take a different route:
That was the whole point of the environment variable suggestion; the environment variable would default to off so nobody got new behaviour, but anyone who wanted to boot in secure mode could simply set the environment variable and get it. That way, nobody who doesn't want the feature needs to co-ordinate U-Boot and kernel updates. Why doesn't that work?
Because it breaks SMP on Linux, namely the boot of secondary cores. Don't ask me why, I didn't debug the details. But you can probably reproduce by specifying bootm_boot_mode=sec with current U-boot and recent upstream kernels.
I suspect the environment variable isn't working, and Linux is still being booted in non-secure mode. That would be a bug in U-Boot.
On 2015-04-17 16:12, Stephen Warren wrote:
On 04/17/2015 08:02 AM, Jan Kiszka wrote:
On 2015-04-17 15:57, Stephen Warren wrote:
On 04/17/2015 12:47 AM, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote:
On 2015-04-14 16:06, Stephen Warren wrote: > On 04/14/2015 07:46 AM, Tom Rini wrote: >> On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote: >> >>> Changes in v6: >>> - rebased over master >>> - included Thierry's SMMU enabling patch >>> - moved activation patch at the end so that it can be held >>> back >>> >>> This version can also be found at >>> https://github.com/siemens/u-boot/tree/jetson-tk1-v6. >> >> So what level of coordination do we need on applying this series >> so that >> kernels (both old and new) can continue to function? And perhaps >> README >> updates or similar? Thanks! > > Hopefully this series doesn't change anything by default, and simply > allows people to turn on support for booting kernels in non-secure > mode > if they want to? If so, there shouldn't be any co-ordination > required. > If it changes the default behaviour, co-ordination is probably > required, > and that'd be a bad thing.
Sorry, forgot to mention: I can't flip the default behaviour to leave virtualization support off only for the TK1. That's a generic default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
This turned out to not work as expected: booting in secure mode seems to prevent that Linux can bring up CPUs 1-3. Not sure if this is to be expected or a bug, but I will now take a different route:
That was the whole point of the environment variable suggestion; the environment variable would default to off so nobody got new behaviour, but anyone who wanted to boot in secure mode could simply set the environment variable and get it. That way, nobody who doesn't want the feature needs to co-ordinate U-Boot and kernel updates. Why doesn't that work?
Because it breaks SMP on Linux, namely the boot of secondary cores. Don't ask me why, I didn't debug the details. But you can probably reproduce by specifying bootm_boot_mode=sec with current U-boot and recent upstream kernels.
I suspect the environment variable isn't working, and Linux is still being booted in non-secure mode. That would be a bug in U-Boot.
Yes, might be. Linux reports being started (with that single CPU) in SVC mode. I'll recheck if it is a regression of my series.
Anyway, even if it worked, I think the cleaner way is flipping defaults at the config root level (virt support) as long as things are not working out of the box with Linux. That's what my other patch is doing now. Any concerns regarding it?
Thanks, Jan
On 04/17/2015 08:20 AM, Jan Kiszka wrote:
On 2015-04-17 16:12, Stephen Warren wrote:
On 04/17/2015 08:02 AM, Jan Kiszka wrote:
On 2015-04-17 15:57, Stephen Warren wrote:
On 04/17/2015 12:47 AM, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote:
On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote: > On 2015-04-14 16:06, Stephen Warren wrote: >> On 04/14/2015 07:46 AM, Tom Rini wrote: >>> On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote: >>> >>>> Changes in v6: >>>> - rebased over master >>>> - included Thierry's SMMU enabling patch >>>> - moved activation patch at the end so that it can be held >>>> back >>>> >>>> This version can also be found at >>>> https://github.com/siemens/u-boot/tree/jetson-tk1-v6. >>> >>> So what level of coordination do we need on applying this series >>> so that >>> kernels (both old and new) can continue to function? And perhaps >>> README >>> updates or similar? Thanks! >> >> Hopefully this series doesn't change anything by default, and simply >> allows people to turn on support for booting kernels in non-secure >> mode >> if they want to? If so, there shouldn't be any co-ordination >> required. >> If it changes the default behaviour, co-ordination is probably >> required, >> and that'd be a bad thing. > > Sorry, forgot to mention: I can't flip the default behaviour to leave > virtualization support off only for the TK1. That's a generic > default.
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
This turned out to not work as expected: booting in secure mode seems to prevent that Linux can bring up CPUs 1-3. Not sure if this is to be expected or a bug, but I will now take a different route:
That was the whole point of the environment variable suggestion; the environment variable would default to off so nobody got new behaviour, but anyone who wanted to boot in secure mode could simply set the environment variable and get it. That way, nobody who doesn't want the feature needs to co-ordinate U-Boot and kernel updates. Why doesn't that work?
Because it breaks SMP on Linux, namely the boot of secondary cores. Don't ask me why, I didn't debug the details. But you can probably reproduce by specifying bootm_boot_mode=sec with current U-boot and recent upstream kernels.
I suspect the environment variable isn't working, and Linux is still being booted in non-secure mode. That would be a bug in U-Boot.
Yes, might be. Linux reports being started (with that single CPU) in SVC mode. I'll recheck if it is a regression of my series.
Anyway, even if it worked, I think the cleaner way is flipping defaults at the config root level (virt support) as long as things are not working out of the box with Linux. That's what my other patch is doing now. Any concerns regarding it?
Yes.
This should be runtime configurable so that there's no need for a user to co-ordinate U-Boot and kernel updates. This also allows users to easily test the kernel in secure and NS mode without having to rebuild a bootloader.
On 2015-04-17 16:43, Stephen Warren wrote:
On 04/17/2015 08:20 AM, Jan Kiszka wrote:
On 2015-04-17 16:12, Stephen Warren wrote:
On 04/17/2015 08:02 AM, Jan Kiszka wrote:
On 2015-04-17 15:57, Stephen Warren wrote:
On 04/17/2015 12:47 AM, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote: > On Tue, 2015-04-14 at 16:12 +0200, Jan Kiszka wrote: >> On 2015-04-14 16:06, Stephen Warren wrote: >>> On 04/14/2015 07:46 AM, Tom Rini wrote: >>>> On Mon, Apr 13, 2015 at 06:48:05AM +0200, Jan Kiszka wrote: >>>> >>>>> Changes in v6: >>>>> - rebased over master >>>>> - included Thierry's SMMU enabling patch >>>>> - moved activation patch at the end so that it can be held >>>>> back >>>>> >>>>> This version can also be found at >>>>> https://github.com/siemens/u-boot/tree/jetson-tk1-v6. >>>> >>>> So what level of coordination do we need on applying this series >>>> so that >>>> kernels (both old and new) can continue to function? And perhaps >>>> README >>>> updates or similar? Thanks! >>> >>> Hopefully this series doesn't change anything by default, and >>> simply >>> allows people to turn on support for booting kernels in non-secure >>> mode >>> if they want to? If so, there shouldn't be any co-ordination >>> required. >>> If it changes the default behaviour, co-ordination is probably >>> required, >>> and that'd be a bad thing. >> >> Sorry, forgot to mention: I can't flip the default behaviour to >> leave >> virtualization support off only for the TK1. That's a generic >> default. > > Would enabling it in the compile but adding "bootm_boot_mode=sec" > to the > default environment (so it isn't used by default) be considered > sufficiently backwards compatible?
This turned out to not work as expected: booting in secure mode seems to prevent that Linux can bring up CPUs 1-3. Not sure if this is to be expected or a bug, but I will now take a different route:
That was the whole point of the environment variable suggestion; the environment variable would default to off so nobody got new behaviour, but anyone who wanted to boot in secure mode could simply set the environment variable and get it. That way, nobody who doesn't want the feature needs to co-ordinate U-Boot and kernel updates. Why doesn't that work?
Because it breaks SMP on Linux, namely the boot of secondary cores. Don't ask me why, I didn't debug the details. But you can probably reproduce by specifying bootm_boot_mode=sec with current U-boot and recent upstream kernels.
I suspect the environment variable isn't working, and Linux is still being booted in non-secure mode. That would be a bug in U-Boot.
Yes, might be. Linux reports being started (with that single CPU) in SVC mode. I'll recheck if it is a regression of my series.
Anyway, even if it worked, I think the cleaner way is flipping defaults at the config root level (virt support) as long as things are not working out of the box with Linux. That's what my other patch is doing now. Any concerns regarding it?
Yes.
This should be runtime configurable so that there's no need for a user to co-ordinate U-Boot and kernel updates. This also allows users to easily test the kernel in secure and NS mode without having to rebuild a bootloader.
Got it fixed. The bug was the unconditional execution of ap_pm_init in my series. Now I can freely switch between secure and non-secure mode during boot, and all CPUs will be brought up.
I will flip CONFIG_ARMV7_BOOT_SEC_DEFAULT to y for Tegra so that we keep the current behaviour (secure boot), the virt folks among us can tune to permanent non-sec boot via expert settings, and other can define bootm_boot_mode=nonsec for single experiments without rebuilds.
Jan
On Fri, 2015-04-17 at 07:57 -0600, Stephen Warren wrote:
On 04/17/2015 12:47 AM, Jan Kiszka wrote:
On 2015-04-14 16:30, Ian Campbell wrote:
Would enabling it in the compile but adding "bootm_boot_mode=sec" to the default environment (so it isn't used by default) be considered sufficiently backwards compatible?
This turned out to not work as expected: booting in secure mode seems to prevent that Linux can bring up CPUs 1-3. Not sure if this is to be expected or a bug, but I will now take a different route:
That was the whole point of the environment variable suggestion; the environment variable would default to off so nobody got new behaviour, but anyone who wanted to boot in secure mode could simply set the environment variable and get it. That way, nobody who doesn't want the feature needs to co-ordinate U-Boot and kernel updates. Why doesn't that work?
Right, if this doesn't work then it is certainly a bug somewhere on the u-boot side.
I fixed one such issue a while back ("ARM: bootm: do not add PSCI to fdt when booting in secure mode.") but that's ancient history, but I know that at that time it was working for me (I'm reasonably sure that was on Tegra with my earlier PSCI patches)
Ian.
Upstream Linux is broken with default configs when PSCI is enabled. So the user should explicitly enable virtualization, e.g. when she disabled CONFIG_CPU_IDLE in Linux (in which case it's safe to use). We can revert this workaround once Linux got fixed.
Signed-off-by: Jan Kiszka jan.kiszka@siemens.com --- arch/arm/cpu/armv7/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm/cpu/armv7/Kconfig b/arch/arm/cpu/armv7/Kconfig index 61e7c82..e5d4dcc 100644 --- a/arch/arm/cpu/armv7/Kconfig +++ b/arch/arm/cpu/armv7/Kconfig @@ -27,7 +27,7 @@ config ARMV7_BOOT_SEC_DEFAULT config ARMV7_VIRT boolean "Enable support for hardware virtualization" if EXPERT depends on CPU_V7_HAS_VIRT && ARMV7_NONSEC - default y + default y if !TEGRA ---help--- Say Y here to boot in hypervisor (HYP) mode when booting non-secure.
participants (5)
-
Ian Campbell -
Jan Kiszka -
Jan Kiszka -
Stephen Warren -
Tom Rini