[U-Boot] [PATCH v5 0/8] ARMv7: Add HYP mode switching support
(for GIT URL and Changelog see below)
ARM CPUs with the virtualization extension have a new mode called HYP mode, which allows hypervisors to safely control and monitor guests. The current hypervisor implementations (KVM and Xen) require the kernel to be entered in that HYP mode.
This patch series introduces a configuration variable CONFIG_ARMV7_VIRT which enables code to switch all cores into HYP mode. This is done automatically during execution of the bootm command.
The process of switching into HYP mode requires the CPU to be in secure state initially when entering u-boot, it will then setup some register and switch to non-secure state. This requires the GIC to be programmed properly first. Explanations about the details are in the commit messages of the respective patches.
The patches are structured like this: 1/8: prepare header file 2/8: add monitor handler (assembly) 3/8: add per CPU non-secure switch routine (assembly) 4/8: add system wide non-secure setup (C) 5/8: trigger non-secure switch during bootm command 6/8: add generic SMP functionality 7/8: add HYP mode switching 8/8: board specific code for ARM Versatile Express TC2
Since up to patch 6/8 this code works on non-virtualization capable CPUs also and as there has been a request, there is now a second configuration variable CONFIG_ARMV7_NONSEC, which omits the final HYP mode switch and just goes into non-secure SVC state. You can specify either (or none) of them, the code cares about the dependency.
The code aims to be as generic as possible, though currently it has only been tested on the Versatile Express TC-2 board. The last patch thus enables the feature for that board and should serve as an example for supporting other boards. I have had reports from different parties of successful adaptions to other boards as well (Arndale, Cubieboard, Chromebook), so I will create a hypmode-ports branch in the below repository to collect those patches and prepare them for upstreaming.
For convenience there is a GIT tree which you can pull these patches from ("hypmode_v5" branch): git://git.linaro.org/people/aprzywara/u-boot.git
Changes RFC..v1 * not a dedicated command anymore, code run by bootm & friends * protecting code with #ifdefs to avoid unnecessary inclusion and accidental crashing (when accessing restricted registers) * moving prototypes to header file to meet checkpatch recommendation * adding comment as proposed by Christoffer
Changes v1..v2 mostly style and code layout changes * restructure assembly code to live in a new file and not start.S * split smp, nonsec_init and hyp_init to be separate functions * used named constants from common header files * split C function to be more readable * extend comments to be more precise and elaborate * add provision to override GIC base address (needed for Arndale?) * add configuration variable to enable VExpress specific SMP code * use writel/readl for MMIO GIC accesses * remove superfluous isb instructions * more minor fixes
Changes v2..v3 * fix clobbering of GICC address actually spoiling the stack * do CNTFRQ setup in assembly per core (and not only once per SoC) * moving the new code files into arch/arm/cpu/armv7 * add config variable for doing non-secure switch only * use actual HYP and secure instructions mnemonics instead of the encoded byte sequence. This requires more recent compilers. * make the identification of the CPU core more robust and saner * use enum for error codes and rename them * lots of smaller layout and style fixes
Changes v3..v4 * mask reserved bits in CBAR register * move the VExpress board specific SMP code into the board directory * embed error reporting in the respective functions and getting rid of the error code enum at all (by popular demand ;-) * minor style fixes
Changes v4..v5: * remove unneeded padding in exception table * only clear unbanked interrupts in GIC distributor for non-secure * add a default weak implementation for the smp_waitloop() function, there only needs to be provided the address of the SMP pen now * fix compilation error with non-A15 VExpress boards * adding patch specific changelogs for v3..v4 and v4..v5
Please review, comment ... and commit ;-)
Contributions and comments to support other boards are welcome.
Andre Przywara (8): ARM: prepare armv7.h to be included from assembly source ARM: add secure monitor handler to switch to non-secure state ARM: add assembly routine to switch to non-secure state ARM: add C function to switch to non-secure state ARM: trigger non-secure state switch during bootm execution ARM: add SMP support for non-secure switch ARM: extend non-secure switch to also go into HYP mode ARM: VExpress: enable ARMv7 virt support for VExpress A15
arch/arm/cpu/armv7/Makefile | 5 + arch/arm/cpu/armv7/nonsec_virt.S | 208 ++++++++++++++++++++++++++++++++ arch/arm/cpu/armv7/virt-v7.c | 173 ++++++++++++++++++++++++++ arch/arm/include/asm/armv7.h | 33 ++++- arch/arm/include/asm/gic.h | 19 +++ arch/arm/lib/bootm.c | 18 +++ board/armltd/vexpress/vexpress_common.c | 15 +++ include/common.h | 2 + include/configs/vexpress_ca15_tc2.h | 5 +- 9 files changed, 476 insertions(+), 2 deletions(-) create mode 100644 arch/arm/cpu/armv7/nonsec_virt.S create mode 100644 arch/arm/cpu/armv7/virt-v7.c create mode 100644 arch/arm/include/asm/gic.h
armv7.h contains some useful constants, but also C prototypes. To include it also in assembly files, protect the non-assembly part appropriately.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/include/asm/armv7.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
Changes: v3..v4: none v4..v5: none
diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h index 392d6a2..0f7cbbf 100644 --- a/arch/arm/include/asm/armv7.h +++ b/arch/arm/include/asm/armv7.h @@ -7,7 +7,6 @@ */ #ifndef ARMV7_H #define ARMV7_H -#include <linux/types.h>
/* Cortex-A9 revisions */ #define MIDR_CORTEX_A9_R0P1 0x410FC091 @@ -41,6 +40,9 @@ #define ARMV7_CLIDR_CTYPE_INSTRUCTION_DATA 3 #define ARMV7_CLIDR_CTYPE_UNIFIED 4
+#ifndef __ASSEMBLY__ +#include <linux/types.h> + /* * CP15 Barrier instructions * Please note that we have separate barrier instructions in ARMv7 @@ -58,4 +60,6 @@ void v7_outer_cache_inval_all(void); void v7_outer_cache_flush_range(u32 start, u32 end); void v7_outer_cache_inval_range(u32 start, u32 end);
+#endif /* ! __ASSEMBLY__ */ + #endif
A prerequisite for using virtualization is to be in HYP mode, which requires the CPU to be in non-secure state first. Add a new file in arch/arm/cpu/armv7 to hold a monitor handler routine which switches the CPU to non-secure state by setting the NS and associated bits. According to the ARM architecture reference manual this should not be done in SVC mode, so we have to setup a SMC handler for this. We create a new vector table to avoid interference with other boards. The MVBAR register will be programmed later just before the smc call.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/cpu/armv7/Makefile | 4 +++ arch/arm/cpu/armv7/nonsec_virt.S | 54 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100644 arch/arm/cpu/armv7/nonsec_virt.S
Changes: v3..v4: clarify comments, w/s fixes v4..v5: remove unneeded padding in the exception table
diff --git a/arch/arm/cpu/armv7/Makefile b/arch/arm/cpu/armv7/Makefile index b723e22..3466c7a 100644 --- a/arch/arm/cpu/armv7/Makefile +++ b/arch/arm/cpu/armv7/Makefile @@ -20,6 +20,10 @@ ifneq ($(CONFIG_AM43XX)$(CONFIG_AM33XX)$(CONFIG_OMAP44XX)$(CONFIG_OMAP54XX)$(CON SOBJS += lowlevel_init.o endif
+ifneq ($(CONFIG_ARMV7_NONSEC),) +SOBJS += nonsec_virt.o +endif + SRCS := $(START:.o=.S) $(COBJS:.o=.c) OBJS := $(addprefix $(obj),$(COBJS) $(SOBJS)) START := $(addprefix $(obj),$(START)) diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S new file mode 100644 index 0000000..c21bca3 --- /dev/null +++ b/arch/arm/cpu/armv7/nonsec_virt.S @@ -0,0 +1,54 @@ +/* + * code for switching cores into non-secure state + * + * Copyright (c) 2013 Andre Przywara andre.przywara@linaro.org + * + * See file CREDITS for list of people who contributed to this + * project. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, + * MA 02111-1307 USA + */ + +#include <config.h> + +/* the vector table for secure state */ +_monitor_vectors: + .word 0 /* reset */ + .word 0 /* undef */ + adr pc, _secure_monitor + .word 0 + .word 0 + .word 0 + .word 0 + .word 0 + +/* + * secure monitor handler + * U-boot calls this "software interrupt" in start.S + * This is executed on a "smc" instruction, we use a "smc #0" to switch + * to non-secure state. + * We use only r0 and r1 here, due to constraints in the caller. + */ + .align 5 +_secure_monitor: + mrc p15, 0, r1, c1, c1, 0 @ read SCR + bic r1, r1, #0x4e @ clear IRQ, FIQ, EA, nET bits + orr r1, r1, #0x31 @ enable NS, AW, FW bits + + mcr p15, 0, r1, c1, c1, 0 @ write SCR (with NS bit set) + + movs pc, lr @ return to non-secure SVC +
Just checking, is the mcr p15,0,r1,c1,c1,0 in sync with the following text . I could be wrong here, just checking
B1.5.1 Arm Arch Ref Manual
-
To avoid security holes, software must not: -
— Change from Secure to Non-secure state by using an MSR or CPS instruction to switch from Monitor
mode to some other mode while SCR.NS is 1. -
— Use an MCR instruction that writes SCR.NS to change from Secure to Non-secure state. This means ARM recommends that software does not alter SCR.NS in any mode except Monitor mode. ARM deprecates changing SCR.NS in any other mode.
On Thu, Sep 19, 2013 at 9:36 PM, Andre Przywara andre.przywara@linaro.orgwrote:
A prerequisite for using virtualization is to be in HYP mode, which requires the CPU to be in non-secure state first. Add a new file in arch/arm/cpu/armv7 to hold a monitor handler routine which switches the CPU to non-secure state by setting the NS and associated bits. According to the ARM architecture reference manual this should not be done in SVC mode, so we have to setup a SMC handler for this. We create a new vector table to avoid interference with other boards. The MVBAR register will be programmed later just before the smc call.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/Makefile | 4 +++ arch/arm/cpu/armv7/nonsec_virt.S | 54 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100644 arch/arm/cpu/armv7/nonsec_virt.S
Changes: v3..v4: clarify comments, w/s fixes v4..v5: remove unneeded padding in the exception table
diff --git a/arch/arm/cpu/armv7/Makefile b/arch/arm/cpu/armv7/Makefile index b723e22..3466c7a 100644 --- a/arch/arm/cpu/armv7/Makefile +++ b/arch/arm/cpu/armv7/Makefile @@ -20,6 +20,10 @@ ifneq ($(CONFIG_AM43XX)$(CONFIG_AM33XX)$(CONFIG_OMAP44XX)$(CONFIG_OMAP54XX)$(CON SOBJS += lowlevel_init.o endif
+ifneq ($(CONFIG_ARMV7_NONSEC),) +SOBJS += nonsec_virt.o +endif
SRCS := $(START:.o=.S) $(COBJS:.o=.c) OBJS := $(addprefix $(obj),$(COBJS) $(SOBJS)) START := $(addprefix $(obj),$(START)) diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S new file mode 100644 index 0000000..c21bca3 --- /dev/null +++ b/arch/arm/cpu/armv7/nonsec_virt.S @@ -0,0 +1,54 @@ +/*
- code for switching cores into non-secure state
- Copyright (c) 2013 Andre Przywara andre.przywara@linaro.org
- See file CREDITS for list of people who contributed to this
- project.
- This program is free software; you can redistribute it and/or
- modify it under the terms of the GNU General Public License as
- published by the Free Software Foundation; either version 2 of
- the License, or (at your option) any later version.
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
- You should have received a copy of the GNU General Public License
- along with this program; if not, write to the Free Software
- Foundation, Inc., 59 Temple Place, Suite 330, Boston,
- MA 02111-1307 USA
- */
+#include <config.h>
+/* the vector table for secure state */ +_monitor_vectors:
.word 0 /* reset */.word 0 /* undef */adr pc, _secure_monitor.word 0.word 0.word 0.word 0.word 0+/*
- secure monitor handler
- U-boot calls this "software interrupt" in start.S
- This is executed on a "smc" instruction, we use a "smc #0" to switch
- to non-secure state.
- We use only r0 and r1 here, due to constraints in the caller.
- */
.align 5+_secure_monitor:
mrc p15, 0, r1, c1, c1, 0 @ read SCRbic r1, r1, #0x4e @ clear IRQ, FIQ, EA, nETbits
orr r1, r1, #0x31 @ enable NS, AW, FW bitsmcr p15, 0, r1, c1, c1, 0 @ write SCR (with NS bitset)
movs pc, lr @ return to non-secure SVC-- 1.7.12.1
U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
On Fri, Sep 20, 2013 at 03:20:15AM +0530, Mj Embd wrote:
Just checking, is the mcr p15,0,r1,c1,c1,0 in sync with the following text . I could be wrong here, just checking
In the future, if you can comment specifically inline on the lines of code you are targeting, it is easier for other people to address your concerns.
B1.5.1 Arm Arch Ref Manual
To avoid security holes, software must not: -
— Change from Secure to Non-secure state by using an MSR or CPSinstruction to switch from Monitor
The important part here is that we don't change from S to NS by modifying the SCR, because monitor mode is always in secure mode, so the change only happens on the exception return.
So yes, it's safe.
-Christoffer
mode to some other mode while SCR.NS is 1. - — Use an MCR instruction that writes SCR.NS to change from Secure to Non-secure state. This means ARM recommends that software does not alter SCR.NS in any mode except Monitor mode. ARM deprecates changing SCR.NS in any other mode.On Thu, Sep 19, 2013 at 9:36 PM, Andre Przywara andre.przywara@linaro.orgwrote:
A prerequisite for using virtualization is to be in HYP mode, which requires the CPU to be in non-secure state first. Add a new file in arch/arm/cpu/armv7 to hold a monitor handler routine which switches the CPU to non-secure state by setting the NS and associated bits. According to the ARM architecture reference manual this should not be done in SVC mode, so we have to setup a SMC handler for this. We create a new vector table to avoid interference with other boards. The MVBAR register will be programmed later just before the smc call.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/Makefile | 4 +++ arch/arm/cpu/armv7/nonsec_virt.S | 54 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100644 arch/arm/cpu/armv7/nonsec_virt.S
Changes: v3..v4: clarify comments, w/s fixes v4..v5: remove unneeded padding in the exception table
diff --git a/arch/arm/cpu/armv7/Makefile b/arch/arm/cpu/armv7/Makefile index b723e22..3466c7a 100644 --- a/arch/arm/cpu/armv7/Makefile +++ b/arch/arm/cpu/armv7/Makefile @@ -20,6 +20,10 @@ ifneq ($(CONFIG_AM43XX)$(CONFIG_AM33XX)$(CONFIG_OMAP44XX)$(CONFIG_OMAP54XX)$(CON SOBJS += lowlevel_init.o endif
+ifneq ($(CONFIG_ARMV7_NONSEC),) +SOBJS += nonsec_virt.o +endif
SRCS := $(START:.o=.S) $(COBJS:.o=.c) OBJS := $(addprefix $(obj),$(COBJS) $(SOBJS)) START := $(addprefix $(obj),$(START)) diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S new file mode 100644 index 0000000..c21bca3 --- /dev/null +++ b/arch/arm/cpu/armv7/nonsec_virt.S @@ -0,0 +1,54 @@ +/*
- code for switching cores into non-secure state
- Copyright (c) 2013 Andre Przywara andre.przywara@linaro.org
- See file CREDITS for list of people who contributed to this
- project.
- This program is free software; you can redistribute it and/or
- modify it under the terms of the GNU General Public License as
- published by the Free Software Foundation; either version 2 of
- the License, or (at your option) any later version.
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
- You should have received a copy of the GNU General Public License
- along with this program; if not, write to the Free Software
- Foundation, Inc., 59 Temple Place, Suite 330, Boston,
- MA 02111-1307 USA
- */
+#include <config.h>
+/* the vector table for secure state */ +_monitor_vectors:
.word 0 /* reset */.word 0 /* undef */adr pc, _secure_monitor.word 0.word 0.word 0.word 0.word 0+/*
- secure monitor handler
- U-boot calls this "software interrupt" in start.S
- This is executed on a "smc" instruction, we use a "smc #0" to switch
- to non-secure state.
- We use only r0 and r1 here, due to constraints in the caller.
- */
.align 5+_secure_monitor:
mrc p15, 0, r1, c1, c1, 0 @ read SCRbic r1, r1, #0x4e @ clear IRQ, FIQ, EA, nETbits
orr r1, r1, #0x31 @ enable NS, AW, FW bitsmcr p15, 0, r1, c1, c1, 0 @ write SCR (with NS bitset)
movs pc, lr @ return to non-secure SVC-- 1.7.12.1
U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
-- -mj
On Fri, Sep 20, 2013 at 6:12 AM, Christoffer Dall < christoffer.dall@linaro.org> wrote:
On Fri, Sep 20, 2013 at 03:20:15AM +0530, Mj Embd wrote:
Just checking, is the mcr p15,0,r1,c1,c1,0 in sync with the following
text
. I could be wrong here, just checking
In the future, if you can comment specifically inline on the lines of code you are targeting, it is easier for other people to address your concerns.
B1.5.1 Arm Arch Ref Manual
To avoid security holes, software must not: -
— Change from Secure to Non-secure state by using an MSR or CPSinstruction to switch from Monitor
The important part here is that we don't change from S to NS by modifying the SCR, because monitor mode is always in secure mode, so the change only happens on the exception return.
So yes, it's safe.
-Christoffer
Ok. Good Discussion. Thanks, PS: Gmail auto wraps the previous msg in 3 dots, so sometimes I miss inlining. Thanks for pointing out.
mode to some other mode while SCR.NS is 1. - — Use an MCR instruction that writes SCR.NS to change from Secureto
Non-secure state. This means ARM recommends that software does notalter
SCR.NS in any mode except Monitor mode. ARM deprecates changingSCR.NS
in any other mode.On Thu, Sep 19, 2013 at 9:36 PM, Andre Przywara andre.przywara@linaro.orgwrote:
A prerequisite for using virtualization is to be in HYP mode, which requires the CPU to be in non-secure state first. Add a new file in arch/arm/cpu/armv7 to hold a monitor handler routine which switches the CPU to non-secure state by setting the NS and associated bits. According to the ARM architecture reference manual this should not be done in SVC mode, so we have to setup a SMC handler for this. We create a new vector table to avoid interference with other boards. The MVBAR register will be programmed later just before the smc call.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/Makefile | 4 +++ arch/arm/cpu/armv7/nonsec_virt.S | 54 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100644 arch/arm/cpu/armv7/nonsec_virt.S
Changes: v3..v4: clarify comments, w/s fixes v4..v5: remove unneeded padding in the exception table
diff --git a/arch/arm/cpu/armv7/Makefile b/arch/arm/cpu/armv7/Makefile index b723e22..3466c7a 100644 --- a/arch/arm/cpu/armv7/Makefile +++ b/arch/arm/cpu/armv7/Makefile @@ -20,6 +20,10 @@ ifneq
($(CONFIG_AM43XX)$(CONFIG_AM33XX)$(CONFIG_OMAP44XX)$(CONFIG_OMAP54XX)$(CON
SOBJS += lowlevel_init.o endif
+ifneq ($(CONFIG_ARMV7_NONSEC),) +SOBJS += nonsec_virt.o +endif
SRCS := $(START:.o=.S) $(COBJS:.o=.c) OBJS := $(addprefix $(obj),$(COBJS) $(SOBJS)) START := $(addprefix $(obj),$(START)) diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S new file mode 100644 index 0000000..c21bca3 --- /dev/null +++ b/arch/arm/cpu/armv7/nonsec_virt.S @@ -0,0 +1,54 @@ +/*
- code for switching cores into non-secure state
- Copyright (c) 2013 Andre Przywara andre.przywara@linaro.org
- See file CREDITS for list of people who contributed to this
- project.
- This program is free software; you can redistribute it and/or
- modify it under the terms of the GNU General Public License as
- published by the Free Software Foundation; either version 2 of
- the License, or (at your option) any later version.
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
the
- GNU General Public License for more details.
- You should have received a copy of the GNU General Public License
- along with this program; if not, write to the Free Software
- Foundation, Inc., 59 Temple Place, Suite 330, Boston,
- MA 02111-1307 USA
- */
+#include <config.h>
+/* the vector table for secure state */ +_monitor_vectors:
.word 0 /* reset */.word 0 /* undef */adr pc, _secure_monitor.word 0.word 0.word 0.word 0.word 0+/*
- secure monitor handler
- U-boot calls this "software interrupt" in start.S
- This is executed on a "smc" instruction, we use a "smc #0" to
switch
- to non-secure state.
- We use only r0 and r1 here, due to constraints in the caller.
- */
.align 5+_secure_monitor:
mrc p15, 0, r1, c1, c1, 0 @ read SCRbic r1, r1, #0x4e @ clear IRQ, FIQ, EA,nET
bits
orr r1, r1, #0x31 @ enable NS, AW, FWbits
mcr p15, 0, r1, c1, c1, 0 @ write SCR (with NSbit
set)
movs pc, lr @ return to non-secureSVC
-- 1.7.12.1
U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
-- -mj
-- Christoffer
On Fri, Sep 20, 2013 at 08:08:45AM +0530, Mj Embd wrote:
On Fri, Sep 20, 2013 at 6:12 AM, Christoffer Dall < christoffer.dall@linaro.org> wrote:
On Fri, Sep 20, 2013 at 03:20:15AM +0530, Mj Embd wrote:
Just checking, is the mcr p15,0,r1,c1,c1,0 in sync with the following
text
. I could be wrong here, just checking
In the future, if you can comment specifically inline on the lines of code you are targeting, it is easier for other people to address your concerns.
B1.5.1 Arm Arch Ref Manual
To avoid security holes, software must not: -
— Change from Secure to Non-secure state by using an MSR or CPSinstruction to switch from Monitor
The important part here is that we don't change from S to NS by modifying the SCR, because monitor mode is always in secure mode, so the change only happens on the exception return.
So yes, it's safe.
-Christoffer
Ok. Good Discussion. Thanks, PS: Gmail auto wraps the previous msg in 3 dots, so sometimes I miss inlining. Thanks for pointing out.
No problem, thanks for looking at the code.
-Christoffer
Hi Andre,
On Thu, 19 Sep 2013 18:06:40 +0200, Andre Przywara andre.przywara@linaro.org wrote:
A prerequisite for using virtualization is to be in HYP mode, which requires the CPU to be in non-secure state first. Add a new file in arch/arm/cpu/armv7 to hold a monitor handler routine which switches the CPU to non-secure state by setting the NS and associated bits. According to the ARM architecture reference manual this should not be done in SVC mode, so we have to setup a SMC handler for this. We create a new vector table to avoid interference with other boards. The MVBAR register will be programmed later just before the smc call.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/Makefile | 4 +++ arch/arm/cpu/armv7/nonsec_virt.S | 54 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100644 arch/arm/cpu/armv7/nonsec_virt.S
BTW, this has an "empty line at EOF" warning when applying.
Amicalement,
Hi, experts: I found Linaro engineers had pushed some secure monitor related code. But not find these code in 2013.10.rc4 . So, how to get these latest code?
Best wishes,
Hi TigerLiu@viatech.com.cn,
On Sat, 12 Oct 2013 17:27:30 +0800, TigerLiu@viatech.com.cn wrote:
Hi, experts: I found Linaro engineers had pushed some secure monitor related code. But not find these code in 2013.10.rc4 . So, how to get these latest code?
As always: by submitting them, or having someone submit them, to the mainline U-Boot project.
Best wishes,
Amicalement,
Hi, Albert: Thanks ! I used cmd "git clone git://git.denx.de/u-boot.git " to get the master source code.
Best wishes,
Hi TigerLiu@viatech.com.cn,
On Mon, 14 Oct 2013 10:14:44 +0800, TigerLiu@viatech.com.cn wrote:
Hi, Albert: Thanks ! I used cmd "git clone git://git.denx.de/u-boot.git " to get the master source code.
My apologies: I though you had meant "how to get the submission to go into mainline", while you actually meant "how to get accepted submissions" locally.
Let me just add that if you want to (ir)regularly find out which code has been accepted into mainline, there is no need to re-clone the whole repository each time. Clone it once, then 'git fetch origin' whenever you want to re-sync, that will only transfer what's new.
Best wishes,
Amicalement,
Hi, Albert:
My apologies: I though you had meant "how to get the submission to go into mainline", while you actually meant "how to get accepted submissions" locally.
Let me just add that if you want to (ir)regularly find out which code has been accepted into mainline, there is no need to re-clone the whole repository each time. Clone it once, then 'git fetch origin' whenever you want to re-sync, that will only transfer what's new.
Thanks a lot!
Best wishes,
While actually switching to non-secure state is one thing, another part of this process is to make sure that we still have full access to the interrupt controller (GIC). The GIC is fully aware of secure vs. non-secure state, some registers are banked, others may be configured to be accessible from secure state only. To be as generic as possible, we get the GIC memory mapped address based on the PERIPHBASE value in the CBAR register. Since this register is not architecturally defined, we check the MIDR before to be from an A15 or A7. For CPUs not having the CBAR or boards with wrong information herein we allow providing the base address as a configuration variable.
Now that we know the GIC address, we: a) allow private interrupts to be delivered to the core (GICD_IGROUPR0 = 0xFFFFFFFF) b) enable the CPU interface (GICC_CTLR[0] = 1) c) set the priority filter to allow non-secure interrupts (GICC_PMR = 0xFF)
Also we allow access to all coprocessor interfaces from non-secure state by writing the appropriate bits in the NSACR register.
The generic timer base frequency register is only accessible from secure state, so we have to program it now. Actually this should be done from primary firmware before, but some boards seems to omit this, so if needed we do this here with a board specific value. The Versatile Express board does not need this, so we remove the frequency from the configuration file here.
After having switched to non-secure state, we also enable the non-secure GIC CPU interface, since this register is banked.
Since we need to call this routine also directly from the smp_pen later (where we don't have any stack), we can only use caller saved registers r0-r3 and r12 to not mess with the compiler.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/cpu/armv7/nonsec_virt.S | 86 +++++++++++++++++++++++++++++++++++++ arch/arm/include/asm/armv7.h | 21 +++++++++ arch/arm/include/asm/gic.h | 17 ++++++++ include/configs/vexpress_ca15_tc2.h | 2 - 4 files changed, 124 insertions(+), 2 deletions(-) create mode 100644 arch/arm/include/asm/gic.h
Changes: v3..v4: clear reserved bits in CBAR v4..v5: none
diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S index c21bca3..3dd60b7 100644 --- a/arch/arm/cpu/armv7/nonsec_virt.S +++ b/arch/arm/cpu/armv7/nonsec_virt.S @@ -23,6 +23,11 @@ */
#include <config.h> +#include <linux/linkage.h> +#include <asm/gic.h> +#include <asm/armv7.h> + +.arch_extension sec
/* the vector table for secure state */ _monitor_vectors: @@ -52,3 +57,84 @@ _secure_monitor:
movs pc, lr @ return to non-secure SVC
+/* + * Switch a core to non-secure state. + * + * 1. initialize the GIC per-core interface + * 2. allow coprocessor access in non-secure modes + * 3. switch the cpu mode (by calling "smc #0") + * + * Called from smp_pen by secondary cores and directly by the BSP. + * Do not assume that the stack is available and only use registers + * r0-r3 and r12. + * + * PERIPHBASE is used to get the GIC address. This could be 40 bits long, + * though, but we check this in C before calling this function. + */ +ENTRY(_nonsec_init) +#ifdef CONFIG_ARM_GIC_BASE_ADDRESS + ldr r2, =CONFIG_ARM_GIC_BASE_ADDRESS +#else + mrc p15, 4, r2, c15, c0, 0 @ read CBAR + bfc r2, #0, #15 @ clear reserved bits +#endif + add r3, r2, #GIC_DIST_OFFSET @ GIC dist i/f offset + mvn r1, #0 @ all bits to 1 + str r1, [r3, #GICD_IGROUPRn] @ allow private interrupts + + mrc p15, 0, r0, c0, c0, 0 @ read MIDR + ldr r1, =MIDR_PRIMARY_PART_MASK + and r0, r0, r1 @ mask out variant and revision + + ldr r1, =MIDR_CORTEX_A7_R0P0 & MIDR_PRIMARY_PART_MASK + cmp r0, r1 @ check for Cortex-A7 + + ldr r1, =MIDR_CORTEX_A15_R0P0 & MIDR_PRIMARY_PART_MASK + cmpne r0, r1 @ check for Cortex-A15 + + movne r1, #GIC_CPU_OFFSET_A9 @ GIC CPU offset for A9 + moveq r1, #GIC_CPU_OFFSET_A15 @ GIC CPU offset for A15/A7 + add r3, r2, r1 @ r3 = GIC CPU i/f addr + + mov r1, #1 @ set GICC_CTLR[enable] + str r1, [r3, #GICC_CTLR] @ and clear all other bits + mov r1, #0xff + str r1, [r3, #GICC_PMR] @ set priority mask register + + movw r1, #0x3fff + movt r1, #0x0006 + mcr p15, 0, r1, c1, c1, 2 @ NSACR = all copros to non-sec + +/* The CNTFRQ register of the generic timer needs to be + * programmed in secure state. Some primary bootloaders / firmware + * omit this, so if the frequency is provided in the configuration, + * we do this here instead. + * But first check if we have the generic timer. + */ +#ifdef CONFIG_SYS_CLK_FREQ + mrc p15, 0, r0, c0, c1, 1 @ read ID_PFR1 + and r0, r0, #CPUID_ARM_GENTIMER_MASK @ mask arch timer bits + cmp r0, #(1 << CPUID_ARM_GENTIMER_SHIFT) + ldreq r1, =CONFIG_SYS_CLK_FREQ + mcreq p15, 0, r1, c14, c0, 0 @ write CNTFRQ +#endif + + adr r1, _monitor_vectors + mcr p15, 0, r1, c12, c0, 1 @ set MVBAR to secure vectors + + mrc p15, 0, ip, c12, c0, 0 @ save secure copy of VBAR + + isb + smc #0 @ call into MONITOR mode + + mcr p15, 0, ip, c12, c0, 0 @ write non-secure copy of VBAR + + mov r1, #1 + str r1, [r3, #GICC_CTLR] @ enable non-secure CPU i/f + add r2, r2, #GIC_DIST_OFFSET + str r1, [r2, #GICD_CTLR] @ allow private interrupts + + mov r0, r3 @ return GICC address + + bx lr +ENDPROC(_nonsec_init) diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h index 0f7cbbf..3dcfc8f 100644 --- a/arch/arm/include/asm/armv7.h +++ b/arch/arm/include/asm/armv7.h @@ -18,6 +18,22 @@ #define MIDR_CORTEX_A15_R0P0 0x410FC0F0 #define MIDR_CORTEX_A15_R2P2 0x412FC0F2
+/* Cortex-A7 revisions */ +#define MIDR_CORTEX_A7_R0P0 0x410FC070 + +#define MIDR_PRIMARY_PART_MASK 0xFF0FFFF0 + +/* ID_PFR1 feature fields */ +#define CPUID_ARM_SEC_SHIFT 4 +#define CPUID_ARM_SEC_MASK (0xF << CPUID_ARM_SEC_SHIFT) +#define CPUID_ARM_VIRT_SHIFT 12 +#define CPUID_ARM_VIRT_MASK (0xF << CPUID_ARM_VIRT_SHIFT) +#define CPUID_ARM_GENTIMER_SHIFT 16 +#define CPUID_ARM_GENTIMER_MASK (0xF << CPUID_ARM_GENTIMER_SHIFT) + +/* valid bits in CBAR register / PERIPHBASE value */ +#define CBAR_MASK 0xFFFF8000 + /* CCSIDR */ #define CCSIDR_LINE_SIZE_OFFSET 0 #define CCSIDR_LINE_SIZE_MASK 0x7 @@ -60,6 +76,11 @@ void v7_outer_cache_inval_all(void); void v7_outer_cache_flush_range(u32 start, u32 end); void v7_outer_cache_inval_range(u32 start, u32 end);
+#ifdef CONFIG_ARMV7_NONSEC +/* defined in assembly file */ +unsigned int _nonsec_init(void); +#endif /* CONFIG_ARMV7_NONSEC */ + #endif /* ! __ASSEMBLY__ */
#endif diff --git a/arch/arm/include/asm/gic.h b/arch/arm/include/asm/gic.h new file mode 100644 index 0000000..c2b1e28 --- /dev/null +++ b/arch/arm/include/asm/gic.h @@ -0,0 +1,17 @@ +#ifndef __GIC_V2_H__ +#define __GIC_V2_H__ + +/* register offsets for the ARM generic interrupt controller (GIC) */ + +#define GIC_DIST_OFFSET 0x1000 +#define GICD_CTLR 0x0000 +#define GICD_TYPER 0x0004 +#define GICD_IGROUPRn 0x0080 +#define GICD_SGIR 0x0F00 + +#define GIC_CPU_OFFSET_A9 0x0100 +#define GIC_CPU_OFFSET_A15 0x2000 +#define GICC_CTLR 0x0000 +#define GICC_PMR 0x0004 + +#endif diff --git a/include/configs/vexpress_ca15_tc2.h b/include/configs/vexpress_ca15_tc2.h index d1431e5..89ce1c7 100644 --- a/include/configs/vexpress_ca15_tc2.h +++ b/include/configs/vexpress_ca15_tc2.h @@ -15,6 +15,4 @@ #include "vexpress_common.h" #define CONFIG_BOOTP_VCI_STRING "U-boot.armv7.vexpress_ca15x2_tc2"
-#define CONFIG_SYS_CLK_FREQ 24000000 - #endif
Hi Andre,
There is another approach taken in xen. (xen/arch/arm/mode_switch.S) Which do you think is the better approach
Regards -mj
On 9/19/13, Andre Przywara andre.przywara@linaro.org wrote:
While actually switching to non-secure state is one thing, another part of this process is to make sure that we still have full access to the interrupt controller (GIC). The GIC is fully aware of secure vs. non-secure state, some registers are banked, others may be configured to be accessible from secure state only. To be as generic as possible, we get the GIC memory mapped address based on the PERIPHBASE value in the CBAR register. Since this register is not architecturally defined, we check the MIDR before to be from an A15 or A7. For CPUs not having the CBAR or boards with wrong information herein we allow providing the base address as a configuration variable.
Now that we know the GIC address, we: a) allow private interrupts to be delivered to the core (GICD_IGROUPR0 = 0xFFFFFFFF) b) enable the CPU interface (GICC_CTLR[0] = 1) c) set the priority filter to allow non-secure interrupts (GICC_PMR = 0xFF)
Also we allow access to all coprocessor interfaces from non-secure state by writing the appropriate bits in the NSACR register.
The generic timer base frequency register is only accessible from secure state, so we have to program it now. Actually this should be done from primary firmware before, but some boards seems to omit this, so if needed we do this here with a board specific value. The Versatile Express board does not need this, so we remove the frequency from the configuration file here.
After having switched to non-secure state, we also enable the non-secure GIC CPU interface, since this register is banked.
Since we need to call this routine also directly from the smp_pen later (where we don't have any stack), we can only use caller saved registers r0-r3 and r12 to not mess with the compiler.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/nonsec_virt.S | 86 +++++++++++++++++++++++++++++++++++++ arch/arm/include/asm/armv7.h | 21 +++++++++ arch/arm/include/asm/gic.h | 17 ++++++++ include/configs/vexpress_ca15_tc2.h | 2 - 4 files changed, 124 insertions(+), 2 deletions(-) create mode 100644 arch/arm/include/asm/gic.h
Changes: v3..v4: clear reserved bits in CBAR v4..v5: none
diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S index c21bca3..3dd60b7 100644 --- a/arch/arm/cpu/armv7/nonsec_virt.S +++ b/arch/arm/cpu/armv7/nonsec_virt.S @@ -23,6 +23,11 @@ */
#include <config.h> +#include <linux/linkage.h> +#include <asm/gic.h> +#include <asm/armv7.h>
+.arch_extension sec
/* the vector table for secure state */ _monitor_vectors: @@ -52,3 +57,84 @@ _secure_monitor:
movs pc, lr @ return to non-secure SVC
+/*
- Switch a core to non-secure state.
- initialize the GIC per-core interface
- allow coprocessor access in non-secure modes
- switch the cpu mode (by calling "smc #0")
- Called from smp_pen by secondary cores and directly by the BSP.
- Do not assume that the stack is available and only use registers
- r0-r3 and r12.
- PERIPHBASE is used to get the GIC address. This could be 40 bits long,
- though, but we check this in C before calling this function.
- */
+ENTRY(_nonsec_init) +#ifdef CONFIG_ARM_GIC_BASE_ADDRESS
- ldr r2, =CONFIG_ARM_GIC_BASE_ADDRESS
+#else
- mrc p15, 4, r2, c15, c0, 0 @ read CBAR
- bfc r2, #0, #15 @ clear reserved bits
+#endif
- add r3, r2, #GIC_DIST_OFFSET @ GIC dist i/f offset
- mvn r1, #0 @ all bits to 1
- str r1, [r3, #GICD_IGROUPRn] @ allow private interrupts
- mrc p15, 0, r0, c0, c0, 0 @ read MIDR
- ldr r1, =MIDR_PRIMARY_PART_MASK
- and r0, r0, r1 @ mask out variant and revision
- ldr r1, =MIDR_CORTEX_A7_R0P0 & MIDR_PRIMARY_PART_MASK
- cmp r0, r1 @ check for Cortex-A7
- ldr r1, =MIDR_CORTEX_A15_R0P0 & MIDR_PRIMARY_PART_MASK
- cmpne r0, r1 @ check for Cortex-A15
- movne r1, #GIC_CPU_OFFSET_A9 @ GIC CPU offset for A9
- moveq r1, #GIC_CPU_OFFSET_A15 @ GIC CPU offset for A15/A7
- add r3, r2, r1 @ r3 = GIC CPU i/f addr
- mov r1, #1 @ set GICC_CTLR[enable]
- str r1, [r3, #GICC_CTLR] @ and clear all other bits
- mov r1, #0xff
- str r1, [r3, #GICC_PMR] @ set priority mask register
- movw r1, #0x3fff
- movt r1, #0x0006
- mcr p15, 0, r1, c1, c1, 2 @ NSACR = all copros to non-sec
+/* The CNTFRQ register of the generic timer needs to be
- programmed in secure state. Some primary bootloaders / firmware
- omit this, so if the frequency is provided in the configuration,
- we do this here instead.
- But first check if we have the generic timer.
- */
+#ifdef CONFIG_SYS_CLK_FREQ
- mrc p15, 0, r0, c0, c1, 1 @ read ID_PFR1
- and r0, r0, #CPUID_ARM_GENTIMER_MASK @ mask arch timer bits
- cmp r0, #(1 << CPUID_ARM_GENTIMER_SHIFT)
- ldreq r1, =CONFIG_SYS_CLK_FREQ
- mcreq p15, 0, r1, c14, c0, 0 @ write CNTFRQ
+#endif
- adr r1, _monitor_vectors
- mcr p15, 0, r1, c12, c0, 1 @ set MVBAR to secure vectors
- mrc p15, 0, ip, c12, c0, 0 @ save secure copy of VBAR
- isb
- smc #0 @ call into MONITOR mode
- mcr p15, 0, ip, c12, c0, 0 @ write non-secure copy of VBAR
- mov r1, #1
- str r1, [r3, #GICC_CTLR] @ enable non-secure CPU i/f
- add r2, r2, #GIC_DIST_OFFSET
- str r1, [r2, #GICD_CTLR] @ allow private interrupts
- mov r0, r3 @ return GICC address
- bx lr
+ENDPROC(_nonsec_init) diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h index 0f7cbbf..3dcfc8f 100644 --- a/arch/arm/include/asm/armv7.h +++ b/arch/arm/include/asm/armv7.h @@ -18,6 +18,22 @@ #define MIDR_CORTEX_A15_R0P0 0x410FC0F0 #define MIDR_CORTEX_A15_R2P2 0x412FC0F2
+/* Cortex-A7 revisions */ +#define MIDR_CORTEX_A7_R0P0 0x410FC070
+#define MIDR_PRIMARY_PART_MASK 0xFF0FFFF0
+/* ID_PFR1 feature fields */ +#define CPUID_ARM_SEC_SHIFT 4 +#define CPUID_ARM_SEC_MASK (0xF << CPUID_ARM_SEC_SHIFT) +#define CPUID_ARM_VIRT_SHIFT 12 +#define CPUID_ARM_VIRT_MASK (0xF << CPUID_ARM_VIRT_SHIFT) +#define CPUID_ARM_GENTIMER_SHIFT 16 +#define CPUID_ARM_GENTIMER_MASK (0xF << CPUID_ARM_GENTIMER_SHIFT)
+/* valid bits in CBAR register / PERIPHBASE value */ +#define CBAR_MASK 0xFFFF8000
/* CCSIDR */ #define CCSIDR_LINE_SIZE_OFFSET 0 #define CCSIDR_LINE_SIZE_MASK 0x7 @@ -60,6 +76,11 @@ void v7_outer_cache_inval_all(void); void v7_outer_cache_flush_range(u32 start, u32 end); void v7_outer_cache_inval_range(u32 start, u32 end);
+#ifdef CONFIG_ARMV7_NONSEC +/* defined in assembly file */ +unsigned int _nonsec_init(void); +#endif /* CONFIG_ARMV7_NONSEC */
#endif /* ! __ASSEMBLY__ */
#endif diff --git a/arch/arm/include/asm/gic.h b/arch/arm/include/asm/gic.h new file mode 100644 index 0000000..c2b1e28 --- /dev/null +++ b/arch/arm/include/asm/gic.h @@ -0,0 +1,17 @@ +#ifndef __GIC_V2_H__ +#define __GIC_V2_H__
+/* register offsets for the ARM generic interrupt controller (GIC) */
+#define GIC_DIST_OFFSET 0x1000 +#define GICD_CTLR 0x0000 +#define GICD_TYPER 0x0004 +#define GICD_IGROUPRn 0x0080 +#define GICD_SGIR 0x0F00
+#define GIC_CPU_OFFSET_A9 0x0100 +#define GIC_CPU_OFFSET_A15 0x2000 +#define GICC_CTLR 0x0000 +#define GICC_PMR 0x0004
+#endif diff --git a/include/configs/vexpress_ca15_tc2.h b/include/configs/vexpress_ca15_tc2.h index d1431e5..89ce1c7 100644 --- a/include/configs/vexpress_ca15_tc2.h +++ b/include/configs/vexpress_ca15_tc2.h @@ -15,6 +15,4 @@ #include "vexpress_common.h" #define CONFIG_BOOTP_VCI_STRING "U-boot.armv7.vexpress_ca15x2_tc2"
-#define CONFIG_SYS_CLK_FREQ 24000000
#endif
1.7.12.1
U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
On Thu, Sep 19, 2013 at 10:00:03PM +0530, Mj Embd wrote:
Hi Andre,
There is another approach taken in xen. (xen/arch/arm/mode_switch.S) Which do you think is the better approach
Hi there,
I'm not sure I completely understand your question. Do you think this patch series should be changed to take something from Xen? If so, can you please clarify why?
For the record, this patch series has been reviewed and tested quite a lot for U-Boot and we would very much like to get this merged and avoid further churn, unless there's a compelling technical reason to change it.
Thanks, -Christoffer
two quick points (a) xen already has a mode_switch code, so AFAIK xen might not use it (as suggested by comment in another patch in this patch set) (b) There are 2 methods of switching from Secure to Hyp mode one you have proposed another implemented in xen. I was suggesting take the best approach
On 9/20/13, Christoffer Dall christoffer.dall@linaro.org wrote:
On Thu, Sep 19, 2013 at 10:00:03PM +0530, Mj Embd wrote:
Hi Andre,
There is another approach taken in xen. (xen/arch/arm/mode_switch.S) Which do you think is the better approach
Hi there,
I'm not sure I completely understand your question. Do you think this patch series should be changed to take something from Xen? If so, can you please clarify why?
For the record, this patch series has been reviewed and tested quite a lot for U-Boot and we would very much like to get this merged and avoid further churn, unless there's a compelling technical reason to change it.
Thanks, -Christoffer
On Fri, Sep 20, 2013 at 01:27:48AM +0530, Mj Embd wrote:
two quick points (a) xen already has a mode_switch code, so AFAIK xen might not use it (as suggested by comment in another patch in this patch set)
For KVM the boot procedure for Hyp mode is quite clearly defined: the kernel must be booted in Hyp mode.
I was under the impression that Xen wanted to use the same paradigm and rely on bootloaders to switch to Hyp mode and thereby get rid of the code in Xen.
(b) There are 2 methods of switching from Secure to Hyp mode one you have proposed another implemented in xen. I was suggesting take the best approach
Can you please be more specific? Not everyone here knows the Xen low-level mode switch details by heart. As far as I know, there is only one architecturally defined proper mode to switch from secure mode to non-secure mode, and the state that needs to be configured for Hyp-mode and NS-mode is well defined. Obviously two implementation can do things differently (different order, different programminge environment, etc.) but that doesn't mean one is better than the other.
I think it would be more productive if you can simply look at this code and if you think some things are done more efficiently in Xen, please comment on that, which would be very helpful. I'm afraid there's no magic way to apply a block of Xen code into U-Boot wihtout manual adjustment anyway, or the other way around for that matter.
-Christoffer
Hello Christoffer,
I agree with both of you points.
What I found different in 2 approaches is that in your approach S->Monitor->NS->Hyp using svc and hvc
While the other approach is setting the M bits directly in cpsr
Xen uses the following
cpsid aif, #0x16 /* Enter Monitor Mode*/ .... ... mrs r0, cpsr /* Copy the CPSR */ add r0, r0, #0x4 /* 0x16 (Monitor) -> 0x1a (Hyp) */ msr spsr_cxsf, r0 /* into the SPSR */ movs pc, r3 /* Exception-return into Hyp mode */
The second one is a bit simpler as it does not involve fault handlers.
I was just suggesting that the best approach to be used ...
Best Regards, mj
On 9/20/13, Christoffer Dall christoffer.dall@linaro.org wrote:
On Fri, Sep 20, 2013 at 01:27:48AM +0530, Mj Embd wrote:
two quick points (a) xen already has a mode_switch code, so AFAIK xen might not use it (as suggested by comment in another patch in this patch set)
For KVM the boot procedure for Hyp mode is quite clearly defined: the kernel must be booted in Hyp mode.
I was under the impression that Xen wanted to use the same paradigm and rely on bootloaders to switch to Hyp mode and thereby get rid of the code in Xen.
(b) There are 2 methods of switching from Secure to Hyp mode one you have proposed another implemented in xen. I was suggesting take the best approach
Can you please be more specific? Not everyone here knows the Xen low-level mode switch details by heart. As far as I know, there is only one architecturally defined proper mode to switch from secure mode to non-secure mode, and the state that needs to be configured for Hyp-mode and NS-mode is well defined. Obviously two implementation can do things differently (different order, different programminge environment, etc.) but that doesn't mean one is better than the other.
I think it would be more productive if you can simply look at this code and if you think some things are done more efficiently in Xen, please comment on that, which would be very helpful. I'm afraid there's no magic way to apply a block of Xen code into U-Boot wihtout manual adjustment anyway, or the other way around for that matter.
-Christoffer
On 09/19/2013 10:38 PM, Mj Embd wrote:
Hello Christoffer,
I agree with both of you points.
What I found different in 2 approaches is that in your approach S->Monitor->NS->Hyp using svc and hvc
While the other approach is setting the M bits directly in cpsr
Xen uses the following
cpsid aif, #0x16 /* Enter Monitor Mode*/ .... ... mrs r0, cpsr /* Copy the CPSR */ add r0, r0, #0x4 /* 0x16 (Monitor) -> 0x1a (Hyp) */ msr spsr_cxsf, r0 /* into the SPSR */ movs pc, r3 /* Exception-return into Hyp mode */
The second one is a bit simpler as it does not involve fault handlers.
The ARM ARM recommends _not_ to do this: Read more at the ARMv7-A Architecture Reference Manual, section B1.5.1: Security states/Changing from Secure to Non-secure state:
"To avoid security holes, software must not: -- Change from Secure to Non-secure state by using an MSR or CPS instruction to switch from Monitor mode to some other mode while SCR.NS is 1. ..."
I was just suggesting that the best approach to be used ...
Looks like this is what we do ;-)
Regards, Andre.
On 9/20/13, Christoffer Dall christoffer.dall@linaro.org wrote:
On Fri, Sep 20, 2013 at 01:27:48AM +0530, Mj Embd wrote:
two quick points (a) xen already has a mode_switch code, so AFAIK xen might not use it (as suggested by comment in another patch in this patch set)
For KVM the boot procedure for Hyp mode is quite clearly defined: the kernel must be booted in Hyp mode.
I was under the impression that Xen wanted to use the same paradigm and rely on bootloaders to switch to Hyp mode and thereby get rid of the code in Xen.
(b) There are 2 methods of switching from Secure to Hyp mode one you have proposed another implemented in xen. I was suggesting take the best approach
Can you please be more specific? Not everyone here knows the Xen low-level mode switch details by heart. As far as I know, there is only one architecturally defined proper mode to switch from secure mode to non-secure mode, and the state that needs to be configured for Hyp-mode and NS-mode is well defined. Obviously two implementation can do things differently (different order, different programminge environment, etc.) but that doesn't mean one is better than the other.
I think it would be more productive if you can simply look at this code and if you think some things are done more efficiently in Xen, please comment on that, which would be very helpful. I'm afraid there's no magic way to apply a block of Xen code into U-Boot wihtout manual adjustment anyway, or the other way around for that matter.
-Christoffer
On Fri, Sep 20, 2013 at 3:01 AM, Andre Przywara andre.przywara@linaro.orgwrote:
On 09/19/2013 10:38 PM, Mj Embd wrote:
Hello Christoffer,
I agree with both of you points.
What I found different in 2 approaches is that in your approach S->Monitor->NS->Hyp using svc and hvc
While the other approach is setting the M bits directly in cpsr
Xen uses the following
cpsid aif, #0x16 /* Enter Monitor Mode*/ .... ... mrs r0, cpsr /* Copy the CPSR */ add r0, r0, #0x4 /* 0x16 (Monitor) -> 0x1a (Hyp) */ msr spsr_cxsf, r0 /* into the SPSR */ movs pc, r3 /* Exception-return into Hyp mode */
The second one is a bit simpler as it does not involve fault handlers.
The ARM ARM recommends _not_ to do this: Read more at the ARMv7-A Architecture Reference Manual, section B1.5.1: Security states/Changing from Secure to Non-secure state:
"To avoid security holes, software must not: -- Change from Secure to Non-secure state by using an MSR or CPS instruction to switch from Monitor mode to some other mode while SCR.NS is
..."
Good one, As per this statement from Secure Monitor to Hyp Mode, HVC
should be used. But Secure Supervisor to Secure Monitor does not set NS=1 so first part cpsid aif, #0x16 is valid ? Or there is some other statement in the manual which I am not aware of ?
I was just suggesting that the best approach to be used ...
Looks like this is what we do ;-)
Regards, Andre.
On 9/20/13, Christoffer Dall christoffer.dall@linaro.org wrote:
On Fri, Sep 20, 2013 at 01:27:48AM +0530, Mj Embd wrote:
two quick points (a) xen already has a mode_switch code, so AFAIK xen might not use it (as suggested by comment in another patch in this patch set)
For KVM the boot procedure for Hyp mode is quite clearly defined: the kernel must be booted in Hyp mode.
I was under the impression that Xen wanted to use the same paradigm and rely on bootloaders to switch to Hyp mode and thereby get rid of the code in Xen.
(b) There are 2 methods of switching from Secure to Hyp mode
one you have proposed another implemented in xen. I was suggesting take the best approach
Can you please be more specific? Not everyone here knows the Xen low-level mode switch details by heart. As far as I know, there is only one architecturally defined proper mode to switch from secure mode to non-secure mode, and the state that needs to be configured for Hyp-mode and NS-mode is well defined. Obviously two implementation can do things differently (different order, different programminge environment, etc.) but that doesn't mean one is better than the other.
I think it would be more productive if you can simply look at this code and if you think some things are done more efficiently in Xen, please comment on that, which would be very helpful. I'm afraid there's no magic way to apply a block of Xen code into U-Boot wihtout manual adjustment anyway, or the other way around for that matter.
-Christoffer
Hello Andre, I need a bit clarification here, if you read the next line after the line you have quoted. It clearly says that you can use a MCR to change from Secure to NS in Monitor Mode "Use an MCR instruction that writes SCR.NS to change from Secure to Non-secure state. This means ARM recommends that software does not alter SCR.NS in any mode except Monitor mode. ARM deprecates changing SCR.NS in any other mode."
On Fri, Sep 20, 2013 at 3:09 AM, Mj Embd mj.embd@gmail.com wrote:
On Fri, Sep 20, 2013 at 3:01 AM, Andre Przywara <andre.przywara@linaro.org
wrote:
On 09/19/2013 10:38 PM, Mj Embd wrote:
Hello Christoffer,
I agree with both of you points.
What I found different in 2 approaches is that in your approach S->Monitor->NS->Hyp using svc and hvc
While the other approach is setting the M bits directly in cpsr
Xen uses the following
cpsid aif, #0x16 /* Enter Monitor Mode*/ .... ... mrs r0, cpsr /* Copy the CPSR */ add r0, r0, #0x4 /* 0x16 (Monitor) -> 0x1a (Hyp) */ msr spsr_cxsf, r0 /* into the SPSR */ movs pc, r3 /* Exception-return into Hyp mode */
The second one is a bit simpler as it does not involve fault handlers.
The ARM ARM recommends _not_ to do this: Read more at the ARMv7-A Architecture Reference Manual, section B1.5.1: Security states/Changing from Secure to Non-secure state:
"To avoid security holes, software must not: -- Change from Secure to Non-secure state by using an MSR or CPS instruction to switch from Monitor mode to some other mode while SCR.NS is
..."
Good one, As per this statement from Secure Monitor to Hyp Mode, HVC
should be used. But Secure Supervisor to Secure Monitor does not set NS=1 so first part cpsid aif, #0x16 is valid ? Or there is some other statement in the manual which I am not aware of ?
I was just suggesting that the best approach to be used ...
Looks like this is what we do ;-)
Regards, Andre.
On 9/20/13, Christoffer Dall christoffer.dall@linaro.org wrote:
On Fri, Sep 20, 2013 at 01:27:48AM +0530, Mj Embd wrote:
two quick points (a) xen already has a mode_switch code, so AFAIK xen might not use it (as suggested by comment in another patch in this patch set)
For KVM the boot procedure for Hyp mode is quite clearly defined: the kernel must be booted in Hyp mode.
I was under the impression that Xen wanted to use the same paradigm and rely on bootloaders to switch to Hyp mode and thereby get rid of the code in Xen.
(b) There are 2 methods of switching from Secure to Hyp mode
one you have proposed another implemented in xen. I was suggesting take the best approach
Can you please be more specific? Not everyone here knows the Xen low-level mode switch details by heart. As far as I know, there is only one architecturally defined proper mode to switch from secure mode to non-secure mode, and the state that needs to be configured for Hyp-mode and NS-mode is well defined. Obviously two implementation can do things differently (different order, different programminge environment, etc.) but that doesn't mean one is better than the other.
I think it would be more productive if you can simply look at this code and if you think some things are done more efficiently in Xen, please comment on that, which would be very helpful. I'm afraid there's no magic way to apply a block of Xen code into U-Boot wihtout manual adjustment anyway, or the other way around for that matter.
-Christoffer
-- -mj
On 20 September 2013 06:55, Mj Embd mj.embd@gmail.com wrote:
Hello Andre, I need a bit clarification here, if you read the next line after the line you have quoted. It clearly says that you can use a MCR to change from Secure to NS in Monitor Mode
No, it's not saying that, because Monitor mode is always Secure: this is exactly why an MCR to change SCR.NS is OK only in Monitor mode -- it doesn't change from Secure to Non-Secure. Only when you do an exception-return to leave Monitor mode will you drop into the NonSecure world.
"Use an MCR instruction that writes SCR.NS to change from Secure to Non-secure state. This means ARM recommends that software does not alter SCR.NS in any mode except Monitor mode. ARM deprecates changing SCR.NS in any other mode."
The text says "don't change from Secure to NonSecure by flipping SCR.NS". It then lays out the corollary: the only time you then can change SCR.NS is when it won't switch from Secure to NonSecure, which is when you're in Monitor mode.
-- PMM
On Fri, Sep 20, 2013 at 4:05 AM, Peter Maydell peter.maydell@linaro.orgwrote:
On 20 September 2013 06:55, Mj Embd mj.embd@gmail.com wrote:
Hello Andre, I need a bit clarification here, if you read the next line after the line you have quoted. It clearly says that you can use a MCR to change from Secure to NS in Monitor Mode
No, it's not saying that, because Monitor mode is always Secure: this is exactly why an MCR to change SCR.NS is OK only in Monitor mode -- it doesn't change from Secure to Non-Secure. Only when you do an exception-return to leave Monitor mode will you drop into the NonSecure world.
"Use an MCR instruction that writes SCR.NS to change from Secure to Non-secure state. This means ARM recommends that software does not alter SCR.NS in any mode except Monitor mode. ARM deprecates changing SCR.NS in any other mode."
The text says "don't change from Secure to NonSecure by flipping SCR.NS". It then lays out the corollary: the only time you then can change SCR.NS is when it won't switch from Secure to NonSecure, which is when you're in Monitor mode.
[MJ] Ok got your point. Then what would be the steps to return from
Monitor to Hyp mode?
-- PMM
On 20 September 2013 07:50, Mj Embd mj.embd@gmail.com wrote:
[MJ] Ok got your point. Then what would be the steps to return from Monitor to Hyp mode?
You can't do it directly. While still in the Secure world you set up the HVBAR and a suitable vector table for hyp mode exceptions. Then you can drop back into NS-SVC, and execute an HVC instruction which puts you into hyp mode at the hvc entry point in your vector table. See patch 7 in the series.
-- PMM
On Thu, 2013-09-19 at 21:11 +0100, Christoffer Dall wrote:
On Fri, Sep 20, 2013 at 01:27:48AM +0530, Mj Embd wrote:
two quick points (a) xen already has a mode_switch code, so AFAIK xen might not use it (as suggested by comment in another patch in this patch set)
For KVM the boot procedure for Hyp mode is quite clearly defined: the kernel must be booted in Hyp mode.
I was under the impression that Xen wanted to use the same paradigm and rely on bootloaders to switch to Hyp mode and thereby get rid of the code in Xen.
Absolutely correct, our needs are the same as KVM and we want to be booted in the same way.
Ian.
On 09/19/2013 09:57 PM, Mj Embd wrote:
two quick points (a) xen already has a mode_switch code, so AFAIK xen might not use it (as suggested by comment in another patch in this patch set)
Just a few days ago Ian sent out patches to remove this code from Xen. That code was never meant to survive, just to have something working to let people play with Xen easily. Xen now requires to be started in HYP mode, just like KVM.
http://lists.xen.org/archives/html/xen-devel/2013-09/msg01690.html
Patch 5/7 has more details.
Regards, Andre.
(b) There are 2 methods of switching from Secure to Hyp mode one you have proposed another implemented in xen. I was suggesting take the best approach
On 9/20/13, Christoffer Dall christoffer.dall@linaro.org wrote:
On Thu, Sep 19, 2013 at 10:00:03PM +0530, Mj Embd wrote:
Hi Andre,
There is another approach taken in xen. (xen/arch/arm/mode_switch.S) Which do you think is the better approach
Hi there,
I'm not sure I completely understand your question. Do you think this patch series should be changed to take something from Xen? If so, can you please clarify why?
For the record, this patch series has been reviewed and tested quite a lot for U-Boot and we would very much like to get this merged and avoid further churn, unless there's a compelling technical reason to change it.
Thanks, -Christoffer
On Fri, 2013-09-20 at 01:27 +0530, Mj Embd wrote:
two quick points (a) xen already has a mode_switch code, so AFAIK xen might not use it (as suggested by comment in another patch in this patch set)
Xen absolutely wants to use this code. The stuff in Xen was there as a hack when this stuff wasn't present in the bootloader. The sooner it can be removed the happier I will be.
Ian.
The core specific part of the work is done in the assembly routine in nonsec_virt.S, introduced with the previous patch, but for the full glory we need to setup the GIC distributor interface once for the whole system, which is done in C here. The routine is placed in arch/arm/cpu/armv7 to allow easy access from other ARMv7 boards.
We check the availability of the security extensions first.
Since we need a safe way to access the GIC, we use the PERIPHBASE registers on Cortex-A15 and A7 CPUs and do some sanity checks. Boards not implementing the CBAR can override this value via a configuration file variable.
Then we actually do the GIC enablement: a) enable the GIC distributor, both for non-secure and secure state (GICD_CTLR[1:0] = 11b) b) allow all interrupts to be handled from non-secure state (GICD_IGROUPRn = 0xFFFFFFFF)
The core specific GIC setup is then done in the assembly routine.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/cpu/armv7/Makefile | 1 + arch/arm/cpu/armv7/virt-v7.c | 122 +++++++++++++++++++++++++++++++++++++++++++ arch/arm/include/asm/armv7.h | 3 ++ 3 files changed, 126 insertions(+) create mode 100644 arch/arm/cpu/armv7/virt-v7.c
Changes: v3..v4: w/s fixes, change GIC address variable, embed error reporting, remove bootm part (next patch) v4..v5: only clear unbanked registers during GIC distributor init
diff --git a/arch/arm/cpu/armv7/Makefile b/arch/arm/cpu/armv7/Makefile index 3466c7a..024c28b 100644 --- a/arch/arm/cpu/armv7/Makefile +++ b/arch/arm/cpu/armv7/Makefile @@ -22,6 +22,7 @@ endif
ifneq ($(CONFIG_ARMV7_NONSEC),) SOBJS += nonsec_virt.o +COBJS += virt-v7.o endif
SRCS := $(START:.o=.S) $(COBJS:.o=.c) diff --git a/arch/arm/cpu/armv7/virt-v7.c b/arch/arm/cpu/armv7/virt-v7.c new file mode 100644 index 0000000..068ac85 --- /dev/null +++ b/arch/arm/cpu/armv7/virt-v7.c @@ -0,0 +1,122 @@ +/* + * (C) Copyright 2013 + * Andre Przywara, Linaro + * + * Routines to transition ARMv7 processors from secure into non-secure state + * needed to enable ARMv7 virtualization for current hypervisors + * + * See file CREDITS for list of people who contributed to this + * project. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, + * MA 02111-1307 USA + */ + +#include <common.h> +#include <asm/armv7.h> +#include <asm/gic.h> +#include <asm/io.h> + +unsigned long gic_dist_addr; + +static unsigned int read_id_pfr1(void) +{ + unsigned int reg; + + asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg)); + return reg; +} + +static unsigned long get_gicd_base_address(void) +{ +#ifdef CONFIG_ARM_GIC_BASE_ADDRESS + return CONFIG_ARM_GIC_BASE_ADDRESS + GIC_DIST_OFFSET; +#else + unsigned midr; + unsigned periphbase; + + /* check whether we are an Cortex-A15 or A7. + * The actual HYP switch should work with all CPUs supporting + * the virtualization extension, but we need the GIC address, + * which we know only for sure for those two CPUs. + */ + asm("mrc p15, 0, %0, c0, c0, 0\n" : "=r"(midr)); + switch (midr & MIDR_PRIMARY_PART_MASK) { + case MIDR_CORTEX_A9_R0P1: + case MIDR_CORTEX_A15_R0P0: + case MIDR_CORTEX_A7_R0P0: + break; + default: + printf("nonsec: could not determine GIC address.\n"); + return -1; + } + + /* get the GIC base address from the CBAR register */ + asm("mrc p15, 4, %0, c15, c0, 0\n" : "=r" (periphbase)); + + /* the PERIPHBASE can be mapped above 4 GB (lower 8 bits used to + * encode this). Bail out here since we cannot access this without + * enabling paging. + */ + if ((periphbase & 0xff) != 0) { + printf("nonsec: PERIPHBASE is above 4 GB, no access.\n"); + return -1; + } + + return (periphbase & CBAR_MASK) + GIC_DIST_OFFSET; +#endif +} + +int armv7_switch_nonsec(void) +{ + unsigned int reg; + unsigned itlinesnr, i; + + /* check whether the CPU supports the security extensions */ + reg = read_id_pfr1(); + if ((reg & 0xF0) == 0) { + printf("nonsec: Security extensions not implemented.\n"); + return -1; + } + + /* the SCR register will be set directly in the monitor mode handler, + * according to the spec one should not tinker with it in secure state + * in SVC mode. Do not try to read it once in non-secure state, + * any access to it will trap. + */ + + gic_dist_addr = get_gicd_base_address(); + if (gic_dist_addr == -1) + return -1; + + /* enable the GIC distributor */ + writel(readl(gic_dist_addr + GICD_CTLR) | 0x03, + gic_dist_addr + GICD_CTLR); + + /* TYPER[4:0] contains an encoded number of available interrupts */ + itlinesnr = readl(gic_dist_addr + GICD_TYPER) & 0x1f; + + /* set all bits in the GIC group registers to one to allow access + * from non-secure state. The first 32 interrupts are private per + * CPU and will be set later when enabling the GIC for each core + */ + for (i = 1; i <= itlinesnr; i++) + writel((unsigned)-1, gic_dist_addr + GICD_IGROUPRn + 4 * i); + + /* call the non-sec switching code on this CPU */ + _nonsec_init(); + + return 0; +} diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h index 3dcfc8f..b352d43 100644 --- a/arch/arm/include/asm/armv7.h +++ b/arch/arm/include/asm/armv7.h @@ -77,6 +77,9 @@ void v7_outer_cache_flush_range(u32 start, u32 end); void v7_outer_cache_inval_range(u32 start, u32 end);
#ifdef CONFIG_ARMV7_NONSEC + +int armv7_switch_nonsec(void); + /* defined in assembly file */ unsigned int _nonsec_init(void); #endif /* CONFIG_ARMV7_NONSEC */
To actually trigger the non-secure switch we just implemented, call the switching routine from within the bootm command implementation. This way we automatically enable this feature without further user intervention.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/lib/bootm.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
Changes: v3..v4: (splitted of from former 4/7 patch) v4..v5: none
diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c index eefb456..b3a961a 100644 --- a/arch/arm/lib/bootm.c +++ b/arch/arm/lib/bootm.c @@ -22,6 +22,10 @@ #include <asm/bootm.h> #include <linux/compiler.h>
+#ifdef CONFIG_ARMV7_NONSEC +#include <asm/armv7.h> +#endif + DECLARE_GLOBAL_DATA_PTR;
static struct tag *params; @@ -181,6 +185,14 @@ static void setup_end_tag(bd_t *bd)
__weak void setup_board_tags(struct tag **in_params) {}
+static void do_nonsec_virt_switch(void) +{ +#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT) + if (armv7_switch_nonsec() == 0) + debug("entered non-secure state\n"); +#endif +} + /* Subcommand: PREP */ static void boot_prep_linux(bootm_headers_t *images) { @@ -217,6 +229,7 @@ static void boot_prep_linux(bootm_headers_t *images) printf("FDT and ATAGS support not compiled in - hanging\n"); hang(); } + do_nonsec_virt_switch(); }
/* Subcommand: GO */
Currently the non-secure switch is only done for the boot processor. To enable full SMP support, we have to switch all secondary cores into non-secure state also.
So we add an entry point for secondary CPUs coming out of low-power state and make sure we put them into WFI again after having switched to non-secure state. For this we acknowledge and EOI the wake-up IPI, then go into WFI. Once being kicked out of it later, we sanity check that the start address has actually been changed (since another attempt to switch to non-secure would block the core) and jump to the new address.
The actual CPU kick is done by sending an inter-processor interrupt via the GIC to all CPU interfaces except the requesting processor. The secondary cores will then setup their respective GIC CPU interface. While this approach is pretty universal across several ARMv7 boards, we make this function weak in case someone needs to tweak this for a specific board.
The way of setting the secondary's start address is board specific, but mostly different only in the actual SMP pen address, so we also provide a weak default implementation and just depend on the proper address to be set in the config file.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/cpu/armv7/nonsec_virt.S | 35 +++++++++++++++++++++++++++++++++++ arch/arm/cpu/armv7/virt-v7.c | 16 +++++++++++++++- arch/arm/include/asm/armv7.h | 1 + arch/arm/include/asm/gic.h | 2 ++ include/common.h | 2 ++ 5 files changed, 55 insertions(+), 1 deletion(-)
Changes: v3..v4: require smp_waitloop to be board specific, allow board specific SMP kicking routine v4..v5: provide weak default implementation of smp_waitloop
diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S index 3dd60b7..cbee8f7 100644 --- a/arch/arm/cpu/armv7/nonsec_virt.S +++ b/arch/arm/cpu/armv7/nonsec_virt.S @@ -58,6 +58,28 @@ _secure_monitor: movs pc, lr @ return to non-secure SVC
/* + * Secondary CPUs start here and call the code for the core specific parts + * of the non-secure and HYP mode transition. The GIC distributor specific + * code has already been executed by a C function before. + * Then they go back to wfi and wait to be woken up by the kernel again. + */ +ENTRY(_smp_pen) + mrs r0, cpsr + orr r0, r0, #0xc0 + msr cpsr, r0 @ disable interrupts + ldr r1, =_start + mcr p15, 0, r1, c12, c0, 0 @ set VBAR + + bl _nonsec_init + + ldr r1, [r0, #GICC_IAR] @ acknowledge IPI + str r1, [r0, #GICC_EOIR] @ signal end of interrupt + + adr r0, _smp_pen @ do not use this address again + b smp_waitloop @ wait for IPIs, board specific +ENDPROC(_smp_pen) + +/* * Switch a core to non-secure state. * * 1. initialize the GIC per-core interface @@ -138,3 +160,16 @@ ENTRY(_nonsec_init)
bx lr ENDPROC(_nonsec_init) + +#ifdef CONFIG_SMP_PEN_ADDR +/* void __weak smp_waitloop(unsigned previous_address); */ +ENTRY(smp_waitloop) + wfi + ldr r1, =CONFIG_SMP_PEN_ADDR @ load start address + ldr r1, [r1] + cmp r0, r1 @ make sure we dont execute this code + beq smp_waitloop @ again (due to a spurious wakeup) + mov pc, r1 +ENDPROC(smp_waitloop) +.weak smp_waitloop +#endif diff --git a/arch/arm/cpu/armv7/virt-v7.c b/arch/arm/cpu/armv7/virt-v7.c index 068ac85..a0b8742 100644 --- a/arch/arm/cpu/armv7/virt-v7.c +++ b/arch/arm/cpu/armv7/virt-v7.c @@ -79,6 +79,17 @@ static unsigned long get_gicd_base_address(void) #endif }
+static void kick_secondary_cpus_gic(unsigned long gicdaddr) +{ + /* kick all CPUs (except this one) by writing to GICD_SGIR */ + writel(1U << 24, gicdaddr + GICD_SGIR); +} + +void __weak smp_kick_all_cpus(void) +{ + kick_secondary_cpus_gic(gic_dist_addr); +} + int armv7_switch_nonsec(void) { unsigned int reg; @@ -115,7 +126,10 @@ int armv7_switch_nonsec(void) for (i = 1; i <= itlinesnr; i++) writel((unsigned)-1, gic_dist_addr + GICD_IGROUPRn + 4 * i);
- /* call the non-sec switching code on this CPU */ + smp_set_core_boot_addr((unsigned long)_smp_pen, -1); + smp_kick_all_cpus(); + + /* call the non-sec switching code on this CPU also */ _nonsec_init();
return 0; diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h index b352d43..2efd4bc 100644 --- a/arch/arm/include/asm/armv7.h +++ b/arch/arm/include/asm/armv7.h @@ -82,6 +82,7 @@ int armv7_switch_nonsec(void);
/* defined in assembly file */ unsigned int _nonsec_init(void); +void _smp_pen(void); #endif /* CONFIG_ARMV7_NONSEC */
#endif /* ! __ASSEMBLY__ */ diff --git a/arch/arm/include/asm/gic.h b/arch/arm/include/asm/gic.h index c2b1e28..a0891cc 100644 --- a/arch/arm/include/asm/gic.h +++ b/arch/arm/include/asm/gic.h @@ -13,5 +13,7 @@ #define GIC_CPU_OFFSET_A15 0x2000 #define GICC_CTLR 0x0000 #define GICC_PMR 0x0004 +#define GICC_IAR 0x000C +#define GICC_EOIR 0x0010
#endif diff --git a/include/common.h b/include/common.h index 8addf43..4d2a56d 100644 --- a/include/common.h +++ b/include/common.h @@ -627,6 +627,8 @@ void ft_pci_setup(void *blob, bd_t *bd); #endif #endif
+void smp_set_core_boot_addr(unsigned long addr, int corenr); +void smp_kick_all_cpus(void);
/* $(CPU)/serial.c */ int serial_init (void);
For the KVM and XEN hypervisors to be usable, we need to enter the kernel in HYP mode. Now that we already are in non-secure state, HYP mode switching is within short reach.
While doing the non-secure switch, we have to enable the HVC instruction and setup the HYP mode HVBAR (while still secure).
The actual switch is done by dropping back from a HYP mode handler without actually leaving HYP mode, so we introduce a new handler routine in our new secure exception vector table.
In the assembly switching routine we save and restore the banked LR and SP registers around the hypercall to do the actual HYP mode switch.
The C routine first checks whether we are in HYP mode already and also whether the virtualization extensions are available. It also checks whether the HYP mode switch was finally successful. The bootm command part only calls the new function after the non-secure switch.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/cpu/armv7/Makefile | 2 +- arch/arm/cpu/armv7/nonsec_virt.S | 43 +++++++++++++++++++++++++++++++++++----- arch/arm/cpu/armv7/virt-v7.c | 37 ++++++++++++++++++++++++++++++++++ arch/arm/include/asm/armv7.h | 6 ++++-- arch/arm/lib/bootm.c | 7 ++++++- 5 files changed, 86 insertions(+), 9 deletions(-)
Changes: v3..v4: w/s fixes, embed error output v4..v5: none
diff --git a/arch/arm/cpu/armv7/Makefile b/arch/arm/cpu/armv7/Makefile index 024c28b..ee4b021 100644 --- a/arch/arm/cpu/armv7/Makefile +++ b/arch/arm/cpu/armv7/Makefile @@ -20,7 +20,7 @@ ifneq ($(CONFIG_AM43XX)$(CONFIG_AM33XX)$(CONFIG_OMAP44XX)$(CONFIG_OMAP54XX)$(CON SOBJS += lowlevel_init.o endif
-ifneq ($(CONFIG_ARMV7_NONSEC),) +ifneq ($(CONFIG_ARMV7_NONSEC)$(CONFIG_ARMV7_VIRT),) SOBJS += nonsec_virt.o COBJS += virt-v7.o endif diff --git a/arch/arm/cpu/armv7/nonsec_virt.S b/arch/arm/cpu/armv7/nonsec_virt.S index cbee8f7..358348f 100644 --- a/arch/arm/cpu/armv7/nonsec_virt.S +++ b/arch/arm/cpu/armv7/nonsec_virt.S @@ -1,5 +1,5 @@ /* - * code for switching cores into non-secure state + * code for switching cores into non-secure state and into HYP mode * * Copyright (c) 2013 Andre Przywara andre.przywara@linaro.org * @@ -28,15 +28,16 @@ #include <asm/armv7.h>
.arch_extension sec +.arch_extension virt
-/* the vector table for secure state */ +/* the vector table for secure state and HYP mode */ _monitor_vectors: .word 0 /* reset */ .word 0 /* undef */ adr pc, _secure_monitor .word 0 .word 0 - .word 0 + adr pc, _hyp_trap .word 0 .word 0
@@ -53,10 +54,27 @@ _secure_monitor: bic r1, r1, #0x4e @ clear IRQ, FIQ, EA, nET bits orr r1, r1, #0x31 @ enable NS, AW, FW bits
+#ifdef CONFIG_ARMV7_VIRT + mrc p15, 0, r0, c0, c1, 1 @ read ID_PFR1 + and r0, r0, #CPUID_ARM_VIRT_MASK @ mask virtualization bits + cmp r0, #(1 << CPUID_ARM_VIRT_SHIFT) + orreq r1, r1, #0x100 @ allow HVC instruction +#endif + mcr p15, 0, r1, c1, c1, 0 @ write SCR (with NS bit set)
+#ifdef CONFIG_ARMV7_VIRT + mrceq p15, 0, r0, c12, c0, 1 @ get MVBAR value + mcreq p15, 4, r0, c12, c0, 0 @ write HVBAR +#endif + movs pc, lr @ return to non-secure SVC
+_hyp_trap: + mrs lr, elr_hyp @ for older asm: .byte 0x00, 0xe3, 0x0e, 0xe1 + mov pc, lr @ do no switch modes, but + @ return to caller + /* * Secondary CPUs start here and call the code for the core specific parts * of the non-secure and HYP mode transition. The GIC distributor specific @@ -71,9 +89,13 @@ ENTRY(_smp_pen) mcr p15, 0, r1, c12, c0, 0 @ set VBAR
bl _nonsec_init + mov r12, r0 @ save GICC address +#ifdef CONFIG_ARMV7_VIRT + bl _switch_to_hyp +#endif
- ldr r1, [r0, #GICC_IAR] @ acknowledge IPI - str r1, [r0, #GICC_EOIR] @ signal end of interrupt + ldr r1, [r12, #GICC_IAR] @ acknowledge IPI + str r1, [r12, #GICC_EOIR] @ signal end of interrupt
adr r0, _smp_pen @ do not use this address again b smp_waitloop @ wait for IPIs, board specific @@ -173,3 +195,14 @@ ENTRY(smp_waitloop) ENDPROC(smp_waitloop) .weak smp_waitloop #endif + +ENTRY(_switch_to_hyp) + mov r0, lr + mov r1, sp @ save SVC copy of LR and SP + isb + hvc #0 @ for older asm: .byte 0x70, 0x00, 0x40, 0xe1 + mov sp, r1 + mov lr, r0 @ restore SVC copy of LR and SP + + bx lr +ENDPROC(_switch_to_hyp) diff --git a/arch/arm/cpu/armv7/virt-v7.c b/arch/arm/cpu/armv7/virt-v7.c index a0b8742..6de7fe7 100644 --- a/arch/arm/cpu/armv7/virt-v7.c +++ b/arch/arm/cpu/armv7/virt-v7.c @@ -3,6 +3,7 @@ * Andre Przywara, Linaro * * Routines to transition ARMv7 processors from secure into non-secure state + * and from non-secure SVC into HYP mode * needed to enable ARMv7 virtualization for current hypervisors * * See file CREDITS for list of people who contributed to this @@ -31,6 +32,14 @@
unsigned long gic_dist_addr;
+static unsigned int read_cpsr(void) +{ + unsigned int reg; + + asm volatile ("mrs %0, cpsr\n" : "=r" (reg)); + return reg; +} + static unsigned int read_id_pfr1(void) { unsigned int reg; @@ -90,6 +99,34 @@ void __weak smp_kick_all_cpus(void) kick_secondary_cpus_gic(gic_dist_addr); }
+int armv7_switch_hyp(void) +{ + unsigned int reg; + + /* check whether we are in HYP mode already */ + if ((read_cpsr() & 0x1f) == 0x1a) { + debug("CPU already in HYP mode\n"); + return 0; + } + + /* check whether the CPU supports the virtualization extensions */ + reg = read_id_pfr1(); + if ((reg & CPUID_ARM_VIRT_MASK) != 1 << CPUID_ARM_VIRT_SHIFT) { + printf("HYP mode: Virtualization extensions not implemented.\n"); + return -1; + } + + /* call the HYP switching code on this CPU also */ + _switch_to_hyp(); + + if ((read_cpsr() & 0x1F) != 0x1a) { + printf("HYP mode: switch not successful.\n"); + return -1; + } + + return 0; +} + int armv7_switch_nonsec(void) { unsigned int reg; diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h index 2efd4bc..395444e 100644 --- a/arch/arm/include/asm/armv7.h +++ b/arch/arm/include/asm/armv7.h @@ -76,14 +76,16 @@ void v7_outer_cache_inval_all(void); void v7_outer_cache_flush_range(u32 start, u32 end); void v7_outer_cache_inval_range(u32 start, u32 end);
-#ifdef CONFIG_ARMV7_NONSEC +#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
int armv7_switch_nonsec(void); +int armv7_switch_hyp(void);
/* defined in assembly file */ unsigned int _nonsec_init(void); void _smp_pen(void); -#endif /* CONFIG_ARMV7_NONSEC */ +void _switch_to_hyp(void); +#endif /* CONFIG_ARMV7_NONSEC || CONFIG_ARMV7_VIRT */
#endif /* ! __ASSEMBLY__ */
diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c index b3a961a..f476a89 100644 --- a/arch/arm/lib/bootm.c +++ b/arch/arm/lib/bootm.c @@ -22,7 +22,7 @@ #include <asm/bootm.h> #include <linux/compiler.h>
-#ifdef CONFIG_ARMV7_NONSEC +#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT) #include <asm/armv7.h> #endif
@@ -189,8 +189,13 @@ static void do_nonsec_virt_switch(void) { #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT) if (armv7_switch_nonsec() == 0) +#ifdef CONFIG_ARMV7_VIRT + if (armv7_switch_hyp() == 0) + debug("entered HYP mode\n"); +#else debug("entered non-secure state\n"); #endif +#endif }
/* Subcommand: PREP */
Hi Andre,
On Thu, 19 Sep 2013 18:06:45 +0200, Andre Przywara andre.przywara@linaro.org wrote:
For the KVM and XEN hypervisors to be usable, we need to enter the kernel in HYP mode. Now that we already are in non-secure state, HYP mode switching is within short reach.
While doing the non-secure switch, we have to enable the HVC instruction and setup the HYP mode HVBAR (while still secure).
The actual switch is done by dropping back from a HYP mode handler without actually leaving HYP mode, so we introduce a new handler routine in our new secure exception vector table.
In the assembly switching routine we save and restore the banked LR and SP registers around the hypercall to do the actual HYP mode switch.
The C routine first checks whether we are in HYP mode already and also whether the virtualization extensions are available. It also checks whether the HYP mode switch was finally successful. The bootm command part only calls the new function after the non-secure switch.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/Makefile | 2 +- arch/arm/cpu/armv7/nonsec_virt.S | 43 +++++++++++++++++++++++++++++++++++----- arch/arm/cpu/armv7/virt-v7.c | 37 ++++++++++++++++++++++++++++++++++ arch/arm/include/asm/armv7.h | 6 ++++-- arch/arm/lib/bootm.c | 7 ++++++- 5 files changed, 86 insertions(+), 9 deletions(-)
Changes: v3..v4: w/s fixes, embed error output v4..v5: none
Seems like Christoffer's comment was not addressed here but IIUC, it was in other files (Christoffer, feel free to comment). Any reason why the "older asm" comments form was not replaced in here?
Amicalement,
On Thu, Oct 03, 2013 at 08:24:57AM +0200, Albert ARIBAUD wrote:
Hi Andre,
On Thu, 19 Sep 2013 18:06:45 +0200, Andre Przywara andre.przywara@linaro.org wrote:
For the KVM and XEN hypervisors to be usable, we need to enter the kernel in HYP mode. Now that we already are in non-secure state, HYP mode switching is within short reach.
While doing the non-secure switch, we have to enable the HVC instruction and setup the HYP mode HVBAR (while still secure).
The actual switch is done by dropping back from a HYP mode handler without actually leaving HYP mode, so we introduce a new handler routine in our new secure exception vector table.
In the assembly switching routine we save and restore the banked LR and SP registers around the hypercall to do the actual HYP mode switch.
The C routine first checks whether we are in HYP mode already and also whether the virtualization extensions are available. It also checks whether the HYP mode switch was finally successful. The bootm command part only calls the new function after the non-secure switch.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/Makefile | 2 +- arch/arm/cpu/armv7/nonsec_virt.S | 43 +++++++++++++++++++++++++++++++++++----- arch/arm/cpu/armv7/virt-v7.c | 37 ++++++++++++++++++++++++++++++++++ arch/arm/include/asm/armv7.h | 6 ++++-- arch/arm/lib/bootm.c | 7 ++++++- 5 files changed, 86 insertions(+), 9 deletions(-)
Changes: v3..v4: w/s fixes, embed error output v4..v5: none
Seems like Christoffer's comment was not addressed here but IIUC, it was in other files (Christoffer, feel free to comment). Any reason why the "older asm" comments form was not replaced in here?
I think these comments are a bit superflous, but not exactly harmful, so I didn't raise the flag when they were not corrected. My thought was that if you're building for a board that has support for the virtualization extensions you should be using a toolchain that knows about them too, but Andre pointed out that his (I think Debian) still used an old enough cross toolchain not to have this support.
In any case, I don't think this warrants holding back the patches but can be fixed as a follow-up if the community agrees that we need to support older toolchains by some define that encodes the hvc and eret instructions properly.
Thanks, -Christoffer
Hi Christoffer,
On Thu, 3 Oct 2013 19:55:15 +0100, Christoffer Dall christoffer.dall@linaro.org wrote:
On Thu, Oct 03, 2013 at 08:24:57AM +0200, Albert ARIBAUD wrote:
Hi Andre,
On Thu, 19 Sep 2013 18:06:45 +0200, Andre Przywara andre.przywara@linaro.org wrote:
For the KVM and XEN hypervisors to be usable, we need to enter the kernel in HYP mode. Now that we already are in non-secure state, HYP mode switching is within short reach.
While doing the non-secure switch, we have to enable the HVC instruction and setup the HYP mode HVBAR (while still secure).
The actual switch is done by dropping back from a HYP mode handler without actually leaving HYP mode, so we introduce a new handler routine in our new secure exception vector table.
In the assembly switching routine we save and restore the banked LR and SP registers around the hypercall to do the actual HYP mode switch.
The C routine first checks whether we are in HYP mode already and also whether the virtualization extensions are available. It also checks whether the HYP mode switch was finally successful. The bootm command part only calls the new function after the non-secure switch.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/Makefile | 2 +- arch/arm/cpu/armv7/nonsec_virt.S | 43 +++++++++++++++++++++++++++++++++++----- arch/arm/cpu/armv7/virt-v7.c | 37 ++++++++++++++++++++++++++++++++++ arch/arm/include/asm/armv7.h | 6 ++++-- arch/arm/lib/bootm.c | 7 ++++++- 5 files changed, 86 insertions(+), 9 deletions(-)
Changes: v3..v4: w/s fixes, embed error output v4..v5: none
Seems like Christoffer's comment was not addressed here but IIUC, it was in other files (Christoffer, feel free to comment). Any reason why the "older asm" comments form was not replaced in here?
I think these comments are a bit superflous, but not exactly harmful, so I didn't raise the flag when they were not corrected. My thought was that if you're building for a board that has support for the virtualization extensions you should be using a toolchain that knows about them too, but Andre pointed out that his (I think Debian) still used an old enough cross toolchain not to have this support.
In any case, I don't think this warrants holding back the patches but can be fixed as a follow-up if the community agrees that we need to support older toolchains by some define that encodes the hvc and eret instructions properly.
Thanks for the answer. Works for me, although I don't believe much in followup fixes once the code is in.
Thanks, -Christoffer
Amicalement,
To enable hypervisors utilizing the ARMv7 virtualization extension on the Versatile Express board with the A15 core tile, we add the required configuration variable. Also we define the board specific smp_set_cpu_boot_addr() function to set the start address for secondary cores in the VExpress specific manner. There is no need to provide a custom smp_waitloop() function here.
This also serves as an example for what to do when adding support for new boards.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- board/armltd/vexpress/vexpress_common.c | 15 +++++++++++++++ include/configs/vexpress_ca15_tc2.h | 5 +++++ 2 files changed, 20 insertions(+)
Changes: v3..v4: add VExpress' smp_set_core_boot_addr() and smp_waitloop() v4..v5: remove VExpress specific smp_waitloop() in favor of the generic implementation
diff --git a/board/armltd/vexpress/vexpress_common.c b/board/armltd/vexpress/vexpress_common.c index 4c7a7f4..56febd9 100644 --- a/board/armltd/vexpress/vexpress_common.c +++ b/board/armltd/vexpress/vexpress_common.c @@ -256,3 +256,18 @@ ulong get_tbclk(void) { return (ulong)CONFIG_SYS_HZ; } + +#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT) +/* Setting the address at which secondary cores start from. + * Versatile Express uses one address for all cores, so ignore corenr + */ +void smp_set_core_boot_addr(unsigned long addr, int corenr) +{ + /* The SYSFLAGS register on VExpress needs to be cleared first + * by writing to the next address, since any writes to the address + * at offset 0 will only be ORed in + */ + writel(~0, CONFIG_SYSFLAGS_ADDR + 4); + writel(addr, CONFIG_SYSFLAGS_ADDR); +} +#endif diff --git a/include/configs/vexpress_ca15_tc2.h b/include/configs/vexpress_ca15_tc2.h index 89ce1c7..0806034 100644 --- a/include/configs/vexpress_ca15_tc2.h +++ b/include/configs/vexpress_ca15_tc2.h @@ -15,4 +15,9 @@ #include "vexpress_common.h" #define CONFIG_BOOTP_VCI_STRING "U-boot.armv7.vexpress_ca15x2_tc2"
+#define CONFIG_SYSFLAGS_ADDR 0x1c010030 +#define CONFIG_SMP_PEN_ADDR CONFIG_SYSFLAGS_ADDR + +#define CONFIG_ARMV7_VIRT + #endif
Hi Andre,
On Thu, 19 Sep 2013 18:06:38 +0200, Andre Przywara andre.przywara@linaro.org wrote:
(for GIT URL and Changelog see below)
ARM CPUs with the virtualization extension have a new mode called HYP mode, which allows hypervisors to safely control and monitor guests. The current hypervisor implementations (KVM and Xen) require the kernel to be entered in that HYP mode.
This patch series introduces a configuration variable CONFIG_ARMV7_VIRT which enables code to switch all cores into HYP mode. This is done automatically during execution of the bootm command.
The process of switching into HYP mode requires the CPU to be in secure state initially when entering u-boot, it will then setup some register and switch to non-secure state. This requires the GIC to be programmed properly first. Explanations about the details are in the commit messages of the respective patches.
The patches are structured like this: 1/8: prepare header file 2/8: add monitor handler (assembly) 3/8: add per CPU non-secure switch routine (assembly) 4/8: add system wide non-secure setup (C) 5/8: trigger non-secure switch during bootm command 6/8: add generic SMP functionality 7/8: add HYP mode switching 8/8: board specific code for ARM Versatile Express TC2
Since up to patch 6/8 this code works on non-virtualization capable CPUs also and as there has been a request, there is now a second configuration variable CONFIG_ARMV7_NONSEC, which omits the final HYP mode switch and just goes into non-secure SVC state. You can specify either (or none) of them, the code cares about the dependency.
The code aims to be as generic as possible, though currently it has only been tested on the Versatile Express TC-2 board. The last patch thus enables the feature for that board and should serve as an example for supporting other boards. I have had reports from different parties of successful adaptions to other boards as well (Arndale, Cubieboard, Chromebook), so I will create a hypmode-ports branch in the below repository to collect those patches and prepare them for upstreaming.
For convenience there is a GIT tree which you can pull these patches from ("hypmode_v5" branch): git://git.linaro.org/people/aprzywara/u-boot.git
Changes RFC..v1
- not a dedicated command anymore, code run by bootm & friends
- protecting code with #ifdefs to avoid unnecessary inclusion and accidental crashing (when accessing restricted registers)
- moving prototypes to header file to meet checkpatch recommendation
- adding comment as proposed by Christoffer
Changes v1..v2 mostly style and code layout changes
- restructure assembly code to live in a new file and not start.S
- split smp, nonsec_init and hyp_init to be separate functions
- used named constants from common header files
- split C function to be more readable
- extend comments to be more precise and elaborate
- add provision to override GIC base address (needed for Arndale?)
- add configuration variable to enable VExpress specific SMP code
- use writel/readl for MMIO GIC accesses
- remove superfluous isb instructions
- more minor fixes
Changes v2..v3
- fix clobbering of GICC address actually spoiling the stack
- do CNTFRQ setup in assembly per core (and not only once per SoC)
- moving the new code files into arch/arm/cpu/armv7
- add config variable for doing non-secure switch only
- use actual HYP and secure instructions mnemonics instead of the encoded byte sequence. This requires more recent compilers.
- make the identification of the CPU core more robust and saner
- use enum for error codes and rename them
- lots of smaller layout and style fixes
Changes v3..v4
- mask reserved bits in CBAR register
- move the VExpress board specific SMP code into the board directory
- embed error reporting in the respective functions and getting rid of the error code enum at all (by popular demand ;-)
- minor style fixes
Changes v4..v5:
- remove unneeded padding in exception table
- only clear unbanked interrupts in GIC distributor for non-secure
- add a default weak implementation for the smp_waitloop() function, there only needs to be provided the address of the SMP pen now
- fix compilation error with non-A15 VExpress boards
- adding patch specific changelogs for v3..v4 and v4..v5
Please review, comment ... and commit ;-)
Contributions and comments to support other boards are welcome.
Andre Przywara (8): ARM: prepare armv7.h to be included from assembly source ARM: add secure monitor handler to switch to non-secure state ARM: add assembly routine to switch to non-secure state ARM: add C function to switch to non-secure state ARM: trigger non-secure state switch during bootm execution ARM: add SMP support for non-secure switch ARM: extend non-secure switch to also go into HYP mode ARM: VExpress: enable ARMv7 virt support for VExpress A15
arch/arm/cpu/armv7/Makefile | 5 + arch/arm/cpu/armv7/nonsec_virt.S | 208 ++++++++++++++++++++++++++++++++ arch/arm/cpu/armv7/virt-v7.c | 173 ++++++++++++++++++++++++++ arch/arm/include/asm/armv7.h | 33 ++++- arch/arm/include/asm/gic.h | 19 +++ arch/arm/lib/bootm.c | 18 +++ board/armltd/vexpress/vexpress_common.c | 15 +++ include/common.h | 2 + include/configs/vexpress_ca15_tc2.h | 5 +- 9 files changed, 476 insertions(+), 2 deletions(-) create mode 100644 arch/arm/cpu/armv7/nonsec_virt.S create mode 100644 arch/arm/cpu/armv7/virt-v7.c create mode 100644 arch/arm/include/asm/gic.h
Series applied to u-boot-arm/master, with the 'empty line at EOF' warning in 2/8 fixed.
Amicalement,
participants (7)
-
Albert ARIBAUD -
Andre Przywara -
Christoffer Dall -
Ian Campbell -
Mj Embd -
Peter Maydell -
TigerLiu＠viatech.com.cn