[U-Boot] [RFC PATCH 0/6] ARMv7: Add HYP mode switching support
ARM CPUs with the virtualization extension have a new mode called HYP mode, which allows hypervisors to safely control and monitor guests. The current hypervisor (KVM and Xen) implementations require the kernel to be entered in that HYP mode.
This patch series introduces a new U-Boot command called "hypmode" which can be used at any time at the U-Boot command prompt to switch the CPU into HYP mode - ideally this would be done before starting the kernel. Since U-Boot does not use the MMU, it runs fine in HYP mode, so you could as well enter HYP mode earlier.
The process of switching into HYP mode requires the CPU to be in non-secure state, which requires the GIC to be programmed properly first. Explanations about the details are in the commit messages of the respective patches.
The code aims to be as generic as possible, though currently it has only been tested on the Versatile Express TC-2 board. The last patch thus enables the feature for that board and relies on the Versatile Express updates patches sent out lately[1].
I would like to get some feedback on the patches, especially about: 1) Is the code in the right places? I used arch/arm/lib/ and arch/arm/cpu/armv7/ as directories for the ARM code and common/ for the actual command. 2) Is a command to switch to HYP actually the right thing? Should HYP be entered automatically instead? Or provide an option to skip HYP mode if requested by the user? 3) Does it make sense to provide a "nonsec" command also? I cannot name any usecases by now, but the implementation would be rather easy with basically all the functionality already there.
In general I appreciate any comments about coding style, patch layout or the actual implementation.
Thanks for watching!
Andre.
[1] http://lists.denx.de/pipermail/u-boot/2013-April/151366.html
Andre Przywara (6): ARM: add secure monitor handler to switch to non-secure state ARM: add assembly routine to switch to non-secure state ARM: add U-Boot command "hypmode" to switch to non-secure state ARM: add SMP support for non-secure switch ARM: extend non-secure switch to also go into HYP mode ARM: VExpress: enable ARMv7 virt support for VExpress A15
arch/arm/cpu/armv7/start.S | 116 ++++++++++++++++++++++++++++--- arch/arm/include/asm/armv7.h | 2 + arch/arm/lib/Makefile | 2 + arch/arm/lib/virt-v7.c | 132 ++++++++++++++++++++++++++++++++++++ common/Makefile | 1 + common/cmd_virt.c | 65 ++++++++++++++++++ include/configs/vexpress_ca15_tc2.h | 3 + 7 files changed, 311 insertions(+), 10 deletions(-) create mode 100644 arch/arm/lib/virt-v7.c create mode 100644 common/cmd_virt.c
A prerequisite for using virtualization is to be in HYP mode, which requires the CPU to be in non-secure state. According to the ARM ARM this should not be done in SVC mode, so we have to setup a SMC handler for this. We reuse the current vector table for this and make sure that we only access the MVBAR register if the CPU supports the virtualization extensions.
Introduce a monitor handler routine which switches the CPU to non-secure state by setting the NS bit (and associated bits).
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/cpu/armv7/start.S | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-)
diff --git a/arch/arm/cpu/armv7/start.S b/arch/arm/cpu/armv7/start.S index e9e57e6..7bfb19d 100644 --- a/arch/arm/cpu/armv7/start.S +++ b/arch/arm/cpu/armv7/start.S @@ -155,6 +155,10 @@ reset: /* Set vector address in CP15 VBAR register */ ldr r0, =_start mcr p15, 0, r0, c12, c0, 0 @Set VBAR + + mrc p15, 0, r1, c0, c1, 1 @ check for security extension + ands r1, r1, #0x30 + mcrne p15, 0, r0, c12, c0, 1 @ Set secure monitor MVBAR #endif
/* the mask ROM code should have PLL and others stable */ @@ -256,6 +260,9 @@ ENTRY(c_runtime_cpu_setup) /* Set vector address in CP15 VBAR register */ ldr r0, =_start mcr p15, 0, r0, c12, c0, 0 @Set VBAR + mrc p15, 0, r1, c0, c1, 1 @ check for security extension + ands r1, r1, #0x30 + mcrne p15, 0, r0, c12, c0, 1 @ Set secure monitor MVBAR
bx lr
@@ -492,9 +499,16 @@ undefined_instruction:
.align 5 software_interrupt: - get_bad_stack_swi - bad_save_user_regs - bl do_software_interrupt + mrc p15, 0, r1, c1, c1, 0 @ read SCR + bic r1, r1, #0x07f + orr r1, r1, #0x31 @ enable NS, AW, FW + + mrc p15, 0, r0, c12, c0, 0 @ save secure copy of VBAR + mcr p15, 0, r1, c1, c1, 0 @ write SCR, switch to non-sec + isb + mcr p15, 0, r0, c12, c0, 0 @ write non-secure copy of VBAR + + movs pc, lr
.align 5 prefetch_abort:
On Fri, Apr 26, 2013 at 03:14:54PM +0200, Andre Przywara wrote:
A prerequisite for using virtualization is to be in HYP mode, which requires the CPU to be in non-secure state. According to the ARM ARM this should not be done in SVC mode, so we have to setup a SMC handler for this. We reuse the current vector table for this and make sure that we only access the MVBAR register if the CPU supports the virtualization extensions.
Introduce a monitor handler routine which switches the CPU to non-secure state by setting the NS bit (and associated bits).
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/start.S | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-)
diff --git a/arch/arm/cpu/armv7/start.S b/arch/arm/cpu/armv7/start.S index e9e57e6..7bfb19d 100644 --- a/arch/arm/cpu/armv7/start.S +++ b/arch/arm/cpu/armv7/start.S @@ -155,6 +155,10 @@ reset: /* Set vector address in CP15 VBAR register */ ldr r0, =_start mcr p15, 0, r0, c12, c0, 0 @Set VBAR
- mrc p15, 0, r1, c0, c1, 1 @ check for security extension
- ands r1, r1, #0x30
- mcrne p15, 0, r0, c12, c0, 1 @ Set secure monitor MVBAR
#endif
/* the mask ROM code should have PLL and others stable */ @@ -256,6 +260,9 @@ ENTRY(c_runtime_cpu_setup) /* Set vector address in CP15 VBAR register */ ldr r0, =_start mcr p15, 0, r0, c12, c0, 0 @Set VBAR
mrc p15, 0, r1, c0, c1, 1 @ check for security extension
ands r1, r1, #0x30
mcrne p15, 0, r0, c12, c0, 1 @ Set secure monitor MVBAR
bx lr
@@ -492,9 +499,16 @@ undefined_instruction:
.align 5 software_interrupt:
- get_bad_stack_swi
- bad_save_user_regs
- bl do_software_interrupt
I think you should add a comment block here saying that this is handling an smc #0 call and the intent is to change to non-secure world.
mrc p15, 0, r1, c1, c1, 0 @ read SCR
bic r1, r1, #0x07f
orr r1, r1, #0x31 @ enable NS, AW, FW
mrc p15, 0, r0, c12, c0, 0 @ save secure copy of VBAR
mcr p15, 0, r1, c1, c1, 0 @ write SCR, switch to non-sec
isb
mcr p15, 0, r0, c12, c0, 0 @ write non-secure copy of VBAR
movs pc, lr
.align 5
prefetch_abort:
1.7.12.1
While actually switching to non-secure state is one thing, the more important part of this process is to make sure that we still have full access to the interrupt controller (GIC). The GIC is fully aware of secure vs. non-secure state, some registers are banked, others may be configured to be accessible from secure state only. To be as generic as possible, we get the GIC memory mapped address based on the PERIPHBASE register. We check explicitly for ARM Cortex-A7 and A15 cores, assuming an A9 otherwise, as for those cores we know the offsets for the GIC CPU interface from the PERIPHBASE content. Other cores could be added as needed.
With the GIC accessible, we: a) allow private interrupts to be delivered to the core (GICD_IGROUPR0 = 0xFFFFFFFF) b) enable the CPU interface (GICC_CTLR[0] = 1) c) set the priority filter to allow non-secure interrupts (GICC_PMR = 0x80)
After having switched to non-secure state, we also enable the non-secure GIC CPU interface, since this register is banked.
Also we allow access to all coprocessor interfaces from non-secure state by writing the appropriate bits in the NSACR register.
For reasons obvious later we only use caller saved registers r0-r3.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/cpu/armv7/start.S | 45 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)
diff --git a/arch/arm/cpu/armv7/start.S b/arch/arm/cpu/armv7/start.S index 7bfb19d..401b0eb 100644 --- a/arch/arm/cpu/armv7/start.S +++ b/arch/arm/cpu/armv7/start.S @@ -561,3 +561,48 @@ fiq:
#endif /* CONFIG_USE_IRQ */ #endif /* CONFIG_SPL_BUILD */ + +/* Routine to initialize GIC CPU interface and switch to nonsecure state. + */ +.globl _nonsec_gic_switch +_nonsec_gic_switch: + mrc p15, 4, r2, c15, c0, 0 @ r2 = PERIPHBASE + add r3, r2, #0x1000 @ GIC dist i/f offset + mvn r1, #0 + str r1, [r3, #0x80] @ allow private interrupts + + mrc p15, 0, r0, c0, c0, 0 @ MIDR + bfc r0, #16, #8 @ mask out variant, arch + bfc r0, #0, #4 @ and revision + movw r1, #0xc070 + movt r1, #0x4100 + cmp r0, r1 @ check for Cortex-A7 + orr r1, #0xf0 + cmpne r0, r1 @ check for Cortex-A15 + movne r1, #0x100 @ GIC CPU offset for A9 + moveq r1, #0x2000 @ GIC CPU offset for A15/A7 + add r3, r2, r1 @ r3 = GIC CPU i/f addr + + mov r1, #1 + str r1, [r3, #0] @ set GICC_CTLR[enable] + mov r1, #0x80 + str r1, [r3, #4] @ set GICC_PIMR[7] + + movw r1, #0x3fff + movt r1, #0x0006 + mcr p15, 0, r1, c1, c1, 2 @ NSACR = all copros to non-sec + + ldr r1, =_start + mcr p15, 0, r1, c12, c0, 0 @ set VBAR + mcr p15, 0, r1, c12, c0, 1 @ set MVBAR + + isb + smc #0 @ call into MONITOR mode + isb @ clobbers r0 and r1 + + mov r1, #1 + str r1, [r3, #0] @ set GICC_CTLR[enable] + add r2, r2, #0x1000 @ GIC dist i/f offset + str r1, [r2] @ allow private interrupts + + mov pc, lr
In preparation for the actual HYP mode switch, we introduce a new U-Boot command called "hypmode". For now we only do the non-secure switch here.
Some part of the work is done in the assembly routine in start.S, introduced with the previous patch, but for the full glory we need to setup the GIC distributor interface once for the whole system, which is done in C here. The routine is placed in arch/arm/lib to allow easy access from different boards or CPUs.
First we check for the availability of the security extensions.
The generic timer base frequency register is only accessible from secure state, so we have to program it now. Actually this should be done from primary firmware before, but some boards seems to omit this, so if needed we do this here with a board specific value.
Since we need a safe way to access the GIC, we use the PERIPHBASE registers on Cortex-A15 and A7 CPUs and do some sanity checks.
Then we actually do the GIC enablement: a) enable the GIC distributor, both for non-secure and secure state (GICD_CTLR[1:0] = 11b) b) allow all interrupts to be handled from non-secure state (GICD_IGROUPRn = 0xFFFFFFFF) The core specific GIC setup is then done in the assembly routine.
The actual U-Boot command is pretty small: calling the routine and doing some error reporting. A return value of 1 will be added later.
To enable the whole code we introduce the CONFIG_CMD_VIRT variable.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/include/asm/armv7.h | 2 + arch/arm/lib/Makefile | 2 + arch/arm/lib/virt-v7.c | 116 +++++++++++++++++++++++++++++++++++++++++++ common/Makefile | 1 + common/cmd_virt.c | 59 ++++++++++++++++++++++ 5 files changed, 180 insertions(+) create mode 100644 arch/arm/lib/virt-v7.c create mode 100644 common/cmd_virt.c
diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h index a73630b..3567692 100644 --- a/arch/arm/include/asm/armv7.h +++ b/arch/arm/include/asm/armv7.h @@ -74,4 +74,6 @@ void v7_outer_cache_inval_all(void); void v7_outer_cache_flush_range(u32 start, u32 end); void v7_outer_cache_inval_range(u32 start, u32 end);
+int armv7_switch_nonsec(void); + #endif diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile index 6ae161a..c5b6b69 100644 --- a/arch/arm/lib/Makefile +++ b/arch/arm/lib/Makefile @@ -58,6 +58,8 @@ COBJS-y += reset.o COBJS-y += cache.o COBJS-y += cache-cp15.o
+COBJS-y += virt-v7.o + SRCS := $(GLSOBJS:.o=.S) $(GLCOBJS:.o=.c) \ $(SOBJS-y:.o=.S) $(COBJS-y:.o=.c) OBJS := $(addprefix $(obj),$(SOBJS-y) $(COBJS-y)) diff --git a/arch/arm/lib/virt-v7.c b/arch/arm/lib/virt-v7.c new file mode 100644 index 0000000..416ca29 --- /dev/null +++ b/arch/arm/lib/virt-v7.c @@ -0,0 +1,116 @@ +/* + * (C) Copyright 2013 + * Andre Przywara, Linaro + * + * routines to push ARMv7 processors from secure into non-secure state + * needed to enable ARMv7 virtualization for current hypervisors + * + * See file CREDITS for list of people who contributed to this + * project. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, + * MA 02111-1307 USA + */ + +#include <common.h> +#include <asm/armv7.h> + +/* the assembly routine doing the actual work in start.S */ +void _nonsec_gic_switch(void); + +#define GICD_CTLR 0x000 +#define GICD_TYPER 0x004 +#define GICD_IGROUPR0 0x080 +#define GICD_SGIR 0xf00 + +#define CPU_ARM_CORTEX_A15 0x4100c0f0 +#define CPU_ARM_CORTEX_A7 0x4100c070 + +static inline unsigned int read_cpsr(void) +{ + unsigned int reg; + + asm volatile ("mrs %0, cpsr\n" : "=r" (reg)); + return reg; +} + +int armv7_switch_nonsec(void) +{ + unsigned int reg; + volatile unsigned int *gicdptr; + unsigned itlinesnr, i; + + /* check whether the CPU supports the security extensions */ + asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg)); + if ((reg & 0xF0) == 0) + return 2; + + /* the timer frequency for the generic timer needs to be + * programmed still in secure state, should be done by firmware. + * check whether we have the generic timer first + */ +#ifdef CONFIG_SYS_CLK_FREQ + asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg)); + if ((reg & 0xF0000) == 0x10000) + asm("mcr p15, 0, %0, c14, c0, 0\n" + : : "r"(CONFIG_SYS_CLK_FREQ)); +#endif + + /* the SCR register will be set directly in the monitor mode handler, + * according to the spec one should not tinker with it in secure state + * in SVC mode. Do not try to read it once in non-secure state, + * any access to it will trap. + */ + + /* check whether we are an Cortex-A15 or A7. + * The actual non-secure switch should work with all CPUs supporting + * the security extension, but we need the GIC address, + * which we know only for sure for those two CPUs. + */ + asm("mrc p15, 0, %0, c0, c0, 0\n" : "=r"(reg)); + if (((reg & 0xFF00FFF0) != 0x4100C0F0) && + ((reg & 0xFF00FFF0) != 0x4100C070)) + return 3; + + /* get the GIC base address from the A15 PERIPHBASE register */ + asm("mrc p15, 4, %0, c15, c0, 0\n" : "=r" (reg)); + + /* the PERIPHBASE can be mapped above 4 GB (lower 8 bits used to + * encode this). Bail out here since we cannot access this without + * enabling paging. + */ + if ((reg & 0xff) != 0) + return 4; + + /* GIC distributor registers start at offset 0x1000 */ + gicdptr = (unsigned *)(reg + 0x1000); + + /* enable the GIC distributor */ + gicdptr[GICD_CTLR / 4] |= 0x03; + + /* TYPER[4:0] contains an encoded number of all interrupts */ + itlinesnr = gicdptr[GICD_TYPER / 4] & 0x1f; + + /* set all bits in the GIC group registers to one to allow access + * from non-secure state + */ + for (i = 0; i <= itlinesnr; i++) + gicdptr[GICD_IGROUPR0 / 4 + i] = (unsigned)-1; + + /* call the non-sec switching code on this CPU */ + _nonsec_gic_switch(); + + return 0; +} diff --git a/common/Makefile b/common/Makefile index 0e0fff1..d9ba5ab 100644 --- a/common/Makefile +++ b/common/Makefile @@ -129,6 +129,7 @@ COBJS-y += cmd_load.o COBJS-$(CONFIG_LOGBUFFER) += cmd_log.o COBJS-$(CONFIG_ID_EEPROM) += cmd_mac.o COBJS-$(CONFIG_CMD_MD5SUM) += cmd_md5sum.o +COBJS-$(CONFIG_CMD_VIRT) += cmd_virt.o COBJS-$(CONFIG_CMD_MEMORY) += cmd_mem.o COBJS-$(CONFIG_CMD_IO) += cmd_io.o COBJS-$(CONFIG_CMD_MFSL) += cmd_mfsl.o diff --git a/common/cmd_virt.c b/common/cmd_virt.c new file mode 100644 index 0000000..132b6b1 --- /dev/null +++ b/common/cmd_virt.c @@ -0,0 +1,59 @@ +/* + * (C) Copyright 2013 + * Andre Przywara, Linaro + * + * command to switch an ARMv7 CPU with security extensions into + * non-secure state + * + * See file CREDITS for list of people who contributed to this + * project. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, + * MA 02111-1307 USA + */ + +#include <common.h> +#include <command.h> + +#include <asm/armv7.h> + +static int do_nonsec(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) +{ + unsigned int ret; + + ret = armv7_switch_nonsec(); + + switch (ret) { + case 0: + break; + case 2: + printf("Security extensions not implemented.\n"); + break; + case 3: + printf("CPU not supported, must be either Cortex-A15 or A7.\n"); + break; + case 4: + printf("PERIPHBASE is above 4 GB, cannot access this.\n"); + break; + } + + return ret; +} + +U_BOOT_CMD( + hypmode, 1, 0, do_nonsec, + "switch ARM CPUs into non-secure state", + "" +);
Currently the non-secure switch is only done for the boot processor. To later allow full SMP support, we have to switch all secondary cores into non-secure state also.
So we add an entry point for secondary CPUs coming out of low-power state and make sure we put them into WFI again after having switched to non-secure state. For this we acknowledge and EOI the wake-up IPI, then go into WFI. Once being kicked out of it later, we sanity check that the start address has actually been changed (since another attempt to switch to non-secure would block the core) and jump to the new address.
The actual CPU kick is done by sending an inter-processor interrupt via the GIC to all CPU interfaces except the requesting processor. The secondary cores will then setup their respective GIC CPU interface.
The address secondary cores jump to is board specific, we provide the value here for the Versatile Express board.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/cpu/armv7/start.S | 27 ++++++++++++++++++++++++++- arch/arm/lib/virt-v7.c | 10 +++++++++- include/configs/vexpress_ca15_tc2.h | 1 + 3 files changed, 36 insertions(+), 2 deletions(-)
diff --git a/arch/arm/cpu/armv7/start.S b/arch/arm/cpu/armv7/start.S index 401b0eb..2b47881 100644 --- a/arch/arm/cpu/armv7/start.S +++ b/arch/arm/cpu/armv7/start.S @@ -563,8 +563,19 @@ fiq: #endif /* CONFIG_SPL_BUILD */
/* Routine to initialize GIC CPU interface and switch to nonsecure state. + * Will be executed directly by secondary CPUs after coming out of + * WFI, or can be called directly by C code for CPU 0. + * Those two paths mandate to not use any stack and to only use registers + * r0-r3 to comply with both the C ABI and the requirement of SMP startup + * code. */ .globl _nonsec_gic_switch +.globl _smp_pen +_smp_pen: + mrs r0, cpsr + orr r0, r0, #0xc0 + msr cpsr, r0 @ disable interrupts + mov lr, #0 @ clear LR to mark secondary _nonsec_gic_switch: mrc p15, 4, r2, c15, c0, 0 @ r2 = PERIPHBASE add r3, r2, #0x1000 @ GIC dist i/f offset @@ -605,4 +616,18 @@ _nonsec_gic_switch: add r2, r2, #0x1000 @ GIC dist i/f offset str r1, [r2] @ allow private interrupts
- mov pc, lr + cmp lr, #0 + movne pc, lr @ CPU 0 to return + @ all others: go to sleep +_ack_int: + ldr r1, [r3, #0x0c] @ read GICD acknowledge + str r1, [r3, #0x10] @ write GICD EOI + + adr r1, _smp_pen +waitloop: + wfi + ldr r0, =CONFIG_SYSFLAGS_ADDR @ load start address + ldr r0, [r0] + cmp r0, r1 @ make sure we dont execute this code + beq waitloop @ again (due to a spurious wakeup) + mov pc, r0 diff --git a/arch/arm/lib/virt-v7.c b/arch/arm/lib/virt-v7.c index 416ca29..5ca093a 100644 --- a/arch/arm/lib/virt-v7.c +++ b/arch/arm/lib/virt-v7.c @@ -29,6 +29,7 @@
/* the assembly routine doing the actual work in start.S */ void _nonsec_gic_switch(void); +void _smp_pen(void);
#define GICD_CTLR 0x000 #define GICD_TYPER 0x004 @@ -51,6 +52,7 @@ int armv7_switch_nonsec(void) unsigned int reg; volatile unsigned int *gicdptr; unsigned itlinesnr, i; + unsigned int *sysflags;
/* check whether the CPU supports the security extensions */ asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg)); @@ -109,7 +111,13 @@ int armv7_switch_nonsec(void) for (i = 0; i <= itlinesnr; i++) gicdptr[GICD_IGROUPR0 / 4 + i] = (unsigned)-1;
- /* call the non-sec switching code on this CPU */ + /* now kick all CPUs (expect this one) by writing to GICD_SIGR */ + sysflags = (void *)CONFIG_SYSFLAGS_ADDR; + sysflags[1] = (unsigned)-1; + sysflags[0] = (uintptr_t)_smp_pen; + gicdptr[GICD_SGIR / 4] = 1U << 24; + + /* call the non-sec switching code on this CPU also */ _nonsec_gic_switch();
return 0; diff --git a/include/configs/vexpress_ca15_tc2.h b/include/configs/vexpress_ca15_tc2.h index 9e230ad..210a27c 100644 --- a/include/configs/vexpress_ca15_tc2.h +++ b/include/configs/vexpress_ca15_tc2.h @@ -32,5 +32,6 @@ #define CONFIG_BOOTP_VCI_STRING "U-boot.armv7.vexpress_ca15x2_tc2"
#define CONFIG_SYS_CLK_FREQ 24000000 +#define CONFIG_SYSFLAGS_ADDR 0x1c010030
#endif
On Fri, Apr 26, 2013 at 6:14 AM, Andre Przywara andre.przywara@linaro.org wrote:
Currently the non-secure switch is only done for the boot processor. To later allow full SMP support, we have to switch all secondary cores into non-secure state also.
So we add an entry point for secondary CPUs coming out of low-power state and make sure we put them into WFI again after having switched to non-secure state. For this we acknowledge and EOI the wake-up IPI, then go into WFI. Once being kicked out of it later, we sanity check that the start address has actually been changed (since another attempt to switch to non-secure would block the core) and jump to the new address.
The actual CPU kick is done by sending an inter-processor interrupt via the GIC to all CPU interfaces except the requesting processor. The secondary cores will then setup their respective GIC CPU interface.
The address secondary cores jump to is board specific, we provide the value here for the Versatile Express board.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/start.S | 27 ++++++++++++++++++++++++++- arch/arm/lib/virt-v7.c | 10 +++++++++- include/configs/vexpress_ca15_tc2.h | 1 + 3 files changed, 36 insertions(+), 2 deletions(-)
diff --git a/arch/arm/cpu/armv7/start.S b/arch/arm/cpu/armv7/start.S index 401b0eb..2b47881 100644 --- a/arch/arm/cpu/armv7/start.S +++ b/arch/arm/cpu/armv7/start.S @@ -563,8 +563,19 @@ fiq: #endif /* CONFIG_SPL_BUILD */
/* Routine to initialize GIC CPU interface and switch to nonsecure state.
- Will be executed directly by secondary CPUs after coming out of
- WFI, or can be called directly by C code for CPU 0.
- Those two paths mandate to not use any stack and to only use registers
- r0-r3 to comply with both the C ABI and the requirement of SMP startup
*/
- code.
.globl _nonsec_gic_switch +.globl _smp_pen +_smp_pen:
mrs r0, cpsrorr r0, r0, #0xc0msr cpsr, r0 @ disable interruptsmov lr, #0 @ clear LR to mark secondary_nonsec_gic_switch: mrc p15, 4, r2, c15, c0, 0 @ r2 = PERIPHBASE add r3, r2, #0x1000 @ GIC dist i/f offset @@ -605,4 +616,18 @@ _nonsec_gic_switch: add r2, r2, #0x1000 @ GIC dist i/f offset str r1, [r2] @ allow private interrupts
mov pc, lr
cmp lr, #0movne pc, lr @ CPU 0 to return@ all others: go to sleep+_ack_int:
ldr r1, [r3, #0x0c] @ read GICD acknowledgestr r1, [r3, #0x10] @ write GICD EOIadr r1, _smp_pen+waitloop:
wfildr r0, =CONFIG_SYSFLAGS_ADDR @ load start addressldr r0, [r0]cmp r0, r1 @ make sure we dont execute this codebeq waitloop @ again (due to a spurious wakeup)mov pc, r0diff --git a/arch/arm/lib/virt-v7.c b/arch/arm/lib/virt-v7.c index 416ca29..5ca093a 100644 --- a/arch/arm/lib/virt-v7.c +++ b/arch/arm/lib/virt-v7.c @@ -29,6 +29,7 @@
/* the assembly routine doing the actual work in start.S */ void _nonsec_gic_switch(void); +void _smp_pen(void);
#define GICD_CTLR 0x000 #define GICD_TYPER 0x004 @@ -51,6 +52,7 @@ int armv7_switch_nonsec(void) unsigned int reg; volatile unsigned int *gicdptr; unsigned itlinesnr, i;
unsigned int *sysflags; /* check whether the CPU supports the security extensions */ asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg));@@ -109,7 +111,13 @@ int armv7_switch_nonsec(void) for (i = 0; i <= itlinesnr; i++) gicdptr[GICD_IGROUPR0 / 4 + i] = (unsigned)-1;
/* call the non-sec switching code on this CPU */
/* now kick all CPUs (expect this one) by writing to GICD_SIGR */sysflags = (void *)CONFIG_SYSFLAGS_ADDR;sysflags[1] = (unsigned)-1;sysflags[0] = (uintptr_t)_smp_pen;gicdptr[GICD_SGIR / 4] = 1U << 24;/* call the non-sec switching code on this CPU also */ _nonsec_gic_switch(); return 0;diff --git a/include/configs/vexpress_ca15_tc2.h b/include/configs/vexpress_ca15_tc2.h index 9e230ad..210a27c 100644 --- a/include/configs/vexpress_ca15_tc2.h +++ b/include/configs/vexpress_ca15_tc2.h @@ -32,5 +32,6 @@ #define CONFIG_BOOTP_VCI_STRING "U-boot.armv7.vexpress_ca15x2_tc2"
#define CONFIG_SYS_CLK_FREQ 24000000 +#define CONFIG_SYSFLAGS_ADDR 0x1c010030
#endif
hmm, is all this likely to work across all armv7 cores? I imagined that the SMP boot protocol could be potentially vastly different on different implementations and this stuff would have to call into board specific code...?
On 04/27/2013 12:13 AM, Christoffer Dall wrote:
On Fri, Apr 26, 2013 at 6:14 AM, Andre Przywara andre.przywara@linaro.org wrote:
Currently the non-secure switch is only done for the boot processor. To later allow full SMP support, we have to switch all secondary cores into non-secure state also.
So we add an entry point for secondary CPUs coming out of low-power state and make sure we put them into WFI again after having switched to non-secure state. For this we acknowledge and EOI the wake-up IPI, then go into WFI. Once being kicked out of it later, we sanity check that the start address has actually been changed (since another attempt to switch to non-secure would block the core) and jump to the new address.
The actual CPU kick is done by sending an inter-processor interrupt via the GIC to all CPU interfaces except the requesting processor. The secondary cores will then setup their respective GIC CPU interface.
The address secondary cores jump to is board specific, we provide the value here for the Versatile Express board.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/start.S | 27 ++++++++++++++++++++++++++- arch/arm/lib/virt-v7.c | 10 +++++++++- include/configs/vexpress_ca15_tc2.h | 1 + 3 files changed, 36 insertions(+), 2 deletions(-)
diff --git a/arch/arm/cpu/armv7/start.S b/arch/arm/cpu/armv7/start.S index 401b0eb..2b47881 100644 --- a/arch/arm/cpu/armv7/start.S +++ b/arch/arm/cpu/armv7/start.S @@ -563,8 +563,19 @@ fiq: #endif /* CONFIG_SPL_BUILD */
/* Routine to initialize GIC CPU interface and switch to nonsecure state.
- Will be executed directly by secondary CPUs after coming out of
- WFI, or can be called directly by C code for CPU 0.
- Those two paths mandate to not use any stack and to only use registers
- r0-r3 to comply with both the C ABI and the requirement of SMP startup
*/ .globl _nonsec_gic_switch
- code.
+.globl _smp_pen +_smp_pen:
mrs r0, cpsrorr r0, r0, #0xc0msr cpsr, r0 @ disable interruptsmov lr, #0 @ clear LR to mark secondary@@ -605,4 +616,18 @@ _nonsec_gic_switch: add r2, r2, #0x1000 @ GIC dist i/f offset str r1, [r2] @ allow private interrupts
mov pc, lr
cmp lr, #0movne pc, lr @ CPU 0 to return@ all others: go to sleep+_ack_int:
ldr r1, [r3, #0x0c] @ read GICD acknowledgestr r1, [r3, #0x10] @ write GICD EOIadr r1, _smp_pen+waitloop:
wfildr r0, =CONFIG_SYSFLAGS_ADDR @ load start addressldr r0, [r0]cmp r0, r1 @ make sure we dont execute this codebeq waitloop @ again (due to a spurious wakeup)mov pc, r0diff --git a/arch/arm/lib/virt-v7.c b/arch/arm/lib/virt-v7.c index 416ca29..5ca093a 100644 --- a/arch/arm/lib/virt-v7.c +++ b/arch/arm/lib/virt-v7.c @@ -29,6 +29,7 @@
/* the assembly routine doing the actual work in start.S */ void _nonsec_gic_switch(void); +void _smp_pen(void);
#define GICD_CTLR 0x000 #define GICD_TYPER 0x004 @@ -51,6 +52,7 @@ int armv7_switch_nonsec(void) unsigned int reg; volatile unsigned int *gicdptr; unsigned itlinesnr, i;
unsigned int *sysflags; /* check whether the CPU supports the security extensions */ asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg));@@ -109,7 +111,13 @@ int armv7_switch_nonsec(void) for (i = 0; i <= itlinesnr; i++) gicdptr[GICD_IGROUPR0 / 4 + i] = (unsigned)-1;
/* call the non-sec switching code on this CPU */
/* now kick all CPUs (expect this one) by writing to GICD_SIGR */sysflags = (void *)CONFIG_SYSFLAGS_ADDR;sysflags[1] = (unsigned)-1;sysflags[0] = (uintptr_t)_smp_pen;gicdptr[GICD_SGIR / 4] = 1U << 24;/* call the non-sec switching code on this CPU also */ _nonsec_gic_switch(); return 0;diff --git a/include/configs/vexpress_ca15_tc2.h b/include/configs/vexpress_ca15_tc2.h index 9e230ad..210a27c 100644 --- a/include/configs/vexpress_ca15_tc2.h +++ b/include/configs/vexpress_ca15_tc2.h @@ -32,5 +32,6 @@ #define CONFIG_BOOTP_VCI_STRING "U-boot.armv7.vexpress_ca15x2_tc2"
#define CONFIG_SYS_CLK_FREQ 24000000 +#define CONFIG_SYSFLAGS_ADDR 0x1c010030
#endif
hmm, is all this likely to work across all armv7 cores?
Probably not.
I imagined that the SMP boot protocol could be potentially vastly different on different implementations and this stuff would have to call into board specific code...?
Right. For now I am trying to get this code upstream. Support for other boards can then be merged in on demand. Looks like at least the Arndale board can be kicked very similarly, only the SYSFLAGS address is different. I am willing to re-architecture the SMP code as needed.
Regards, Andre.
On Mon, May 06, 2013 at 03:19:40PM +0200, Andre Przywara wrote:
On 04/27/2013 12:13 AM, Christoffer Dall wrote:
On Fri, Apr 26, 2013 at 6:14 AM, Andre Przywara andre.przywara@linaro.org wrote:
Currently the non-secure switch is only done for the boot processor. To later allow full SMP support, we have to switch all secondary cores into non-secure state also.
So we add an entry point for secondary CPUs coming out of low-power state and make sure we put them into WFI again after having switched to non-secure state. For this we acknowledge and EOI the wake-up IPI, then go into WFI. Once being kicked out of it later, we sanity check that the start address has actually been changed (since another attempt to switch to non-secure would block the core) and jump to the new address.
The actual CPU kick is done by sending an inter-processor interrupt via the GIC to all CPU interfaces except the requesting processor. The secondary cores will then setup their respective GIC CPU interface.
The address secondary cores jump to is board specific, we provide the value here for the Versatile Express board.
Signed-off-by: Andre Przywara andre.przywara@linaro.org
arch/arm/cpu/armv7/start.S | 27 ++++++++++++++++++++++++++- arch/arm/lib/virt-v7.c | 10 +++++++++- include/configs/vexpress_ca15_tc2.h | 1 + 3 files changed, 36 insertions(+), 2 deletions(-)
diff --git a/arch/arm/cpu/armv7/start.S b/arch/arm/cpu/armv7/start.S index 401b0eb..2b47881 100644 --- a/arch/arm/cpu/armv7/start.S +++ b/arch/arm/cpu/armv7/start.S @@ -563,8 +563,19 @@ fiq: #endif /* CONFIG_SPL_BUILD */
/* Routine to initialize GIC CPU interface and switch to nonsecure state.
- Will be executed directly by secondary CPUs after coming out of
- WFI, or can be called directly by C code for CPU 0.
- Those two paths mandate to not use any stack and to only use registers
- r0-r3 to comply with both the C ABI and the requirement of SMP startup
*/
- code.
.globl _nonsec_gic_switch +.globl _smp_pen +_smp_pen:
mrs r0, cpsrorr r0, r0, #0xc0msr cpsr, r0 @ disable interruptsmov lr, #0 @ clear LR to mark secondary_nonsec_gic_switch: mrc p15, 4, r2, c15, c0, 0 @ r2 = PERIPHBASE add r3, r2, #0x1000 @ GIC dist i/f offset @@ -605,4 +616,18 @@ _nonsec_gic_switch: add r2, r2, #0x1000 @ GIC dist i/f offset str r1, [r2] @ allow private interrupts
mov pc, lr
cmp lr, #0movne pc, lr @ CPU 0 to return@ all others: go to sleep+_ack_int:
ldr r1, [r3, #0x0c] @ read GICD acknowledgestr r1, [r3, #0x10] @ write GICD EOIadr r1, _smp_pen+waitloop:
wfildr r0, =CONFIG_SYSFLAGS_ADDR @ load start addressldr r0, [r0]cmp r0, r1 @ make sure we dont execute this codebeq waitloop @ again (due to a spurious wakeup)mov pc, r0diff --git a/arch/arm/lib/virt-v7.c b/arch/arm/lib/virt-v7.c index 416ca29..5ca093a 100644 --- a/arch/arm/lib/virt-v7.c +++ b/arch/arm/lib/virt-v7.c @@ -29,6 +29,7 @@
/* the assembly routine doing the actual work in start.S */ void _nonsec_gic_switch(void); +void _smp_pen(void);
#define GICD_CTLR 0x000 #define GICD_TYPER 0x004 @@ -51,6 +52,7 @@ int armv7_switch_nonsec(void) unsigned int reg; volatile unsigned int *gicdptr; unsigned itlinesnr, i;
unsigned int *sysflags; /* check whether the CPU supports the security extensions */ asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg));@@ -109,7 +111,13 @@ int armv7_switch_nonsec(void) for (i = 0; i <= itlinesnr; i++) gicdptr[GICD_IGROUPR0 / 4 + i] = (unsigned)-1;
/* call the non-sec switching code on this CPU */
/* now kick all CPUs (expect this one) by writing to GICD_SIGR */sysflags = (void *)CONFIG_SYSFLAGS_ADDR;sysflags[1] = (unsigned)-1;sysflags[0] = (uintptr_t)_smp_pen;gicdptr[GICD_SGIR / 4] = 1U << 24;/* call the non-sec switching code on this CPU also */ _nonsec_gic_switch(); return 0;diff --git a/include/configs/vexpress_ca15_tc2.h b/include/configs/vexpress_ca15_tc2.h index 9e230ad..210a27c 100644 --- a/include/configs/vexpress_ca15_tc2.h +++ b/include/configs/vexpress_ca15_tc2.h @@ -32,5 +32,6 @@ #define CONFIG_BOOTP_VCI_STRING "U-boot.armv7.vexpress_ca15x2_tc2"
#define CONFIG_SYS_CLK_FREQ 24000000 +#define CONFIG_SYSFLAGS_ADDR 0x1c010030
#endif
hmm, is all this likely to work across all armv7 cores?
Probably not.
I imagined that the SMP boot protocol could be potentially vastly different on different implementations and this stuff would have to call into board specific code...?
Right. For now I am trying to get this code upstream. Support for other boards can then be merged in on demand. Looks like at least the Arndale board can be kicked very similarly, only the SYSFLAGS address is different. I am willing to re-architecture the SMP code as needed.
Sorry for the late response, I've been out sick.
So, I'm not very familiar with conventions and practices for the U-boot project, but it seems quite clear to me that since these changes go in arch/arm/lib/... paths those changes must be generic.
Therefore, I'm pretty sure that you need to factor out anything that's specific to vexpress into a part of the code that's specific to vexpress and thereby establish a board interface that generic code can call into for other boards too, for example Arndale; in fact it would be cool if you could show this working for both platforms at the same time, since the working code is already available.
-Christoffer
For the KVM and XEN hypervisors to be usable, we need to enter the kernel in HYP mode. Now that we already are in non-secure state, HYP mode switching is within short reach.
While doing the non-secure switch, we have to enable the HVC instruction and setup the HYP mode HVBAR (while still secure).
The actual switch is done by dropping back from a HYP mode handler without actually leaving HYP mode, so we introduce a new handler routine in the exception vector table.
In the assembly switching routine - which we rename to hyp_gic_switch on the way - we save and restore the banked LR and SP registers around the hypercall to do the actual HYP mode switch.
The C routine first checks whether we are in HYP mode already and also whether the virtualization extensions are available. It also checks whether the HYP mode switch was finally successful. The U-Boot command only adds and adjusts some error reporting.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- arch/arm/cpu/armv7/start.S | 34 +++++++++++++++++++++++----------- arch/arm/include/asm/armv7.h | 2 +- arch/arm/lib/virt-v7.c | 24 ++++++++++++++++-------- common/cmd_virt.c | 20 +++++++++++++------- 4 files changed, 53 insertions(+), 27 deletions(-)
diff --git a/arch/arm/cpu/armv7/start.S b/arch/arm/cpu/armv7/start.S index 2b47881..00890a3 100644 --- a/arch/arm/cpu/armv7/start.S +++ b/arch/arm/cpu/armv7/start.S @@ -41,7 +41,7 @@ _start: b reset ldr pc, _software_interrupt ldr pc, _prefetch_abort ldr pc, _data_abort - ldr pc, _not_used + ldr pc, _hyp_trap ldr pc, _irq ldr pc, _fiq #ifdef CONFIG_SPL_BUILD @@ -49,7 +49,7 @@ _undefined_instruction: .word _undefined_instruction _software_interrupt: .word _software_interrupt _prefetch_abort: .word _prefetch_abort _data_abort: .word _data_abort -_not_used: .word _not_used +_hyp_trap: .word _hyp_trap _irq: .word _irq _fiq: .word _fiq _pad: .word 0x12345678 /* now 16*4=64 */ @@ -58,7 +58,7 @@ _undefined_instruction: .word undefined_instruction _software_interrupt: .word software_interrupt _prefetch_abort: .word prefetch_abort _data_abort: .word data_abort -_not_used: .word not_used +_hyp_trap: .word hyp_trap _irq: .word irq _fiq: .word fiq _pad: .word 0x12345678 /* now 16*4=64 */ @@ -502,12 +502,18 @@ software_interrupt: mrc p15, 0, r1, c1, c1, 0 @ read SCR bic r1, r1, #0x07f orr r1, r1, #0x31 @ enable NS, AW, FW + mrc p15, 0, r0, c0, c1, 1 @ check for Virt ext + and r0, r0, #0xf000 + cmp r0, #0x1000 + orreq r1, r1, #0x100 @ allow HVC instruction
mrc p15, 0, r0, c12, c0, 0 @ save secure copy of VBAR mcr p15, 0, r1, c1, c1, 0 @ write SCR, switch to non-sec isb mcr p15, 0, r0, c12, c0, 0 @ write non-secure copy of VBAR
+ mcreq p15, 4, r0, c12, c0, 0 @ write HYP mode HVBAR + movs pc, lr
.align 5 @@ -523,10 +529,9 @@ data_abort: bl do_data_abort
.align 5 -not_used: - get_bad_stack - bad_save_user_regs - bl do_not_used +hyp_trap: + .byte 0x00, 0xe3, 0x0e, 0xe1 @ mrs lr, elr_hyp + mov pc, lr
#ifdef CONFIG_USE_IRQ
@@ -562,21 +567,21 @@ fiq: #endif /* CONFIG_USE_IRQ */ #endif /* CONFIG_SPL_BUILD */
-/* Routine to initialize GIC CPU interface and switch to nonsecure state. - * Will be executed directly by secondary CPUs after coming out of +/* Routine to initialize GIC CPU interface, switch to nonsecure and to HYP + * mode. Will be executed directly by secondary CPUs after coming out of * WFI, or can be called directly by C code for CPU 0. * Those two paths mandate to not use any stack and to only use registers * r0-r3 to comply with both the C ABI and the requirement of SMP startup * code. */ -.globl _nonsec_gic_switch +.globl _hyp_gic_switch .globl _smp_pen _smp_pen: mrs r0, cpsr orr r0, r0, #0xc0 msr cpsr, r0 @ disable interrupts mov lr, #0 @ clear LR to mark secondary -_nonsec_gic_switch: +_hyp_gic_switch: mrc p15, 4, r2, c15, c0, 0 @ r2 = PERIPHBASE add r3, r2, #0x1000 @ GIC dist i/f offset mvn r1, #0 @@ -616,6 +621,13 @@ _nonsec_gic_switch: add r2, r2, #0x1000 @ GIC dist i/f offset str r1, [r2] @ allow private interrupts
+ mov r2, lr + mov r1, sp + .byte 0x70, 0x00, 0x40, 0xe1 @ hvc #0 + isb + mov sp, r1 + mov lr, r2 + cmp lr, #0 movne pc, lr @ CPU 0 to return @ all others: go to sleep diff --git a/arch/arm/include/asm/armv7.h b/arch/arm/include/asm/armv7.h index 3567692..6c6955b 100644 --- a/arch/arm/include/asm/armv7.h +++ b/arch/arm/include/asm/armv7.h @@ -74,6 +74,6 @@ void v7_outer_cache_inval_all(void); void v7_outer_cache_flush_range(u32 start, u32 end); void v7_outer_cache_inval_range(u32 start, u32 end);
-int armv7_switch_nonsec(void); +int armv7_switch_hyp(void);
#endif diff --git a/arch/arm/lib/virt-v7.c b/arch/arm/lib/virt-v7.c index 5ca093a..72481c6 100644 --- a/arch/arm/lib/virt-v7.c +++ b/arch/arm/lib/virt-v7.c @@ -3,6 +3,7 @@ * Andre Przywara, Linaro * * routines to push ARMv7 processors from secure into non-secure state + * and from non-secure SVC into HYP mode * needed to enable ARMv7 virtualization for current hypervisors * * See file CREDITS for list of people who contributed to this @@ -28,7 +29,7 @@ #include <asm/armv7.h>
/* the assembly routine doing the actual work in start.S */ -void _nonsec_gic_switch(void); +void _hyp_gic_switch(void); void _smp_pen(void);
#define GICD_CTLR 0x000 @@ -47,16 +48,20 @@ static inline unsigned int read_cpsr(void) return reg; }
-int armv7_switch_nonsec(void) +int armv7_switch_hyp(void) { unsigned int reg; volatile unsigned int *gicdptr; unsigned itlinesnr, i; unsigned int *sysflags;
- /* check whether the CPU supports the security extensions */ + /* check whether we are in HYP mode already */ + if ((read_cpsr() & 0x1F) == 0x1a) + return 1; + + /* check whether the CPU supports the virtualization extensions */ asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg)); - if ((reg & 0xF0) == 0) + if ((reg & 0xF000) != 0x1000) return 2;
/* the timer frequency for the generic timer needs to be @@ -77,8 +82,8 @@ int armv7_switch_nonsec(void) */
/* check whether we are an Cortex-A15 or A7. - * The actual non-secure switch should work with all CPUs supporting - * the security extension, but we need the GIC address, + * The actual HYP switch should work with all CPUs supporting + * the virtualization extension, but we need the GIC address, * which we know only for sure for those two CPUs. */ asm("mrc p15, 0, %0, c0, c0, 0\n" : "=r"(reg)); @@ -117,8 +122,11 @@ int armv7_switch_nonsec(void) sysflags[0] = (uintptr_t)_smp_pen; gicdptr[GICD_SGIR / 4] = 1U << 24;
- /* call the non-sec switching code on this CPU also */ - _nonsec_gic_switch(); + /* call the HYP switching code on this CPU also */ + _hyp_gic_switch(); + + if ((read_cpsr() & 0x1F) != 0x1a) + return 5;
return 0; } diff --git a/common/cmd_virt.c b/common/cmd_virt.c index 132b6b1..78b1aa1 100644 --- a/common/cmd_virt.c +++ b/common/cmd_virt.c @@ -2,8 +2,8 @@ * (C) Copyright 2013 * Andre Przywara, Linaro * - * command to switch an ARMv7 CPU with security extensions into - * non-secure state + * command to switch an ARMv7 CPU with virtualization extensions into + * HYP mode to allow hypervisors to install themselves * * See file CREDITS for list of people who contributed to this * project. @@ -29,17 +29,20 @@
#include <asm/armv7.h>
-static int do_nonsec(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) +static int do_hyp(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) { unsigned int ret;
- ret = armv7_switch_nonsec(); + ret = armv7_switch_hyp();
switch (ret) { case 0: break; + case 1: + printf("Already in HYP mode\n"); + break; case 2: - printf("Security extensions not implemented.\n"); + printf("Virtualization extensions not implemented.\n"); break; case 3: printf("CPU not supported, must be either Cortex-A15 or A7.\n"); @@ -47,13 +50,16 @@ static int do_nonsec(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) case 4: printf("PERIPHBASE is above 4 GB, cannot access this.\n"); break; + case 5: + printf("HYP mode switch not successful.\n"); + break; }
return ret; }
U_BOOT_CMD( - hypmode, 1, 0, do_nonsec, - "switch ARM CPUs into non-secure state", + hypmode, 1, 0, do_hyp, + "switch ARM CPUs into HYP mode", "" );
Enable the "hypmode" command for the Versatile Express TC2 board with the virtualization capable Cortex-A15 CPU to allow booting Xen or a KVM-enabled Linux kernel in HYP mode.
Signed-off-by: Andre Przywara andre.przywara@linaro.org --- include/configs/vexpress_ca15_tc2.h | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/include/configs/vexpress_ca15_tc2.h b/include/configs/vexpress_ca15_tc2.h index 210a27c..9df4400 100644 --- a/include/configs/vexpress_ca15_tc2.h +++ b/include/configs/vexpress_ca15_tc2.h @@ -31,6 +31,8 @@ #include "vexpress_common.h" #define CONFIG_BOOTP_VCI_STRING "U-boot.armv7.vexpress_ca15x2_tc2"
+#define CONFIG_CMD_VIRT + #define CONFIG_SYS_CLK_FREQ 24000000 #define CONFIG_SYSFLAGS_ADDR 0x1c010030
On 26 April 2013 14:14, Andre Przywara andre.przywara@linaro.org wrote:
ARM CPUs with the virtualization extension have a new mode called HYP mode, which allows hypervisors to safely control and monitor guests. The current hypervisor (KVM and Xen) implementations require the kernel to be entered in that HYP mode.
This patch series introduces a new U-Boot command called "hypmode" which can be used at any time at the U-Boot command prompt to switch the CPU into HYP mode - ideally this would be done before starting the kernel.
The obvious question here is "why do we need a new command?". The kernel booting specification says "boot the kernel in Hyp mode" so we should just always do that for booting Linux, surely?
thanks -- PMM
On 04/26/2013 03:18 PM, Peter Maydell wrote:
On 26 April 2013 14:14, Andre Przywara andre.przywara@linaro.org wrote:
ARM CPUs with the virtualization extension have a new mode called HYP mode, which allows hypervisors to safely control and monitor guests. The current hypervisor (KVM and Xen) implementations require the kernel to be entered in that HYP mode.
This patch series introduces a new U-Boot command called "hypmode" which can be used at any time at the U-Boot command prompt to switch the CPU into HYP mode - ideally this would be done before starting the kernel.
The obvious question here is "why do we need a new command?". The kernel booting specification says "boot the kernel in Hyp mode" so we should just always do that for booting Linux, surely?
Because it avoids regressions. I kind of feel uneasy to do a lot of tinkering with secure state and the GIC unconditionally, especially if enabled on many boards with virt-capable CPUs.
As written in question 2) later in that mail, I can also live with a command to _dis_able the HYP mode switching in case it causes problems.
Regards, Andre.
On 26 April 2013 14:24, Andre Przywara andre.przywara@linaro.org wrote:
On 04/26/2013 03:18 PM, Peter Maydell wrote:
The obvious question here is "why do we need a new command?". The kernel booting specification says "boot the kernel in Hyp mode" so we should just always do that for booting Linux, surely?
Because it avoids regressions. I kind of feel uneasy to do a lot of tinkering with secure state and the GIC unconditionally, especially if enabled on many boards with virt-capable CPUs.
There aren't exactly very many of those out there, so if we default to "boot in HYP mode" then (a) KVM will just work out of the box and (b) people doing u-boot ports to their board will find any issues and be able to submit fixes. If we don't turn it on by default then we'll end up stuck in a world where KVM doesn't work on half the virt capable boards out there.
I have no objection to a "turn this off" escape hatch if you think it's useful.
-- PMM
On 04/26/2013 03:42 PM, Peter Maydell wrote:
On 26 April 2013 14:24, Andre Przywara andre.przywara@linaro.org wrote:
On 04/26/2013 03:18 PM, Peter Maydell wrote:
The obvious question here is "why do we need a new command?". The kernel booting specification says "boot the kernel in Hyp mode" so we should just always do that for booting Linux, surely?
Because it avoids regressions. I kind of feel uneasy to do a lot of tinkering with secure state and the GIC unconditionally, especially if enabled on many boards with virt-capable CPUs.
There aren't exactly very many of those out there, so if we default to "boot in HYP mode" then (a) KVM will just work out of the box and (b) people doing u-boot ports to their board will find any issues and be able to submit fixes. If we don't turn it on by default then we'll end up stuck in a world where KVM doesn't work on half the virt capable boards out there.
OK, that's a point. I changed the code already and will include it in the next revision.
Thanks, Andre.
I have no objection to a "turn this off" escape hatch if you think it's useful.
-- PMM
On Fri, Apr 26, 2013 at 7:14 AM, Andre Przywara andre.przywara@linaro.org wrote:
On 04/26/2013 03:42 PM, Peter Maydell wrote:
On 26 April 2013 14:24, Andre Przywara andre.przywara@linaro.org wrote:
On 04/26/2013 03:18 PM, Peter Maydell wrote:
The obvious question here is "why do we need a new command?". The kernel booting specification says "boot the kernel in Hyp mode" so we should just always do that for booting Linux, surely?
Because it avoids regressions. I kind of feel uneasy to do a lot of tinkering with secure state and the GIC unconditionally, especially if enabled on many boards with virt-capable CPUs.
There aren't exactly very many of those out there, so if we default to "boot in HYP mode" then (a) KVM will just work out of the box and (b) people doing u-boot ports to their board will find any issues and be able to submit fixes. If we don't turn it on by default then we'll end up stuck in a world where KVM doesn't work on half the virt capable boards out there.
OK, that's a point. I changed the code already and will include it in the next revision.
I strongly agree with Peter, and in fact I think it should only be a compile time option, not a run-time option, only for those debugging u-boot and a kernel on some new platform.
-Christoffer
participants (3)
-
Andre Przywara -
Christoffer Dall -
Peter Maydell