[PATCH v2 0/6] qemu-arm64: Allow booting via Trusted Firmware
U-Boot on QEMU-arm64 can be used in two configurations: Loaded directly via QEMU's -bios option, or as a non-secure payload (BL33) via ARM Trusted Firmware-A (TF-A). In the latter case we need to define CONFIG_TFABOOT, to accommodate the first flash bank being secure only, and manually set SYS_TEXT_BASE to the address configured in TF-A (currently 0x60000000).
To avoid this poorly documented adventure, we enable a position independent build, and also let the flash regions be always detected through the DTB. This results in a single build to work under both scenarios, and also allows to move the BL33 load address in TF-A to something lower in the future.
For this to work, we have to first make PIE work when booted from ROM. While writing to ROM should not hurt, it might trigger CFI flash sequences, and indeed crashes for me in the middle of the fixup routine. This is covered by patch 2/6, which skips the whole fixup routine if the offset is actually 0 (as it is in our case). To support older toolchains (including the popular Linaro builds), we need to ensure we do the static RELA fixups, even with PIE enabled (patch 1/6). Also we have to decouple the relative initial stack pointer from the PIE option, as we always need to use the fixed version, pointing to RAM (patch 3/6). Patch 4/6 drops the hard-coded flash address, instead U-Boot can already read all required information from QEMU's DTB. Patch 5/6 is a cleanup, while the last patch enables the PIE build.
With this series the very same u-boot.bin file works when directly loaded from the QEMU command line (-bios), but also when embedded into TF-A's fip.bin, removing the need for case-specific build options.
Please have a look!
Cheers, Andre
Changelog v1 .. v2: - Always do STATIC_RELA static fixups (new first patch) - Reword commit messages for 3/6 and 5/6
Andre Przywara (5): arm64: PIE: Skip fixups if distance is zero arm64: PIE: Allow fixed stack pointer qemu-arm: Remove need to specify flash banks qemu: Drop ARCH_SUPPORT_TFABOOT qemu/arm64: Enable POSITION_INDEPENDENT
arch/arm/Kconfig | 4 ++-- arch/arm/cpu/armv8/start.S | 3 ++- configs/qemu_arm64_defconfig | 1 + include/configs/qemu-arm.h | 8 +------- 4 files changed, 6 insertions(+), 10 deletions(-)
When we build an arm64 target and enable POSITION_INDEPENDENT, we were skipping our build-time dynamic relocation fixup routine (STATIC_RELA).
This was probably done because we didn't need it in this case, as the PIE fixup routine in start.S would take care of that at runtime.
However when we now skip this routine (upon detecting that the fixup offset is 0), this might lead to uninitialised pointers.
Remove the exception, so that we always do the build-time relocation.
NOTE: GNU binutils starting with v2.27.1 do this build-time relocation automatically, to be in-line with other architecures. So on newer toolchains our manual fixup is actually not needed. It doesn't hurt to have it, though, so that we keep compatibility with the popular Linaro toolchains, which lack this feature.
Signed-off-by: Andre Przywara andre.przywara@arm.com --- arch/arm/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 80702c23d34..b6fb276b6f8 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -76,7 +76,7 @@ config GIC_V3_ITS
config STATIC_RELA bool - default y if ARM64 && !POSITION_INDEPENDENT + default y if ARM64
config DMA_ADDR_T_64BIT bool
On 9/30/20 10:39 AM, Andre Przywara wrote:
When we build an arm64 target and enable POSITION_INDEPENDENT, we were skipping our build-time dynamic relocation fixup routine (STATIC_RELA).
This was probably done because we didn't need it in this case, as the PIE fixup routine in start.S would take care of that at runtime.
However when we now skip this routine (upon detecting that the fixup offset is 0), this might lead to uninitialised pointers.
Remove the exception, so that we always do the build-time relocation.
NOTE: GNU binutils starting with v2.27.1 do this build-time relocation automatically, to be in-line with other architecures. So on newer toolchains our manual fixup is actually not needed. It doesn't hurt to have it, though, so that we keep compatibility with the popular Linaro toolchains, which lack this feature.
Reviewed-by: Stephen Warren swarren@nvidia.com
Hi,
NOTE: GNU binutils starting with v2.27.1 do this build-time relocation automatically, to be in-line with other architecures. So on newer toolchains our manual fixup is actually not needed. It doesn't hurt to have it, though, so that we keep compatibility with the popular Linaro toolchains, which lack this feature.
Signed-off-by: Andre Przywara andre.przywara@arm.com
With this, U-Boot now runs well from ROM when compiled with "gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu".
Tested-by: Amit Singh Tomar amittomer25@gmail.com
Thanks -Amit
On Wed, Sep 30, 2020 at 05:39:13PM +0100, Andre Przywara wrote:
When we build an arm64 target and enable POSITION_INDEPENDENT, we were skipping our build-time dynamic relocation fixup routine (STATIC_RELA).
This was probably done because we didn't need it in this case, as the PIE fixup routine in start.S would take care of that at runtime.
However when we now skip this routine (upon detecting that the fixup offset is 0), this might lead to uninitialised pointers.
Remove the exception, so that we always do the build-time relocation.
NOTE: GNU binutils starting with v2.27.1 do this build-time relocation automatically, to be in-line with other architecures. So on newer toolchains our manual fixup is actually not needed. It doesn't hurt to have it, though, so that we keep compatibility with the popular Linaro toolchains, which lack this feature.
Signed-off-by: Andre Przywara andre.przywara@arm.com Reviewed-by: Stephen Warren swarren@nvidia.com
Applied to u-boot/master, thanks!
When the actual offset between link and runtime address is zero, there is no need for patching up U-Boot early when running with CONFIG_POSITION_INDEPENDENT.
Skip the whole routine when the distance is 0.
This helps when U-Boot is loaded into ROM, or in otherwise sensitive memory locations.
Signed-off-by: Andre Przywara andre.przywara@arm.com Reviewed-by: Stephen Warren swarren@nvidia.com --- arch/arm/cpu/armv8/start.S | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/arm/cpu/armv8/start.S b/arch/arm/cpu/armv8/start.S index 002698b501c..02b952bb328 100644 --- a/arch/arm/cpu/armv8/start.S +++ b/arch/arm/cpu/armv8/start.S @@ -66,7 +66,8 @@ save_boot_params_ret: pie_fixup: adr x0, _start /* x0 <- Runtime value of _start */ ldr x1, _TEXT_BASE /* x1 <- Linked value of _start */ - sub x9, x0, x1 /* x9 <- Run-vs-link offset */ + subs x9, x0, x1 /* x9 <- Run-vs-link offset */ + beq pie_fixup_done adr x2, __rel_dyn_start /* x2 <- Runtime &__rel_dyn_start */ adr x3, __rel_dyn_end /* x3 <- Runtime &__rel_dyn_end */ pie_fix_loop:
On Wed, Sep 30, 2020 at 05:39:14PM +0100, Andre Przywara wrote:
When the actual offset between link and runtime address is zero, there is no need for patching up U-Boot early when running with CONFIG_POSITION_INDEPENDENT.
Skip the whole routine when the distance is 0.
This helps when U-Boot is loaded into ROM, or in otherwise sensitive memory locations.
Signed-off-by: Andre Przywara andre.przywara@arm.com Reviewed-by: Stephen Warren swarren@nvidia.com
Applied to u-boot/master, thanks!
Currently selecting CONFIG_POSITION_INDEPENDENT also forces us to use an initial stack pointer relative to the beginning of the BSS section. This makes some sense, because this should be writable memory anyway.
However the BSS section is not cleared or used until later in the setup process (after relocation), so memory nearby might not be available early enough to host the initial stack. This is an issue if U-Boot is loaded from (Flash-)ROM, for instance.
Allow CONFIG_INIT_SP_RELATIVE to be turned off by a board's config, to be able to select a fixed stack pointer, for instance in known good DRAM.
This will help QEMU utilising PIE, when it's loaded to (Flash-)ROM.
Signed-off-by: Andre Przywara andre.przywara@arm.com Reviewed-by: Stephen Warren swarren@nvidia.com --- arch/arm/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index b6fb276b6f8..486141478ce 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -12,7 +12,6 @@ config ARM64 if ARM64 config POSITION_INDEPENDENT bool "Generate position-independent pre-relocation code" - select INIT_SP_RELATIVE help U-Boot expects to be linked to a specific hard-coded address, and to be loaded to and run from that address. This option lifts that @@ -23,6 +22,7 @@ config POSITION_INDEPENDENT
config INIT_SP_RELATIVE bool "Specify the early stack pointer relative to the .bss section" + default y if POSITION_INDEPENDENT help U-Boot typically uses a hard-coded value for the stack pointer before relocation. Enable this option to instead calculate the
On Wed, Sep 30, 2020 at 05:39:15PM +0100, Andre Przywara wrote:
Currently selecting CONFIG_POSITION_INDEPENDENT also forces us to use an initial stack pointer relative to the beginning of the BSS section. This makes some sense, because this should be writable memory anyway.
However the BSS section is not cleared or used until later in the setup process (after relocation), so memory nearby might not be available early enough to host the initial stack. This is an issue if U-Boot is loaded from (Flash-)ROM, for instance.
Allow CONFIG_INIT_SP_RELATIVE to be turned off by a board's config, to be able to select a fixed stack pointer, for instance in known good DRAM.
This will help QEMU utilising PIE, when it's loaded to (Flash-)ROM.
Signed-off-by: Andre Przywara andre.przywara@arm.com Reviewed-by: Stephen Warren swarren@nvidia.com
Applied to u-boot/master, thanks!
Currently we hard-code the number and initial addresses of QEMU's flash banks, even though our code is perfectly able to gather the same information from the DTB provided by QEMU. This is especially annoying, since we have two slightly different U-Boot configurations ("bare-metal" vs. loaded via Arm Trusted Firmware), which need to be selected at build time.
Drop the two hard coded alternatives, and use CONFIG_SYS_MAX_FLASH_BANKS_DETECT instead, which relies on the DTB to figure out the actual flash configuration at runtime.
Signed-off-by: Andre Przywara andre.przywara@arm.com --- include/configs/qemu-arm.h | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/include/configs/qemu-arm.h b/include/configs/qemu-arm.h index bc8b7c5c123..273fa1a7d7b 100644 --- a/include/configs/qemu-arm.h +++ b/include/configs/qemu-arm.h @@ -45,13 +45,7 @@ #define CONFIG_SYS_CBSIZE 512
#define CONFIG_SYS_MONITOR_BASE CONFIG_SYS_TEXT_BASE -#ifdef CONFIG_TFABOOT -#define CONFIG_SYS_FLASH_BASE 0x4000000 -#define CONFIG_SYS_MAX_FLASH_BANKS 1 -#else -#define CONFIG_SYS_FLASH_BASE 0x0 -#define CONFIG_SYS_MAX_FLASH_BANKS 2 -#endif +#define CONFIG_SYS_MAX_FLASH_BANKS_DETECT 2 #define CONFIG_SYS_MAX_FLASH_SECT 256 /* Sector: 256K, Bank: 64M */ #define CONFIG_CFI_FLASH_USE_WEAK_ACCESSORS
On Wed, Sep 30, 2020 at 05:39:16PM +0100, Andre Przywara wrote:
Currently we hard-code the number and initial addresses of QEMU's flash
banks, even though our code is perfectly able to gather the same information from the DTB provided by QEMU. This is especially annoying, since we have two slightly different U-Boot configurations ("bare-metal" vs. loaded via Arm Trusted Firmware), which need to be selected at build time.
Drop the two hard coded alternatives, and use CONFIG_SYS_MAX_FLASH_BANKS_DETECT instead, which relies on the DTB to figure out the actual flash configuration at runtime.
Signed-off-by: Andre Przywara andre.przywara@arm.com
Applied to u-boot/master, thanks!
CONFIG_ARCH_SUPPORT_TFABOOT was used on the qemu-arm64 platform to guard a tweak to the flash bank configuration. U-Boot now reads the current flash setup from the devicetree, so there is no need for this option anymore.
Signed-off-by: Andre Przywara andre.przywara@arm.com --- arch/arm/Kconfig | 1 - 1 file changed, 1 deletion(-)
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 486141478ce..5ffa84c1d98 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -929,7 +929,6 @@ config ARCH_OWL
config ARCH_QEMU bool "QEMU Virtual Platform" - select ARCH_SUPPORT_TFABOOT select DM select DM_SERIAL select OF_CONTROL
On Wed, Sep 30, 2020 at 05:39:17PM +0100, Andre Przywara wrote:
CONFIG_ARCH_SUPPORT_TFABOOT was used on the qemu-arm64 platform to guard a tweak to the flash bank configuration. U-Boot now reads the current flash setup from the devicetree, so there is no need for this option anymore.
Signed-off-by: Andre Przywara andre.przywara@arm.com
Reviewed-by: Tom Rini trini@konsulko.com
On Wed, Sep 30, 2020 at 05:39:17PM +0100, Andre Przywara wrote:
CONFIG_ARCH_SUPPORT_TFABOOT was used on the qemu-arm64 platform to guard a tweak to the flash bank configuration. U-Boot now reads the current flash setup from the devicetree, so there is no need for this option anymore.
Signed-off-by: Andre Przywara andre.przywara@arm.com Reviewed-by: Tom Rini trini@konsulko.com
Applied to u-boot/master, thanks!
Now that PIE works when U-Boot is started from ROM, let's enable CONFIG_POSITION_INDEPENDENT, which allows to load U-Boot also via ARM Trusted-Firmware's fip.bin to DRAM, without tweaking the configuration.
To get a writable initial stack, we need to keep the fixed initial stack pointer, which points to DRAM in our case.
Signed-off-by: Andre Przywara andre.przywara@arm.com Reviewed-by: Stephen Warren swarren@nvidia.com --- arch/arm/Kconfig | 1 + configs/qemu_arm64_defconfig | 1 + 2 files changed, 2 insertions(+)
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 5ffa84c1d98..8ba73680699 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -22,6 +22,7 @@ config POSITION_INDEPENDENT
config INIT_SP_RELATIVE bool "Specify the early stack pointer relative to the .bss section" + default n if ARCH_QEMU default y if POSITION_INDEPENDENT help U-Boot typically uses a hard-coded value for the stack pointer diff --git a/configs/qemu_arm64_defconfig b/configs/qemu_arm64_defconfig index 31ea2d342fc..4450e7ced42 100644 --- a/configs/qemu_arm64_defconfig +++ b/configs/qemu_arm64_defconfig @@ -1,6 +1,7 @@ CONFIG_ARM=y CONFIG_ARCH_QEMU=y CONFIG_NR_DRAM_BANKS=1 +CONFIG_POSITION_INDEPENDENT=y CONFIG_ENV_SIZE=0x40000 CONFIG_ENV_SECT_SIZE=0x40000 CONFIG_AHCI=y
On Wed, Sep 30, 2020 at 05:39:18PM +0100, Andre Przywara wrote:
Now that PIE works when U-Boot is started from ROM, let's enable CONFIG_POSITION_INDEPENDENT, which allows to load U-Boot also via ARM Trusted-Firmware's fip.bin to DRAM, without tweaking the configuration.
To get a writable initial stack, we need to keep the fixed initial stack pointer, which points to DRAM in our case.
Signed-off-by: Andre Przywara andre.przywara@arm.com Reviewed-by: Stephen Warren swarren@nvidia.com
Applied to u-boot/master, thanks!
On Wed, Sep 30, 2020 at 05:39:12PM +0100, Andre Przywara wrote:
U-Boot on QEMU-arm64 can be used in two configurations: Loaded directly via QEMU's -bios option, or as a non-secure payload (BL33) via ARM Trusted Firmware-A (TF-A). In the latter case we need to define CONFIG_TFABOOT, to accommodate the first flash bank being secure only, and manually set SYS_TEXT_BASE to the address configured in TF-A (currently 0x60000000).
To avoid this poorly documented adventure, we enable a position independent build, and also let the flash regions be always detected through the DTB. This results in a single build to work under both scenarios, and also allows to move the BL33 load address in TF-A to something lower in the future.
For this to work, we have to first make PIE work when booted from ROM. While writing to ROM should not hurt, it might trigger CFI flash sequences, and indeed crashes for me in the middle of the fixup routine. This is covered by patch 2/6, which skips the whole fixup routine if the offset is actually 0 (as it is in our case). To support older toolchains (including the popular Linaro builds), we need to ensure we do the static RELA fixups, even with PIE enabled (patch 1/6). Also we have to decouple the relative initial stack pointer from the PIE option, as we always need to use the fixed version, pointing to RAM (patch 3/6). Patch 4/6 drops the hard-coded flash address, instead U-Boot can already read all required information from QEMU's DTB. Patch 5/6 is a cleanup, while the last patch enables the PIE build.
With this series the very same u-boot.bin file works when directly loaded from the QEMU command line (-bios), but also when embedded into TF-A's fip.bin, removing the need for case-specific build options.
Please have a look!
Can you please also update doc/board/emulation/qemu-arm.rst with instructions / example of using this configuration? Thanks!
participants (4)
-
Amit Tomar -
Andre Przywara -
Stephen Warren -
Tom Rini