On Mon, Oct 11, 2021 at 3:15 PM Tim Harvey tharvey@gateworks.com wrote:
Greetings,
Is anyone successfully booting U-Boot with OP-TEE support on the IMX8M?
My understanding is that you need to add tee.bin to the images in the FIT image and include it in loadables following the ATF.
While this was done with arch/arm/mach-imx/mkimage_fit_atf.sh before the switch to binman by simply having tee.bin in the U-Boot directory and passing in TEE_LOAD_ADDR (or accepting the default of 0xfe000000) once you switch to binman it needs to be added for your board.
Additionally in order to use OP-TEE from U-Boot (ie for dek_blob command) you need to add a node with compatible=linaro,optee-tz as well.
I've done the following to add OP-TEE for imx8mm_venice: diff --git a/configs/imx8mm_venice_defconfig b/configs/imx8mm_venice_defconfig index d85827b588..85d2b20810 100644 -u-boot.dtsi index e0fa9ff4bf..7a71b974e1 100644 --- a/arch/arm/dts/imx8mm-venice-u-boot.dtsi +++ b/arch/arm/dts/imx8mm-venice-u-boot.dtsi @@ -10,6 +10,13 @@ multiple-images; };
firmware {optee {compatible = "linaro,optee-tz";method = "smc";};};wdt-reboot { compatible = "wdt-reboot"; wdt = <&wdog1>;@@ -152,6 +159,16 @@ }; };
tee {description = "TEE firmware";type = "firmware";arch = "arm64";compression = "none";data = "tee.bin";load = <0xbe000000>;entry = <0xbe000000>;};@fdt-SEQ { description = "NAME"; type = "flat_dt";@@ -165,7 +182,7 @@ @config-SEQ { description = "NAME"; firmware = "uboot";
loadables = "atf";
loadables = "atf", "tee"; fdt = "fdt-SEQ"; }; };However, when I attempt to boot I hang when the ATF is run.
Where does the TEE_LOAD_ADDR come from specifically? I would think this needs to be defined when building tee and needs to match the load address used in the FIT image. It appears that perhaps this is supped to be DDR_BASE + DDR_SIZE - 32MIB but I'm not entirely sure.
I'm currently using NXP's ATF (imx_5.4.3_2.0.0) and NXP's TEE (imx_5.4.70_2.3.0) and would also like to understand if NXP's branches are strictly required here and if so what the pros and cons of using them are.
Anyone using IMX8MM OP-TEE that could point me in the right direction?
Here's what I've learned so far via various responses, research and testing.
There are multiple boot flows that can be used with OP-TEE: 1. SPL -> ATF -> OP-TEE -> U-boot -> LInux 2. SPL -> OP-TEE -> U-boot -> Linux
I'm not really clear what the pros and cons of each are - can anyone shed some light on this?
Method #1 is what I've found the most info on and have mostly gotten working so I will go over that here. As IMX8MM runs on Cortex-A Arm TrustZone is available and thus TF-A can be used.
By the way, I'm a bit confused as to ATF vs TF-A. From what I can find: - ATF (Arm Trusted Firmware) is a Trusted Firmware reference implementation of secure world software for Armv7-A, Armv8-A, and Armv8-M architectures. - TF-A (Trusted Firmware-A) also appears to provide the same thing Are these the same? Which one is it that NXP uses for imx-atf [1]
When OP-TEE is used with TF-A the TF-A gets invoked with the TEE as bl32 and main u-boot as bl33. Once it has done its startup TF-A jumps into the bl32(OP-TEE) for the TEE startup, returns to TF-A and then jumps to bl33(U-Boot).
For this boot flow on IMX8MM you need to do the following: - build TF-A (bl31.bin) with OP-TEE as a Secure Payload Dispatcher (SPD) (add 'SPD=opteed' to env) - build OP-TEE (tee.bin) - add tee.bin to U-Boot FIT image along with TF-A binary in images node * for boards still using arch/arm/mach-imx/mkimage_fit_atf.sh FIT generation this is done automatically when tee.bin is U-Boot's build directory but you also need to set TEE_LOAD_ADDR env to specify the load addr of the TEE which depends on the board DRAM size * for boards using binman the generation of the FIT image has moved to using the 'binman' tool thus adding tee.bin requires you to modify your board's u-boot.dtsi itb node to add tee.bin to itb/fit/images/ and add a reference to it in the loadables property of your boot configuration following atf. This is shown in my patch above except there I neglected to add the required 'tee_blob: blob-ext { filename = "tee.bin"; }. Here you must set the TEE_LOAD_ADDR as the load address manually. - To enable OP-TEE support in U-Boot a /firmware/optee node needs to be added to your board u-boot.dtsi with 'compatible = "linaro,optee-tz"' and 'method = "smc"' nodes * This node is used by the U-Boot dek_blob command * If present then U-Boot will also copy it to your Linux dtb via image_setup_libfdt->optee_copy_fdt_nodes - To enable dek_blob in U-Boot enable CONFIG_CMD_DEKBLOB=y which will enable CONFIG_IMX_OPTEE_DEK_ENCAP=y - To enable optee driver in Linux enable CONFIG_TEE and CONFIG_OPTEE and optionally CONFIG_HW_RANDOM_OPTEE - Also, I believe you need to add memory reservations in the dt via a /reserved-memory/optee_core and /reserved-memory/optee_shm which will also be copied to the linux dtb as well if present and also depend on the OP-TEE memory location/configuration
If you are using NXP's imx-optee-os [2] for IMX8MM then you can use the 'mx8mmevk' platform which simply configures the following which can be overridden via env: CFG_UART_BASE ?= UART2_BASE CFG_DDR_SIZE ?= 0x80000000
In core/arch/arm/plat-imx/conf.mk for imx8mm CFG_DDR_SIZE is used to compute CFG_TZDRAM_START where optee gets loaded (TEE_LOAD_ADDR): # put optee end of ddr for AARCH64 CFG_TZDRAM_START ?= ($(CFG_DRAM_BASE) - 0x02000000 + $(CFG_DDR_SIZE)) CFG_TZDRAM_SIZE ?= 0x01c00000 CFG_SHMEM_START ?= ($(CFG_TZDRAM_START) + $(CFG_TZDRAM_SIZE)) CFG_SHMEM_SIZE ?= 0x00400000
thus: CFG_DDR_SIZE = 0x40000000 (1GiB) results in CFG_TZDRAM_START=0x7e000000 (TEE_LOAD_ADDR) and CFG_SHMEM_START=0x7fc00000 CFG_DDR_SIZE = 0x80000000 (2GiB) results in CFG_TZDRAM_START=0xbe000000 (TEE_LOAD_ADDR) and CFG_SHMEM_START=0xbfc00000 CFG_DDR_SIZE = 0x100000000 (4GiB) results in CFG_TZDRAM_START=0x13e000000 (TEE_LOAD_ADDR) and CFG_SHMEM_START=0x13fc00000
So for a 2GiB board your dt changes for a board already using binman should look like this: @@ -6,6 +6,29 @@ #include "imx8mm-u-boot.dtsi"
/ { + firmware { + optee { + compatible = "linaro,optee-tz"; + method = "smc"; + }; + }; + + reserved-memory { + #address-cells = <2>; + #size-cells = <2>; + ranges; + + /* 32MiB */ + optee_core@be000000 { + reg = <0x00 0xbe000000 0x00 0x2000000>; + }; + + /* 4K */ + optee_shm@bfc00000 { + reg = <0x00 0xbfc00000 0x00 0x400000>; + }; + }; + binman: binman { multiple-images; }; @@ -152,6 +175,19 @@ }; };
+ tee { + description = "TEE firmware"; + type = "firmware"; + arch = "arm64"; + compression = "none"; + load = <0xbe000000>; + entry = <0xbe000000>; + + tee_blob: blob-ext { + filename = "tee.bin"; + }; + }; + @fdt-SEQ { description = "NAME"; type = "flat_dt"; @@ -165,7 +201,7 @@ @config-SEQ { description = "NAME"; firmware = "uboot"; - loadables = "atf"; + loadables = "atf", "tee"; fdt = "fdt-SEQ"; }; };
I have done all of the above using imx-atf [1] and imx-optee-os [2] on a 2GiB IMX8MM board and successfully boot to U-Boot where I verify dek_blob works, and then successfully booting to Linux where I verify that the Linux optee driver probes.
But, what confuses me relates to optee memory usage and location (TEE_LOAD_ADDR) based on testing on 1GiB and 4GiB IMX8MM boards: - CFG_DDR_SIZE = 0x40000000 (1GiB); TEE_LOAD_ADDR=0x7e000000 - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a 1GiB DRAM board (why? makes no sense) - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a 2GiB DRAM board (this makes sense as 0x7e000000 should be out of bounds!) - hangs in opteed_init on a 4GiB DRAM board (why?) - CFG_DDR_SIZE = 0x80000000 (2GiB); TEE_LOAD_ADDR=0xbe000000 - boots on a 1GiB DRAM board... shouldn't that crash when accessing 0xbe000000? - boots on a 2GiB DRAM board - boots on a 4GiB DRAM board CFG_DDR_SIZE = 0x100000000 (4GiB); TEE_LOAD_ADDR=0x13e000000 - fails building: 'Error: u-boot.its:63.13-24 Value out of range for 32-bit array element' - also this may be an issue as currently IMX8MM boards have DRAM limited to 3GiB due to various IP drivers not being 64bit ready
Also, I find that the memory reservations produce an error when I TFTP a file due to (tftp_init_load_addr -> lmb_init_and_reserve) u-boot=> tftpboot $fdt_addr_r venice/imx8mm-venice-gw73xx-0x.dtb Using ethernet@30be0000 device TFTP from server 192.168.1.146; our IP address is 192.168.1.1 Filename 'venice/imx8mm-venice-gw73xx-0x.dtb'. ERROR: reserving fdt memory region failed (addr=bfc00000 size=400000 flags=0)
Can anyone explain the memory configurations needed for IMX8MM OP-TEE?
Best regards,
Tim [1] https://source.codeaurora.org/external/imx/imx-atf/ [2] https://source.codeaurora.org/external/imx/imx-optee-os/