Subject: Re: [U-Boot] imx7d: CPU core issue in secure mode

• Peng

Hi Tobias, Peng,

On Thu, Jul 4, 2019 at 2:20 PM Tobias Junghans tobias.junghans@veyon.io wrote:

...

Hi,

I'm trying to get an imx7d-based Colibris board running in secure mode in order to be able to use the CAAM, especially the HWRNG. However it seems like it's currently not possible to boot a mainline kernel (4.19) in secure mode with both CPU cores powered up, likely due to the missing PSCI firmware in secure mode. When booting in nonsecure mode the kernel recognizes both CPU cores while CAAM isn't working. Basically it's the same issue as discussed at

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.

...

spinics.net%2Flists%2Fu-boot-v2%2Fmsg33873.html&data=02%7C01%7 Cpen

...

g.fan%40nxp.com%7C69f453b8841a47775d7608d705edd3ee%7C686ea1d3b c2b4c6fa

...

92cd99c5c301635%7C0%7C0%7C636984391331231662&sdata=MtD5x 15k3vvgBMr

...

vqBaZBY9G8AFD0WuE9J8XxIP%2Fz%2Bk%3D&reserved=0

I'm using the latest mainline U-Boot (2019.07-rc4) with CONFIG_ARMV7_BOOT_SEC_DEFAULT=y. Is there anything I can do about

this issue?

Try "setenv bootm_boot_mode nonsec" in U-Boot stage.

...

Thank you and best regards

Tobias

---

U-Boot mailing list U-Boot@lists.denx.de https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist

s.denx.de%2Flistinfo%2Fu-boot&data=02%7C01%7Cpeng.fan%40nxp.co m%7C

...

69f453b8841a47775d7608d705edd3ee%7C686ea1d3bc2b4c6fa92cd99c5c30 1635%7C

...

0%7C0%7C636984391331231662&sdata=Ra4mzQpiZpANam1gyhhsy2g WMHNH3JRNr

...

ryP%2BPOiqsM%3D&reserved=0

I might be mistaken, but AFAIK there was on-going work done by Peng Fan regarding proper CAAM initialization in the OP-TEE and further usage in the mainline kernel.

Silvano was doing the CAAM part in OP-TEE.

As I understood, the initial initialization of the jobrings is done in OP-TEE (which is booted before U-boot) in secure world, and then linux kernel, running in normal world, should be able to use it. Regarding PSCI, frankly, I have no idea who particularly should provide it's support here: U-boot or OP-TEE (taking into account that in this setup U-boot is booted in non-secure PL2, so OP-TEE is the only one, who is able to provide secure runtime services, so-called secure monitor).

BTW, I also saw some setups, where similar things to do the same in U-boot (when it's booted in secure mode), which also does have it's own implementation of secure monitor(subsequently PSCI) and CAAM driver, which probably does the same type of initialization, as in OP-TEE.

Peng, Could you please provide some comments regarding this? Thanks!

There is psci services in U-Boot too. If want non-secure kernel without OP-TEE, Need set "setenv bootm_boot_mode nonsec " in U-Boot stage. If want run OP-TEE, not set the env.

Regards, Peng.

-- Best regards - Freundliche Grüsse - Meilleures salutations

Igor Opaniuk

mailto: igor.opaniuk@gmail.com skype: igor.opanyuk +380 (93) 836 40 67 https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fua.linke din.com%2Fin%2Fiopaniuk&data=02%7C01%7Cpeng.fan%40nxp.com%7 C69f453b8841a47775d7608d705edd3ee%7C686ea1d3bc2b4c6fa92cd99c5c3 01635%7C0%7C0%7C636984391331231662&sdata=%2B8TlRt9QP6mV wMhc3TtHxaZdM%2FvSx09Jz%2BpFhJOlgvg%3D&reserved=0

[Adding Bryan and Breno]

Hi Bryan,

I think you worked on allowing the CAAM driver in Linux to work on i.MX7D running in non-secure when you created: commit 22191ac35344 ("drivers/crypto/fsl: assign job-rings to non-TrustZone")

It was reverted later by Breno as it broke secure boot.

If I understand correctly the current solution is to let OP-TEE deal with job-rings initialization.

Is there any other alternative to use the mainline kernel CAAM driver in non-secure if someone is not using OP-TEE?

Thanks,

Fabio Estevam

On Fri, Jul 12, 2019 at 5:20 AM Tobias Junghans tobias.junghans@veyon.io wrote:

Hi Peng,

Am Freitag, 12. Juli 2019, 05:38:21 CEST schrieb Peng Fan:

...

Try "setenv bootm_boot_mode nonsec" in U-Boot stage.

Unfortunately this does not help. I tried the following setups:

CONFIG_SECURE_BOOT=y CONFIG_CPU_V7_HAS_NONSEC=y CONFIG_CPU_V7_HAS_VIRT=y CONFIG_ARCH_SUPPORT_PSCI=y CONFIG_ARMV7_NONSEC=y CONFIG_ARMV7_BOOT_SEC_DEFAULT=y CONFIG_ARMV7_VIRT=y CONFIG_ARMV7_PSCI=y CONFIG_ARMV7_PSCI_NR_CPUS=2 CONFIG_FSL_CAAM=y CONFIG_SYS_FSL_HAS_SEC=y CONFIG_SYS_FSL_SEC_COMPAT_4=y # CONFIG_SYS_FSL_SEC_BE is not set CONFIG_SYS_FSL_SEC_COMPAT=4 CONFIG_SYS_FSL_SEC_LE=y

Booting with bootm_boot_mode=nonsec

U-Boot 2019.07 (Jul 12 2019 - 10:02:31 +0200) CPU: Freescale i.MX7D rev1.3 1000 MHz (running at 792 MHz) .. SEC0: RNG instantiated ..

[ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c5387d [ 0.000000] CPU: div instructions available: patching division code [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache [ 0.000000] percpu: Embedded 16 pages/cpu s34380 r8192 d22964 u65536 [ 0.000000] pcpu-alloc: s34380 r8192 d22964 u65536 alloc=16*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 .. [ 0.000000] psci: probing for conduit method from DT. [ 0.000000] psci: PSCIv1.0 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs [ 0.000000] psci: Trusted OS migration not required [ 0.000000] psci: SMC Calling Convention v1.0 .. [ 0.002872] CPU: Testing write buffer coherency: ok [ 0.003224] CPU0: update cpu_capacity 1024 [ 0.003234] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000 [ 0.004687] smp: Bringing up secondary CPUs ... [ 0.005424] CPU1: update cpu_capacity 1024 [ 0.005432] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001 [ 0.005553] smp: Brought up 1 node, 2 CPUs [ 0.005568] CPU: All CPU(s) started in HYP mode. [ 0.005571] CPU: Virtualization extensions available. .. [ 0.185229] caam 30900000.caam: device ID = 0x0a16030000000000 (Era 8) [ 0.185240] caam 30900000.caam: job rings = 3, qi = 0 [ 0.186894] caam_jr 30901000.jr0: failed to flush job ring 0 [ 0.192721] caam_jr: probe of 30901000.jr0 failed with error -5 [ 0.192846] caam_jr 30902000.jr1: failed to flush job ring 1 [ 0.198796] caam_jr: probe of 30902000.jr1 failed with error -5 [ 0.198989] caam_jr 30903000.jr1: failed to flush job ring 2 [ 0.204957] caam_jr: probe of 30903000.jr1 failed with error -5 [ 0.212619] Job Ring Device allocation for transform failed

Same configuration with

setenv bootm_boot_mode=sec

[ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c5387d [ 0.000000] CPU: div instructions available: patching division code [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache [ 0.000000] percpu: Embedded 16 pages/cpu s34380 r8192 d22964 u65536 [ 0.000000] pcpu-alloc: s34380 r8192 d22964 u65536 alloc=16*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 [ 0.002866] CPU: Testing write buffer coherency: ok [ 0.003217] CPU0: update cpu_capacity 1024 [ 0.003226] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000 [ 0.004673] smp: Bringing up secondary CPUs ... [ 0.005174] smp: Brought up 1 node, 1 CPU [ 0.005188] CPU: All CPU(s) started in SVC mode. .. [ 0.185631] caam 30900000.caam: device ID = 0x0a16030000000000 (Era 8) [ 0.185643] caam 30900000.caam: job rings = 3, qi = 0 [ 0.196909] caam algorithms registered in /proc/crypto [ 0.199620] caam_jr 30901000.jr0: registering rng-caam

=> only 1 CPU core up.

Now I tried to disable the CAAM driver and secure boot support in U-Boot

# CONFIG_SECURE_BOOT is not set CONFIG_CPU_V7_HAS_NONSEC=y CONFIG_CPU_V7_HAS_VIRT=y CONFIG_ARCH_SUPPORT_PSCI=y CONFIG_ARMV7_NONSEC=y CONFIG_ARMV7_BOOT_SEC_DEFAULT=y CONFIG_ARMV7_VIRT=y CONFIG_ARMV7_PSCI=y CONFIG_ARMV7_PSCI_NR_CPUS=2 # CONFIG_FSL_CAAM is not set CONFIG_SYS_FSL_SEC_COMPAT_4=y # CONFIG_SYS_FSL_SEC_BE is not set CONFIG_SYS_FSL_SEC_LE=y

and booting with bootm_boot_mode=nonsec

[ 0.185233] caam 30900000.caam: Entropy delay = 3200 [ 0.212342] caam 30900000.caam: failed to acquire DECO 0 [ 0.217689] caam 30900000.caam: failed to instantiate RNG

=> both CPU cores are up in nonsec mode

Am I missing something?

Thank you and best regards

Tobias