On 1/15/25 12:51 PM, Quentin Schulz wrote:
Hi Marek, Biju,
Hi,
[...]
See 3/3 , U-Boot runs in EL3 , before starting kernel it jumps into TFA BL31 , TFA BL31 does its setup and switches from EL3 to EL2 , TFA BL31 returns to U-Boot armv8_switch_to_el2 which checks if this code is running in EL2 (it is) and does the last few steps before starting kernel, and then finally jumps to the kernel .
Thanks for the explanation. I missed the details in 3/3.
BL2(Secure)->U-boot(Secure)->BL31(Secure)-->U-boot(Normal)->Linux Kernel(Normal)
If the fit image is missing the TF-A loadable property for example (and maybe in other corner case scenario?) the kernel will start in EL3 though and fail very early since it needs TF-A BL31 for a few things.
Not quite, even if the TFA loadable is not present, the kernel will still be started through armv8_switch_to_el2() , which will switch the core into EL2 and then start the kernel. The kernel will however likely fail to boot because it will attempt to access PSCI via SMC and there will be nothing there, so it will fail at that point.
There will be a follow up patch for R-Car Gen4, which will extend and enable the current minimal R-Car Gen4 PSCI support in U-Boot, so even if you do not bundle TFA into the fitImage, the kernel won't fail to boot entirely.
With the current R-Car Gen4 PSCI implementation in U-Boot, the kernel can boot and run on a single CPU core only, because the CPU core start and stop callbacks are not implemented and not advertised to Linux yet.
Any other SoC that would like to move TFA start after U-Boot start would also likely have to implement at least minimal PSCI implementation to provide a safety net for users who do not bundle the TFA BL31 PSCI provider blob into the fitImage for some reason. The current R-Car Gen4 PSCI implementation is generic enough, that it can act as a template for such other future SoC specific PSCI implementations.
However, part of the kernel code will actually run in EL3 and that could be enough to mess things up with a carefully crafted kernel image.
Is this actually possible or am I making stuff up? If it is possible, isn't that an issue?
Currently, this is already possible and has happened a few times already at my company with people forgetting to bundle TF-A in U-Boot and then the kernel would boot but stops early (no console output even IIRC). But the bootloader is not THAT often updated in the field compared to kernel (especially if coming from distros) which would now allow to store the TF-A, so this opens an even bigger can of worms?
See above, I think that answers this question.
In a way, it allows to update TF-A likely more securely
That is very much the point of this, yes.
Also, safely update as much as possible in lockstep, for example in case someone has the idea to extend the TFA PSCI implementation with custom SMC call handler(s) and make vendor Linux kernel fork depend on those custom SMC calls for something (*). If the next kernel fork depends on a different set of custom SMC calls (because stable ABI in such cases may not be a guarantee) then updating only the kernel may lead to unbootable system. Worse, updating the bootloader and kernel may lead to kernel boot failure, AND if (usually seldom updated) recovery system kernel did not get updated in lockstep with bootloader, recovery system which depends on the old SMC ABI would also fail to boot, leaving the system unrecoverable.
If both kernel and the custom TFA implementation are updated and booted as part of the same fitImage file, the ABI problem goes away.
Also, it allows for easy experimentation with the TFA, you can keep swapping blobs in and out as needed to test them out.
(*) This is not the case with Renesas to my knowledge , whew !
(which is VERY welcome on Rockchip for example, where we usually start new SoC bringups with a blob from Rockchip and then have upstream TF-A catch up a few years later. Having BL31 part of U-Boot is a huge pain because it is difficult for some devices to do safe updates of U-Boot) but may introduce some bigger security issues?
U-Boot will run in EL3 instead of EL2 in this case, which gives it complete unrestricted access to the entire system. It can act as a debug tool, the way it was designed to work since the very beginning.
Do you have anything specific in mind here ?
Finally, another question, what happens if we have a U-Boot with bundled TF-A BL31 AND it tries to boot a fitimage with a TF-A BL31 loadable?
It very much depends on the SoC specific TFA BL31 start handler implementation (see patch 3/3). It is possible to add check in there to test whether the system is already running in EL2 for example, and if anything responds to PSCI version SMC call, and if so, skip starting the TFA again.
I don't think TFA BL31 can be started in EL2 in any case.
Should we only allow to load tf-a BL31 if running from EL3 (meaning if in EL2, TF-A is expected to be already running)?
That's for the SoC specific handler to decide, see above.