Getting rid of falcon mode
## Introduction
Today we use "falcon mode" to mean "boot linux straight from SPL". This designation makes sense, since falcons "fly at high speed and change direction rapidly" according to Wikipedia.
The way we implement falcon mode is to reserve two areas of storage: * kernel area/partition * dtb area/partition By using some "special cases", and "spl export", SPL can more or less figure out how to skip u-boot.
## The plot twist
People familiar with FIT, will have recognized that the above is achievable with a very basic FIT image. With some advantages:
- No "special cases" in SPL code - Signed kernel images - Signed kernel devicetree - Devicetree overlays - Automatic selection of correct devicetree
## The problems
The advantages of FIT are not obvious by looking at SPL code. A noticeable amount of SPL code is hidden under #ifdef CONFIG_SPL_OS_BOOT, leading one to believe that SPL_OS_BOOT is very important. It must be since it takes up so much code.
Enabling falcon mode is not well documented, and requires a lot of trial and error. I've had to define 7 macros, and one new function to get it working on my board -- and vividly remember the grief. This is an antiquated way of doing things, and completely ignores the u-boot devicetree -- we could just as well have defined those seven values in the dtb.
SPL assumes that it must load u-boot, unless in instances under CONFIG_SPL_OS_BOOT. This has cause me a huge amount of grief and confusion over the past several months. I have no less than three patch series trying to address shortfalls there. It's awful.
## The proposal
I propose we drop falcon mode support for legacy images.
- Drop CONFIG_SPL_OS_BOOT. Support for this is implied by SPL_FIT - Drop the "dtb area/partition". The dtb is derived from the FIT - Drop "spl export"
How do we deal with devicetrees in the FIT then? The options are to use a modified devicetree which has the desired "/chosen" node, or use DTB overlays.
What are the reasons why we shouldn't go this route?
Alex
On Mon, Apr 12, 2021 at 04:32:49PM -0500, Alex G. wrote:
## Introduction
Today we use "falcon mode" to mean "boot linux straight from SPL". This designation makes sense, since falcons "fly at high speed and change direction rapidly" according to Wikipedia.
The way we implement falcon mode is to reserve two areas of storage:
- kernel area/partition
- dtb area/partition
By using some "special cases", and "spl export", SPL can more or less figure out how to skip u-boot.
## The plot twist
People familiar with FIT, will have recognized that the above is achievable with a very basic FIT image. With some advantages:
- No "special cases" in SPL code - Signed kernel images - Signed kernel devicetree - Devicetree overlays - Automatic selection of correct devicetree## The problems
The advantages of FIT are not obvious by looking at SPL code. A noticeable amount of SPL code is hidden under #ifdef CONFIG_SPL_OS_BOOT, leading one to believe that SPL_OS_BOOT is very important. It must be since it takes up so much code.
Enabling falcon mode is not well documented, and requires a lot of trial and error. I've had to define 7 macros, and one new function to get it working on my board -- and vividly remember the grief. This is an antiquated way of doing things, and completely ignores the u-boot devicetree -- we could just as well have defined those seven values in the dtb.
SPL assumes that it must load u-boot, unless in instances under CONFIG_SPL_OS_BOOT. This has cause me a huge amount of grief and confusion over the past several months. I have no less than three patch series trying to address shortfalls there. It's awful.
## The proposal
I propose we drop falcon mode support for legacy images.
- Drop CONFIG_SPL_OS_BOOT. Support for this is implied by SPL_FIT
- Drop the "dtb area/partition". The dtb is derived from the FIT
- Drop "spl export"
How do we deal with devicetrees in the FIT then? The options are to use a modified devicetree which has the desired "/chosen" node, or use DTB overlays.
What are the reasons why we shouldn't go this route?
Please send an RFC that shows what doc/usage/falcon.rst would look like (base it as much or as little on doc/README.falon as you like). Conceptually, yes, this sounds like the right way to go moving forward. It should just be removing code, except for around the "boot to U-Boot not OS" stuff. Thanks!
Hi,
On Mon, Apr 12, 2021 at 04:32:49PM -0500, Alex G. wrote:
## Introduction
Today we use "falcon mode" to mean "boot linux straight from SPL". This designation makes sense, since falcons "fly at high speed and change direction rapidly" according to Wikipedia.
The way we implement falcon mode is to reserve two areas of storage:
- kernel area/partition
- dtb area/partition
By using some "special cases", and "spl export", SPL can more or less figure out how to skip u-boot.
## The plot twist
People familiar with FIT, will have recognized that the above is achievable with a very basic FIT image. With some advantages:
- No "special cases" in SPL code - Signed kernel images - Signed kernel devicetree - Devicetree overlays - Automatic selection of correct devicetree## The problems
The advantages of FIT are not obvious by looking at SPL code. A noticeable amount of SPL code is hidden under #ifdef CONFIG_SPL_OS_BOOT, leading one to believe that SPL_OS_BOOT is very important. It must be since it takes up so much code.
Enabling falcon mode is not well documented, and requires a lot of trial and error. I've had to define 7 macros, and one new function to get it working on my board -- and vividly remember the grief. This is an antiquated way of doing things, and completely ignores the u-boot devicetree -- we could just as well have defined those seven values in the dtb.
SPL assumes that it must load u-boot, unless in instances under CONFIG_SPL_OS_BOOT. This has cause me a huge amount of grief and confusion over the past several months. I have no less than three patch series trying to address shortfalls there. It's awful.
## The proposal
I propose we drop falcon mode support for legacy images.
- Drop CONFIG_SPL_OS_BOOT. Support for this is implied by SPL_FIT
- Drop the "dtb area/partition". The dtb is derived from the FIT
- Drop "spl export"
How do we deal with devicetrees in the FIT then? The options are to use a modified devicetree which has the desired "/chosen" node, or use DTB overlays.
What are the reasons why we shouldn't go this route?
I can see at least two, that are mainly due to a FIT image being essentially a compiled device tree:
- Not all platforms have enough head-space in their SPL to have libfdt in addition to what is already there.
- Parsing a DT is fairly slow too. Last time I checked booting a FIT image took around 150ms more than a legacy image on a Cortex-A7. If one is using the Falcon mode in the first place chances are it's to improve the boot-time, so this seems like a step backward.
Maxime
Hi Maxime,
On 4/13/21 3:56 AM, Maxime Ripard wrote:
Hi,
On Mon, Apr 12, 2021 at 04:32:49PM -0500, Alex G. wrote:
## Introduction
Today we use "falcon mode" to mean "boot linux straight from SPL". This designation makes sense, since falcons "fly at high speed and change direction rapidly" according to Wikipedia.
The way we implement falcon mode is to reserve two areas of storage:
- kernel area/partition
- dtb area/partition
By using some "special cases", and "spl export", SPL can more or less figure out how to skip u-boot.
## The plot twist
People familiar with FIT, will have recognized that the above is achievable with a very basic FIT image. With some advantages:
- No "special cases" in SPL code - Signed kernel images - Signed kernel devicetree - Devicetree overlays - Automatic selection of correct devicetree## The problems
The advantages of FIT are not obvious by looking at SPL code. A noticeable amount of SPL code is hidden under #ifdef CONFIG_SPL_OS_BOOT, leading one to believe that SPL_OS_BOOT is very important. It must be since it takes up so much code.
Enabling falcon mode is not well documented, and requires a lot of trial and error. I've had to define 7 macros, and one new function to get it working on my board -- and vividly remember the grief. This is an antiquated way of doing things, and completely ignores the u-boot devicetree -- we could just as well have defined those seven values in the dtb.
SPL assumes that it must load u-boot, unless in instances under CONFIG_SPL_OS_BOOT. This has cause me a huge amount of grief and confusion over the past several months. I have no less than three patch series trying to address shortfalls there. It's awful.
## The proposal
I propose we drop falcon mode support for legacy images.
- Drop CONFIG_SPL_OS_BOOT. Support for this is implied by SPL_FIT
- Drop the "dtb area/partition". The dtb is derived from the FIT
- Drop "spl export"
How do we deal with devicetrees in the FIT then? The options are to use a modified devicetree which has the desired "/chosen" node, or use DTB overlays.
What are the reasons why we shouldn't go this route?
I can see at least two, that are mainly due to a FIT image being essentially a compiled device tree:
- Not all platforms have enough head-space in their SPL to have libfdt in addition to what is already there.
Do we know which platforms fall in this category? We can investigate if it might be possible to disable just enough of the legacy support to make room for libfdt.
- Parsing a DT is fairly slow too. Last time I checked booting a FIT image took around 150ms more than a legacy image on a Cortex-A7. If one is using the Falcon mode in the first place chances are it's to improve the boot-time, so this seems like a step backward.
I'm skeptical of the 150ms delta, as I also did heavy measurements on this a few months back. But maybe there's something that causes certain platforms to bog down, (caching issues ?).
I've used grabserial (e.g. "grabserial -d /dev/ttyACM0 -b 2000000 -m"U-Boot SPL" -t") to time the boot. If you have similar logs, maybe there's something there that tells us why parsing the FIT bogs down.
Alex
On Tue, Apr 13, 2021 at 09:08:01AM -0500, Alex G. wrote:
Hi Maxime,
On 4/13/21 3:56 AM, Maxime Ripard wrote:
Hi,
On Mon, Apr 12, 2021 at 04:32:49PM -0500, Alex G. wrote:
## Introduction
Today we use "falcon mode" to mean "boot linux straight from SPL". This designation makes sense, since falcons "fly at high speed and change direction rapidly" according to Wikipedia.
The way we implement falcon mode is to reserve two areas of storage:
- kernel area/partition
- dtb area/partition
By using some "special cases", and "spl export", SPL can more or less figure out how to skip u-boot.
## The plot twist
People familiar with FIT, will have recognized that the above is achievable with a very basic FIT image. With some advantages:
- No "special cases" in SPL code - Signed kernel images - Signed kernel devicetree - Devicetree overlays - Automatic selection of correct devicetree## The problems
The advantages of FIT are not obvious by looking at SPL code. A noticeable amount of SPL code is hidden under #ifdef CONFIG_SPL_OS_BOOT, leading one to believe that SPL_OS_BOOT is very important. It must be since it takes up so much code.
Enabling falcon mode is not well documented, and requires a lot of trial and error. I've had to define 7 macros, and one new function to get it working on my board -- and vividly remember the grief. This is an antiquated way of doing things, and completely ignores the u-boot devicetree -- we could just as well have defined those seven values in the dtb.
SPL assumes that it must load u-boot, unless in instances under CONFIG_SPL_OS_BOOT. This has cause me a huge amount of grief and confusion over the past several months. I have no less than three patch series trying to address shortfalls there. It's awful.
## The proposal
I propose we drop falcon mode support for legacy images.
- Drop CONFIG_SPL_OS_BOOT. Support for this is implied by SPL_FIT
- Drop the "dtb area/partition". The dtb is derived from the FIT
- Drop "spl export"
How do we deal with devicetrees in the FIT then? The options are to use a modified devicetree which has the desired "/chosen" node, or use DTB overlays.
What are the reasons why we shouldn't go this route?
I can see at least two, that are mainly due to a FIT image being essentially a compiled device tree:
- Not all platforms have enough head-space in their SPL to have libfdt in addition to what is already there.
Do we know which platforms fall in this category? We can investigate if it might be possible to disable just enough of the legacy support to make room for libfdt.
sunxi is both modern and small, so a good thing to benchmark.
- Parsing a DT is fairly slow too. Last time I checked booting a FIT image took around 150ms more than a legacy image on a Cortex-A7. If one is using the Falcon mode in the first place chances are it's to improve the boot-time, so this seems like a step backward.
I'm skeptical of the 150ms delta, as I also did heavy measurements on this a few months back. But maybe there's something that causes certain platforms to bog down, (caching issues ?).
I've used grabserial (e.g. "grabserial -d /dev/ttyACM0 -b 2000000 -m"U-Boot SPL" -t") to time the boot. If you have similar logs, maybe there's something there that tells us why parsing the FIT bogs down.
Since you have this working on some platforms, can you show some logs? Sample configurations? Thanks.
On Tue, Apr 13, 2021 at 10:13:50AM -0400, Tom Rini wrote:
On Tue, Apr 13, 2021 at 09:08:01AM -0500, Alex G. wrote:
Hi Maxime,
On 4/13/21 3:56 AM, Maxime Ripard wrote:
Hi,
On Mon, Apr 12, 2021 at 04:32:49PM -0500, Alex G. wrote:
## Introduction
Today we use "falcon mode" to mean "boot linux straight from SPL". This designation makes sense, since falcons "fly at high speed and change direction rapidly" according to Wikipedia.
The way we implement falcon mode is to reserve two areas of storage:
- kernel area/partition
- dtb area/partition
By using some "special cases", and "spl export", SPL can more or less figure out how to skip u-boot.
## The plot twist
People familiar with FIT, will have recognized that the above is achievable with a very basic FIT image. With some advantages:
- No "special cases" in SPL code - Signed kernel images - Signed kernel devicetree - Devicetree overlays - Automatic selection of correct devicetree## The problems
The advantages of FIT are not obvious by looking at SPL code. A noticeable amount of SPL code is hidden under #ifdef CONFIG_SPL_OS_BOOT, leading one to believe that SPL_OS_BOOT is very important. It must be since it takes up so much code.
Enabling falcon mode is not well documented, and requires a lot of trial and error. I've had to define 7 macros, and one new function to get it working on my board -- and vividly remember the grief. This is an antiquated way of doing things, and completely ignores the u-boot devicetree -- we could just as well have defined those seven values in the dtb.
SPL assumes that it must load u-boot, unless in instances under CONFIG_SPL_OS_BOOT. This has cause me a huge amount of grief and confusion over the past several months. I have no less than three patch series trying to address shortfalls there. It's awful.
## The proposal
I propose we drop falcon mode support for legacy images.
- Drop CONFIG_SPL_OS_BOOT. Support for this is implied by SPL_FIT
- Drop the "dtb area/partition". The dtb is derived from the FIT
- Drop "spl export"
How do we deal with devicetrees in the FIT then? The options are to use a modified devicetree which has the desired "/chosen" node, or use DTB overlays.
What are the reasons why we shouldn't go this route?
I can see at least two, that are mainly due to a FIT image being essentially a compiled device tree:
- Not all platforms have enough head-space in their SPL to have libfdt in addition to what is already there.
Do we know which platforms fall in this category? We can investigate if it might be possible to disable just enough of the legacy support to make room for libfdt.
sunxi is both modern and small, so a good thing to benchmark.
Another issue with sunxi is that it implements PSCI in U-Boot, so you'll need in the SPL as well if you want to do falcon boot. This is doable on the later SoCs that have a bigger SRAM, but for that kind of test you'll want to test on the A10 or A13
Maxime
On Tue, Apr 13, 2021 at 09:08:01AM -0500, Alex G. wrote:
On 4/13/21 3:56 AM, Maxime Ripard wrote:
On Mon, Apr 12, 2021 at 04:32:49PM -0500, Alex G. wrote:
## Introduction
Today we use "falcon mode" to mean "boot linux straight from SPL". This designation makes sense, since falcons "fly at high speed and change direction rapidly" according to Wikipedia.
The way we implement falcon mode is to reserve two areas of storage:
- kernel area/partition
- dtb area/partition
By using some "special cases", and "spl export", SPL can more or less figure out how to skip u-boot.
## The plot twist
People familiar with FIT, will have recognized that the above is achievable with a very basic FIT image. With some advantages:
- No "special cases" in SPL code - Signed kernel images - Signed kernel devicetree - Devicetree overlays - Automatic selection of correct devicetree## The problems
The advantages of FIT are not obvious by looking at SPL code. A noticeable amount of SPL code is hidden under #ifdef CONFIG_SPL_OS_BOOT, leading one to believe that SPL_OS_BOOT is very important. It must be since it takes up so much code.
Enabling falcon mode is not well documented, and requires a lot of trial and error. I've had to define 7 macros, and one new function to get it working on my board -- and vividly remember the grief. This is an antiquated way of doing things, and completely ignores the u-boot devicetree -- we could just as well have defined those seven values in the dtb.
SPL assumes that it must load u-boot, unless in instances under CONFIG_SPL_OS_BOOT. This has cause me a huge amount of grief and confusion over the past several months. I have no less than three patch series trying to address shortfalls there. It's awful.
## The proposal
I propose we drop falcon mode support for legacy images.
- Drop CONFIG_SPL_OS_BOOT. Support for this is implied by SPL_FIT
- Drop the "dtb area/partition". The dtb is derived from the FIT
- Drop "spl export"
How do we deal with devicetrees in the FIT then? The options are to use a modified devicetree which has the desired "/chosen" node, or use DTB overlays.
What are the reasons why we shouldn't go this route?
I can see at least two, that are mainly due to a FIT image being essentially a compiled device tree:
- Not all platforms have enough head-space in their SPL to have libfdt in addition to what is already there.
Do we know which platforms fall in this category? We can investigate if it might be possible to disable just enough of the legacy support to make room for libfdt.
The earliest Allwinner SoC (A10, A13) must be fairly close to their limit. I haven't checked for a while but they had 13kB overall iirc.
- Parsing a DT is fairly slow too. Last time I checked booting a FIT image took around 150ms more than a legacy image on a Cortex-A7. If one is using the Falcon mode in the first place chances are it's to improve the boot-time, so this seems like a step backward.
I'm skeptical of the 150ms delta, as I also did heavy measurements on this a few months back. But maybe there's something that causes certain platforms to bog down, (caching issues ?).
I found my old notes back, and while the 150ms figure was correct, the FIT parsing was only half that (the other one was actually patching the DT to fixup whatever was needed).
80ms is still fairly expensive to just parse the image. I'm not sure indeed whether sunxi has the MMU enabled in the SPL though. I think it is but might very well be wrong.
I've used grabserial (e.g. "grabserial -d /dev/ttyACM0 -b 2000000 -m"U-Boot SPL" -t") to time the boot. If you have similar logs, maybe there's something there that tells us why parsing the FIT bogs down.
Unfortunately this was part of my former job and don't have access to them anymore :/
Maxime
Hi,
On Tue, 13 Apr 2021 at 09:32, Alex G. mr.nuke.me@gmail.com wrote:
## Introduction
Today we use "falcon mode" to mean "boot linux straight from SPL". This designation makes sense, since falcons "fly at high speed and change direction rapidly" according to Wikipedia.
The way we implement falcon mode is to reserve two areas of storage:
- kernel area/partition
- dtb area/partition
By using some "special cases", and "spl export", SPL can more or less figure out how to skip u-boot.
## The plot twist
People familiar with FIT, will have recognized that the above is achievable with a very basic FIT image. With some advantages:
- No "special cases" in SPL code - Signed kernel images - Signed kernel devicetree - Devicetree overlays - Automatic selection of correct devicetree## The problems
The advantages of FIT are not obvious by looking at SPL code. A noticeable amount of SPL code is hidden under #ifdef CONFIG_SPL_OS_BOOT, leading one to believe that SPL_OS_BOOT is very important. It must be since it takes up so much code.
Enabling falcon mode is not well documented, and requires a lot of trial and error. I've had to define 7 macros, and one new function to get it working on my board -- and vividly remember the grief. This is an antiquated way of doing things, and completely ignores the u-boot devicetree -- we could just as well have defined those seven values in the dtb.
SPL assumes that it must load u-boot, unless in instances under CONFIG_SPL_OS_BOOT. This has cause me a huge amount of grief and confusion over the past several months. I have no less than three patch series trying to address shortfalls there. It's awful.
## The proposal
I propose we drop falcon mode support for legacy images.
- Drop CONFIG_SPL_OS_BOOT. Support for this is implied by SPL_FIT
- Drop the "dtb area/partition". The dtb is derived from the FIT
- Drop "spl export"
How do we deal with devicetrees in the FIT then? The options are to use a modified devicetree which has the desired "/chosen" node, or use DTB overlays.
What are the reasons why we shouldn't go this route?
I think this makes sense, on the basis that it is a legacy image and people can always use the U-Boot path if needed.
I believe Falcon boot made a lot more sense when the cache was off. But at least in my experience, we were able to get through two SPLs and two U-Boots and boot a kernel about 800ms on a Cortex-A15, so Falcon mode might not be so relevant anymore, and supporting a legacy image seems like unnecessary complexity.
Regards, Simon
On Wed, Apr 14, 2021 at 08:37:07PM +0100, Simon Glass wrote:
Hi,
On Tue, 13 Apr 2021 at 09:32, Alex G. mr.nuke.me@gmail.com wrote:
## Introduction
Today we use "falcon mode" to mean "boot linux straight from SPL". This designation makes sense, since falcons "fly at high speed and change direction rapidly" according to Wikipedia.
The way we implement falcon mode is to reserve two areas of storage:
- kernel area/partition
- dtb area/partition
By using some "special cases", and "spl export", SPL can more or less figure out how to skip u-boot.
## The plot twist
People familiar with FIT, will have recognized that the above is achievable with a very basic FIT image. With some advantages:
- No "special cases" in SPL code - Signed kernel images - Signed kernel devicetree - Devicetree overlays - Automatic selection of correct devicetree## The problems
The advantages of FIT are not obvious by looking at SPL code. A noticeable amount of SPL code is hidden under #ifdef CONFIG_SPL_OS_BOOT, leading one to believe that SPL_OS_BOOT is very important. It must be since it takes up so much code.
Enabling falcon mode is not well documented, and requires a lot of trial and error. I've had to define 7 macros, and one new function to get it working on my board -- and vividly remember the grief. This is an antiquated way of doing things, and completely ignores the u-boot devicetree -- we could just as well have defined those seven values in the dtb.
SPL assumes that it must load u-boot, unless in instances under CONFIG_SPL_OS_BOOT. This has cause me a huge amount of grief and confusion over the past several months. I have no less than three patch series trying to address shortfalls there. It's awful.
## The proposal
I propose we drop falcon mode support for legacy images.
- Drop CONFIG_SPL_OS_BOOT. Support for this is implied by SPL_FIT
- Drop the "dtb area/partition". The dtb is derived from the FIT
- Drop "spl export"
How do we deal with devicetrees in the FIT then? The options are to use a modified devicetree which has the desired "/chosen" node, or use DTB overlays.
What are the reasons why we shouldn't go this route?
I think this makes sense, on the basis that it is a legacy image and people can always use the U-Boot path if needed.
I believe Falcon boot made a lot more sense when the cache was off. But at least in my experience, we were able to get through two SPLs and two U-Boots and boot a kernel about 800ms on a Cortex-A15, so Falcon mode might not be so relevant anymore, and supporting a legacy image seems like unnecessary complexity.
I'm curious where the end of that 800ms is, and I think we might need to post grabserial logs so everyone is on the same page about where / when we're at in booting.
participants (4)
-
Alex G. -
Maxime Ripard -
Simon Glass -
Tom Rini