[U-Boot] [RFC] Exporting defaul environment
Hi,
I am thinking about how it is possible to export in a clean way the default environment from u-boot. The general use case happens when the environment must be changed from user space (via fw_setenv or whatever) and no environment was still stored - board boots with default environment.
Up now, the trick is to compile the environment tools (u-boot-fw-utils in Yocto) with exactly the same U-Boot sources. This works, but it is error prone when a new Yocto version is delivered (and maybe with a different u-boot-fw-utils) or even if the bootloader is updated. I am searching for an alterntive method.
The code in tools/env is really generic, but it becomes board specific just for the default environment. If the default environment can read in a reliable way from u-boot itslef (its binary), the tool can search for the default environment instead of linking again, and tools/env becomes "distro capable" - one binary that works on all boards.
I have some thoughts and I would like to share, maybe some of you has better ideas. So what I would like to get directly from the binary is the offset of "default_environment" (default_environment) CONFIG_SYS_TEXT_BASE:
- put this offset in the .img header. Yes, it works just for SPL / u-boot.img (drawback).
- surround default environment with some easy to recognize magic number. Offset is not known, but user space can find it, like
magic (long string) ... default_environment end default environment (long string)
- adding a section in the linker. I think we tried something in the past, see __UBOOT_ENV_SECTION__ (it replaced __PPCENV__ from the old good mpc8xx times..) that is currently unused. I do not like this approach, it can increase (due to alignment) footprint in SPL, where I am starting to have issues for smaller i.MX.
- export the deafult environment with such a tool. This dump can be passed as input to the program in user space. Well, it could be done, but I still suffer if something is changed (sync is required to have always the correct dump, I do not see improvement here).
Any thought ?
Best regards, Stefano
Hi Stefano,
On 2 May 2018 at 02:48, Stefano Babic sbabic@denx.de wrote:
Hi,
I am thinking about how it is possible to export in a clean way the default environment from u-boot. The general use case happens when the environment must be changed from user space (via fw_setenv or whatever) and no environment was still stored - board boots with default environment.
Up now, the trick is to compile the environment tools (u-boot-fw-utils in Yocto) with exactly the same U-Boot sources. This works, but it is error prone when a new Yocto version is delivered (and maybe with a different u-boot-fw-utils) or even if the bootloader is updated. I am searching for an alterntive method.
The code in tools/env is really generic, but it becomes board specific just for the default environment. If the default environment can read in a reliable way from u-boot itslef (its binary), the tool can search for the default environment instead of linking again, and tools/env becomes "distro capable" - one binary that works on all boards.
I have some thoughts and I would like to share, maybe some of you has better ideas. So what I would like to get directly from the binary is the offset of "default_environment" (default_environment) CONFIG_SYS_TEXT_BASE:
- put this offset in the .img header. Yes, it works just for SPL /
u-boot.img (drawback).
How about putting the default environment in a FIT? That is typically easy to find using the magic word.
- surround default environment with some easy to recognize magic number.
Offset is not known, but user space can find it, like
magic (long string) ... default_environment end default environment (long string)
Seems like FIT would take care of this for you.
- adding a section in the linker. I think we tried something in the
past, see __UBOOT_ENV_SECTION__ (it replaced __PPCENV__ from the old good mpc8xx times..) that is currently unused. I do not like this approach, it can increase (due to alignment) footprint in SPL, where I am starting to have issues for smaller i.MX.
Also I wonder if compression might be useful? FIT supports that fairly easily.
- export the deafult environment with such a tool. This dump can be
passed as input to the program in user space. Well, it could be done, but I still suffer if something is changed (sync is required to have always the correct dump, I do not see improvement here).
Any thought ?
Regards, Simon
On 02/05/18 21:32, Simon Glass wrote:
Hi Stefano,
On 2 May 2018 at 02:48, Stefano Babic sbabic@denx.de wrote:
Hi,
I am thinking about how it is possible to export in a clean way the default environment from u-boot. The general use case happens when the environment must be changed from user space (via fw_setenv or whatever) and no environment was still stored - board boots with default environment.
Up now, the trick is to compile the environment tools (u-boot-fw-utils in Yocto) with exactly the same U-Boot sources. This works, but it is error prone when a new Yocto version is delivered (and maybe with a different u-boot-fw-utils) or even if the bootloader is updated. I am searching for an alterntive method.
The code in tools/env is really generic, but it becomes board specific just for the default environment. If the default environment can read in a reliable way from u-boot itslef (its binary), the tool can search for the default environment instead of linking again, and tools/env becomes "distro capable" - one binary that works on all boards.
I have some thoughts and I would like to share, maybe some of you has better ideas. So what I would like to get directly from the binary is the offset of "default_environment" (default_environment) CONFIG_SYS_TEXT_BASE:
- put this offset in the .img header. Yes, it works just for SPL /
u-boot.img (drawback).
How about putting the default environment in a FIT? That is typically easy to find using the magic word.
- surround default environment with some easy to recognize magic number.
Offset is not known, but user space can find it, like
magic (long string) ... default_environment end default environment (long string)Seems like FIT would take care of this for you.
- adding a section in the linker. I think we tried something in the
past, see __UBOOT_ENV_SECTION__ (it replaced __PPCENV__ from the old good mpc8xx times..) that is currently unused. I do not like this approach, it can increase (due to alignment) footprint in SPL, where I am starting to have issues for smaller i.MX.
Also I wonder if compression might be useful? FIT supports that fairly easily.
I like the FIT idea, but in all proposals so far, you still have the problem of locating either the U-Boot binary or the FIT image. It's not always clear where they can be found.
I've been having another idea in the back of my head for while, using a very different approach: Instead of requiring that the tool be able to fall back to a default environment, require that U-Boot write the environment to storage if the CRC check fails when it boots, and remove the ability to write to an uninitialized environment from the tools. This would guarantee that the environment is available in its expected storage location when entering userspace, and would also have the benefit of exposing dangerous situations where a newly flashed root filesystem has a /etc/fw_env.config which is wrong.
The downside is that if the environment somehow gets messed up from userspace, you have to reboot to fix the problem. This could be somewhat, but not completely remedied by requiring a redundant environment.
Hi Kristian,
On 03/05/2018 14:57, Kristian Amlie wrote:
On 02/05/18 21:32, Simon Glass wrote:
Hi Stefano,
On 2 May 2018 at 02:48, Stefano Babic sbabic@denx.de wrote:
Hi,
I am thinking about how it is possible to export in a clean way the default environment from u-boot. The general use case happens when the environment must be changed from user space (via fw_setenv or whatever) and no environment was still stored - board boots with default environment.
Up now, the trick is to compile the environment tools (u-boot-fw-utils in Yocto) with exactly the same U-Boot sources. This works, but it is error prone when a new Yocto version is delivered (and maybe with a different u-boot-fw-utils) or even if the bootloader is updated. I am searching for an alterntive method.
The code in tools/env is really generic, but it becomes board specific just for the default environment. If the default environment can read in a reliable way from u-boot itslef (its binary), the tool can search for the default environment instead of linking again, and tools/env becomes "distro capable" - one binary that works on all boards.
I have some thoughts and I would like to share, maybe some of you has better ideas. So what I would like to get directly from the binary is the offset of "default_environment" (default_environment) CONFIG_SYS_TEXT_BASE:
- put this offset in the .img header. Yes, it works just for SPL /
u-boot.img (drawback).
How about putting the default environment in a FIT? That is typically easy to find using the magic word.
- surround default environment with some easy to recognize magic number.
Offset is not known, but user space can find it, like
magic (long string) ... default_environment end default environment (long string)Seems like FIT would take care of this for you.
- adding a section in the linker. I think we tried something in the
past, see __UBOOT_ENV_SECTION__ (it replaced __PPCENV__ from the old good mpc8xx times..) that is currently unused. I do not like this approach, it can increase (due to alignment) footprint in SPL, where I am starting to have issues for smaller i.MX.
Also I wonder if compression might be useful? FIT supports that fairly easily.
I like the FIT idea, but in all proposals so far, you still have the problem of locating either the U-Boot binary or the FIT image. It's not always clear where they can be found.
Right. Things get even worse when a board can boot from several storages, like from SD, from NOR and so on. It is then difficult to find at runtime which is the correct image to check.
I've been having another idea in the back of my head for while, using a very different approach: Instead of requiring that the tool be able to fall back to a default environment, require that U-Boot write the environment to storage if the CRC check fails when it boots, and remove the ability to write to an uninitialized environment from the tools.
From my experience, this is often not desired by customers that do not
like that flash is touched at first start. It does not work in case of CONFIG_ENV_IS_EMBEDDED, where we should skip the save. Anyway, I agree that this helps in User Space because it is guaranteed that a suitable environment is always present.
This would guarantee that the environment is available in its expected storage location when entering userspace, and would also have the benefit of exposing dangerous situations where a newly flashed root filesystem has a /etc/fw_env.config which is wrong.
Well, if /etc/fw_env.config is broken, we cannot do a lot. But this is like a bug and should be solved during the test phase.
The downside is that if the environment somehow gets messed up from userspace, you have to reboot to fix the problem. This could be somewhat, but not completely remedied by requiring a redundant environment.
If userspace does not know how to handle the environment and the environemnt gets messed up, the board could not boot anymore. We do not know. The redundant environment guarantees us that changing the environment is power-off safe, but there is no transactions mechanism where the bootloader can easy step back to the previous one.
Łukasz told me yesterday that he has already sent some patches to OE-Core to extract the default environment, even if this is not suitable for an "updater" (our use case).
http://lists.openembedded.org/pipermail/openembedded-core/2018-April/150194....
The patch is useful to generate a ready-to-flash image, but the exported default environment could be used as input by the env tools in user space when no environment is found.
Regards, Stefano
On 03/05/18 18:00, Stefano Babic wrote:
On 03/05/2018 14:57, Kristian Amlie wrote:
I've been having another idea in the back of my head for while, using a very different approach: Instead of requiring that the tool be able to fall back to a default environment, require that U-Boot write the environment to storage if the CRC check fails when it boots, and remove the ability to write to an uninitialized environment from the tools.
From my experience, this is often not desired by customers that do not like that flash is touched at first start. It does not work in case of CONFIG_ENV_IS_EMBEDDED, where we should skip the save. Anyway, I agree that this helps in User Space because it is guaranteed that a suitable environment is always present.
As long as it's possible to opt out, I think resiliency and ease of use should take priority over such needs. Making the common case easy, and the hard case possible. Also, with the patch you mentioned from Łukasz below, if the proper environment is already in storage, no writing should be necessary.
This would guarantee that the environment is available in its expected storage location when entering userspace, and would also have the benefit of exposing dangerous situations where a newly flashed root filesystem has a /etc/fw_env.config which is wrong.
Well, if /etc/fw_env.config is broken, we cannot do a lot. But this is like a bug and should be solved during the test phase.
Of course. This is just a bonus, and not an argument for choosing this approach.
Łukasz told me yesterday that he has already sent some patches to OE-Core to extract the default environment, even if this is not suitable for an "updater" (our use case).
http://lists.openembedded.org/pipermail/openembedded-core/2018-April/150194....
The patch is useful to generate a ready-to-flash image, but the exported default environment could be used as input by the env tools in user space when no environment is found.
Hi Kristian,
On 3 May 2018 at 06:57, Kristian Amlie kristian.amlie@northern.tech wrote:
On 02/05/18 21:32, Simon Glass wrote:
Hi Stefano,
On 2 May 2018 at 02:48, Stefano Babic sbabic@denx.de wrote:
Hi,
I am thinking about how it is possible to export in a clean way the default environment from u-boot. The general use case happens when the environment must be changed from user space (via fw_setenv or whatever) and no environment was still stored - board boots with default environment.
Is a magic number not enough to find it?
Alternatively could we put the default environment in /chosen in the device tree before jumping to Linux?
Regards, Simon
participants (3)
-
Kristian Amlie -
Simon Glass -
Stefano Babic