[U-Boot] SWUpdate - U-Boot environment library dependency
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
* Distros like Debian cannot provide a devel package for SWUpdate like u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
* If we have a board with U-Boot already preinstalled and want to compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
* U-Boot does not provide any default means to install a development library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Thanks, Andreas
On 04.09.2018 12:30, Andreas Reichel wrote:
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
Distros like Debian cannot provide a devel package for SWUpdate like u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
If we have a board with U-Boot already preinstalled and want to compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
U-Boot does not provide any default means to install a development library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread. I do think the current situation should be improved, making fw_setenv independent of the target that is built.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
Simon
On 11/20/2018 10:11 PM, Simon Goldschmidt wrote:
On 04.09.2018 12:30, Andreas Reichel wrote:
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
- Distros like Debian cannot provide a devel package for SWUpdate like
u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
- If we have a board with U-Boot already preinstalled and want to
compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
- U-Boot does not provide any default means to install a development
library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread. I do think the current situation should be improved, making fw_setenv independent of the target that is built.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
Wouldn't it make sense to have U-Boot build the env code as LGPL library, in addition to executable ?
Hi Marek,
On 21/11/18 00:10, Marek Vasut wrote:
On 11/20/2018 10:11 PM, Simon Goldschmidt wrote:
On 04.09.2018 12:30, Andreas Reichel wrote:
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
- Distros like Debian cannot provide a devel package for SWUpdate like
u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
- If we have a board with U-Boot already preinstalled and want to
compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
- U-Boot does not provide any default means to install a development
library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread. I do think the current situation should be improved, making fw_setenv independent of the target that is built.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
Wouldn't it make sense to have U-Boot build the env code as LGPL library, in addition to executable ?
+1
The library is already built (tools/env/lib.a), but IMHO the best thing is to export it official and let that in OE we have a u-boot-fw-tools-dev with header and library.
I would like to see the library under LGPL instead of GPL2, too, and I raised this issue when I started SWUpdate, but I was not very active to promote this. Tom, Wolfgang, is there chances to switch license ?
A env library is very welcomed by many customers, because they could integrate it in their application if license allows it.
Best regards, Stefano Babic
Dear Stefano,
In message 96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de you wrote:
I would like to see the library under LGPL instead of GPL2, too, and I raised this issue when I started SWUpdate, but I was not very active to promote this. Tom, Wolfgang, is there chances to switch license ?
Relicensing requires permission from all who contributed to that code.
Consider mine as granted.
But someone hat to invest the efforts to analyze the code so we know who to ask, and then collect all the permissions...
A env library is very welcomed by many customers, because they could integrate it in their application if license allows it.
Agreed.
Best regards,
Wolfgang Denk
On 11/21/2018 10:20 AM, Wolfgang Denk wrote:
Dear Stefano,
Hi,
In message 96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de you wrote:
I would like to see the library under LGPL instead of GPL2, too, and I raised this issue when I started SWUpdate, but I was not very active to promote this. Tom, Wolfgang, is there chances to switch license ?
Relicensing requires permission from all who contributed to that code.
Consider mine as granted.
But someone hat to invest the efforts to analyze the code so we know who to ask, and then collect all the permissions...
A env library is very welcomed by many customers, because they could integrate it in their application if license allows it.
Agreed.
Then again, U-Boot environment structure is trivial, crc, flags, data, there is no complexity involved. There is probably some complexity in the backing store access stuff (MTD, block devs, legacy stuff), but that should either use some MTD utils libs, basic block access primitives or be given a once-over and possibly be dropped.
I think prototyping a library from scratch that's LGPL would be a few days' work and the benefit would be tremendous all over.
Hi Marek,
On 21/11/18 15:31, Marek Vasut wrote:
On 11/21/2018 10:20 AM, Wolfgang Denk wrote:
Dear Stefano,
Hi,
In message 96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de you wrote:
I would like to see the library under LGPL instead of GPL2, too, and I raised this issue when I started SWUpdate, but I was not very active to promote this. Tom, Wolfgang, is there chances to switch license ?
Relicensing requires permission from all who contributed to that code.
Consider mine as granted.
But someone hat to invest the efforts to analyze the code so we know who to ask, and then collect all the permissions...
A env library is very welcomed by many customers, because they could integrate it in their application if license allows it.
Agreed.
Then again, U-Boot environment structure is trivial, crc, flags, data, there is no complexity involved. There is probably some complexity in the backing store access stuff (MTD, block devs, legacy stuff), but that should either use some MTD utils libs, basic block access primitives or be given a once-over and possibly be dropped.
I think prototyping a library from scratch that's LGPL would be a few days' work and the benefit would be tremendous all over.
I confess I had the same idea - why not ignore the code in tools/env (they have also some drawbacks, see the locking mechanism in my previous e-mail) and start with a new library from scratch ? Then LGPL is not an issue anymore, it is a new development. And I already took this way for "grubenv" (I had to, grubenv is not license compatible).
But something in my head is telling me that this is not else as a fork of u-boot (ok, a partial fork, just tools/env). And if the U-Boot community decides to follow other ways for the environment, the "forked" project aka "new library" should follow. But yes, I guess it is easier, and I agree with you this is just a few days work.
This can solve the issue with license - the other topic regarding "default" (or "initialization") environment is not changed by an external library.
Best regards, Stefano Babic
On Wednesday, November 21, 2018, Stefano Babic sbabic@denx.de wrote:
Hi Marek,
On 21/11/18 15:31, Marek Vasut wrote:
On 11/21/2018 10:20 AM, Wolfgang Denk wrote:
Dear Stefano,
Hi,
In message 96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de you wrote:
I would like to see the library under LGPL instead of GPL2, too, and I raised this issue when I started SWUpdate, but I was not very active to promote this. Tom, Wolfgang, is there chances to switch license ?
Relicensing requires permission from all who contributed to that code.
Consider mine as granted.
But someone hat to invest the efforts to analyze the code so we know who to ask, and then collect all the permissions...
A env library is very welcomed by many customers, because they could integrate it in their application if license allows it.
Agreed.
Then again, U-Boot environment structure is trivial, crc, flags, data, there is no complexity involved. There is probably some complexity in the backing store access stuff (MTD, block devs, legacy stuff), but that should either use some MTD utils libs, basic block access primitives or be given a once-over and possibly be dropped.
I think prototyping a library from scratch that's LGPL would be a few days' work and the benefit would be tremendous all over.
I confess I had the same idea - why not ignore the code in tools/env (they have also some drawbacks, see the locking mechanism in my previous e-mail) and start with a new library from scratch ? Then LGPL is not an issue anymore, it is a new development. And I already took this way for "grubenv" (I had to, grubenv is not license compatible).
But something in my head is telling me that this is not else as a fork of u-boot (ok, a partial fork, just tools/env). And if the U-Boot community decides to follow other ways for the environment, the "forked" project aka "new library" should follow. But yes, I guess it is easier, and I agree with you this is just a few days work.
Once you had this new lgpl library, would it be a big problem to backport it to the U-Boot tree? Wouldn't that also ensure future commiters accept the lgpl license when adding their new code?
Simon
On 11/21/2018 04:31 PM, Simon Goldschmidt wrote:
On Wednesday, November 21, 2018, Stefano Babic <sbabic@denx.de mailto:sbabic@denx.de> wrote:
Hi Marek, On 21/11/18 15:31, Marek Vasut wrote: > On 11/21/2018 10:20 AM, Wolfgang Denk wrote: >> Dear Stefano, > > Hi, > >> In message <96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de <mailto:96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de>> you wrote: >>> >>> I would like to see the library under LGPL instead of GPL2, too, and I >>> raised this issue when I started SWUpdate, but I was not very active to >>> promote this. Tom, Wolfgang, is there chances to switch license ? >> >> Relicensing requires permission from all who contributed to that >> code. >> >> Consider mine as granted. >> >> But someone hat to invest the efforts to analyze the code so we >> know who to ask, and then collect all the permissions... >> >>> A env library is very welcomed by many customers, because they could >>> integrate it in their application if license allows it. >> >> Agreed. > > Then again, U-Boot environment structure is trivial, crc, flags, data, > there is no complexity involved. There is probably some complexity in > the backing store access stuff (MTD, block devs, legacy stuff), but that > should either use some MTD utils libs, basic block access primitives or > be given a once-over and possibly be dropped. > > I think prototyping a library from scratch that's LGPL would be a few > days' work and the benefit would be tremendous all over. I confess I had the same idea - why not ignore the code in tools/env (they have also some drawbacks, see the locking mechanism in my previous e-mail) and start with a new library from scratch ? Then LGPL is not an issue anymore, it is a new development. And I already took this way for "grubenv" (I had to, grubenv is not license compatible). But something in my head is telling me that this is not else as a fork of u-boot (ok, a partial fork, just tools/env). And if the U-Boot community decides to follow other ways for the environment, the "forked" project aka "new library" should follow. But yes, I guess it is easier, and I agree with you this is just a few days work.Once you had this new lgpl library, would it be a big problem to backport it to the U-Boot tree? Wouldn't that also ensure future commiters accept the lgpl license when adding their new code?
Maybe it can be part of the U-Boot tree and the old env tools can just be migrated over, make them a husk which calls the new lib .
On Wed, Nov 21, 2018 at 7:10 AM Stefano Babic sbabic@denx.de wrote:
On 21/11/18 00:10, Marek Vasut wrote:
On 11/20/2018 10:11 PM, Simon Goldschmidt wrote:
On 04.09.2018 12:30, Andreas Reichel wrote:
The library is already built (tools/env/lib.a), but IMHO the best thing is to export it official and let that in OE we have a u-boot-fw-tools-dev with header and library.
I would like to see the library under LGPL instead of GPL2, too, and I raised this issue when I started SWUpdate, but I was not very active to promote this. Tom, Wolfgang, is there chances to switch license ?
From OE perspective, it ought to be a different recipe as
libu-boot-env or similar, so it does allow to be used without being connected to the BSP in use. Especially if the option to load default env from a file or similar is added.
Dear Simon,
In message 90014ca9-64d8-8e38-a6c0-4c06e991c92c@gmail.com you wrote:
However, this approach creates several difficulties:
As usual, "difficulties" are made for solving.
I mean, none of these are technically challenging in any way.
- Distros like Debian cannot provide a devel package for SWUpdate like u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent.
Thta's fine.
If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
One possible way to fix this is not to use a compiled in default environment for the fw_env tools (or only use this as a very last resort), but instead provide a configuration option to read the default settings from a file. In this case the user could provide his own, board specific default environment.
- If we have a board with U-Boot already preinstalled and want to compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
- Why does building the U-Boot package not also create these libraries, either directly or as a separate package? - What prevents the creation of such a dev package that only builds the needed lib?
- U-Boot does not provide any default means to install a development library. Thus anything we modify on-top might break with the next version.
Why don't you add this, then? Patches are welcome!
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread. I do think the current situation should be improved, making fw_setenv independent of the target that is built.
To be honest, I think this is not a clever approach. This whole concept of the compiled in default environment is antediluvian. It comes from a time where we were booting from parallel NOR flasah (or other boot ROM where you directly started executing code) and where we wanted to make sure that we can read the console baud rate very, very soon. Today, many boards don't have environment access in the SPL, and nobody complains that these run with a fixed baud rate only.
Did anybody ever ask who actually _needs_ this compiled in default environment? We could provide other, more efficient configurations if we drop a few of the old requirements (which are not met by many recent boards anyway).
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
IMO this is just a matter of taste. Both ways should be possible.
Best regards,
Wolfgang Denk
Hi Wolfgang,
On 21/11/18 09:31, Wolfgang Denk wrote:
Dear Simon,
In message 90014ca9-64d8-8e38-a6c0-4c06e991c92c@gmail.com you wrote:
However, this approach creates several difficulties:
As usual, "difficulties" are made for solving.
I mean, none of these are technically challenging in any way.
- Distros like Debian cannot provide a devel package for SWUpdate like u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent.
Thta's fine.
If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
One possible way to fix this is not to use a compiled in default environment for the fw_env tools (or only use this as a very last resort), but instead provide a configuration option to read the default settings from a file. In this case the user could provide his own, board specific default environment.
I tried this, but it is very difficult to maintain this in sync, specially in projects where the number of developers is > 1 (all projects). If the file does not correspond to the linked environment, strange behaviors are expected.
Much better will be if u-boot generates in its build process this file. It becomes just another artifact like SPL or u-boot.img and we are sure that it is in sync.
- If we have a board with U-Boot already preinstalled and want to compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
- Why does building the U-Boot package not also create these libraries, either directly or as a separate package?
Issue is that there is not a u-boot-dev package
- What prevents the creation of such a dev package that only builds the needed lib?
I think Simon wants to prevent to rebuild u-boot.
- U-Boot does not provide any default means to install a development library. Thus anything we modify on-top might break with the next version.
Why don't you add this, then? Patches are welcome!
+1
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread. I do think the current situation should be improved, making fw_setenv independent of the target that is built.
To be honest, I think this is not a clever approach. This whole concept of the compiled in default environment is antediluvian.
Fully agree - I am glad if there will be an improvement.
To be clear: the issue just raises if a board runs with default environment. User Space cannot know this environment because it is linked in the bootloader - the current way is to link the same environment in tools/env (that means, u-boot-fw-tools in OE).
There is no issue at all for boards that are not using default environment at all, that means they rely to have always a valid environment in flash.
It comes from a time where we were booting from parallel NOR flasah (or other boot ROM where you directly started executing code) and where we wanted to make sure that we can read the console baud rate very, very soon. Today, many boards don't have environment access in the SPL, and nobody complains that these run with a fixed baud rate only.
Did anybody ever ask who actually _needs_ this compiled in default environment?
Rather, quite all boards....see CONFIG_EXTRA_ENV in include/configs/* and the number of patches sent just to fix something in it.
UMHO the default environment should be like an extra image in u-boot, like an header added by mkimage, because there is no changes in code and behavior. Not to mention that CONFIG_EXTRA_ENV is really error prone with the requirement to put each line as a C string with trailing "\0".
We could provide other, more efficient configurations if we drop a few of the old requirements (which are not met by many recent boards anyway).
All recent boards, specially the one that switched to distro_, use the default environment. Thing is that there is no need to add an extra environment and the board can boot without any intervention by operator (or by factory) simply after flashing the bootloader.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
IMO this is just a matter of taste. Both ways should be possible.
Best regards, Stefano Babic
On Wed, Nov 21, 2018 at 10:33 AM Stefano Babic sbabic@denx.de wrote:
Hi Wolfgang,
On 21/11/18 09:31, Wolfgang Denk wrote:
Dear Simon,
In message 90014ca9-64d8-8e38-a6c0-4c06e991c92c@gmail.com you wrote:
However, this approach creates several difficulties:
As usual, "difficulties" are made for solving.
I mean, none of these are technically challenging in any way.
- Distros like Debian cannot provide a devel package for SWUpdate like u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent.
Thta's fine.
If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
One possible way to fix this is not to use a compiled in default environment for the fw_env tools (or only use this as a very last resort), but instead provide a configuration option to read the default settings from a file. In this case the user could provide his own, board specific default environment.
I tried this, but it is very difficult to maintain this in sync, specially in projects where the number of developers is > 1 (all projects). If the file does not correspond to the linked environment, strange behaviors are expected.
Much better will be if u-boot generates in its build process this file. It becomes just another artifact like SPL or u-boot.img and we are sure that it is in sync.
- If we have a board with U-Boot already preinstalled and want to compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
- Why does building the U-Boot package not also create these libraries, either directly or as a separate package?
Issue is that there is not a u-boot-dev package
- What prevents the creation of such a dev package that only builds the needed lib?
I think Simon wants to prevent to rebuild u-boot.
No, I want to prevent recompiling the u-boot-tools package (fw_setenv) and Swupdate after changing U-Boot's default environment.
In fact, we are running a Debian-based system and I would like to run upstream Debian u-boot-tools. And of course running an upstream swupdate package would be preferred as well.
One way to fix this would be to let fw_setenv somehow write an environment that doesn't need to know the default environment and is loaded in a "add" way, not replacing the whole environment as we normally do on load.
And security wise, I think linking in things as static libs in Swupdate is not a good decision as you can end up with 2 different versions of the same sources in your target. I can understand calling external binaries is not a good solution as well. Using shared libs would be best. That would mean u-boot-tools fw_setenv would use a shared library to access the environment and swupdate could use it, too.
- U-Boot does not provide any default means to install a development library. Thus anything we modify on-top might break with the next version.
Why don't you add this, then? Patches are welcome!
+1
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread. I do think the current situation should be improved, making fw_setenv independent of the target that is built.
To be honest, I think this is not a clever approach. This whole concept of the compiled in default environment is antediluvian.
Fully agree - I am glad if there will be an improvement.
To be clear: the issue just raises if a board runs with default environment. User Space cannot know this environment because it is linked in the bootloader - the current way is to link the same environment in tools/env (that means, u-boot-fw-tools in OE).
There is no issue at all for boards that are not using default environment at all, that means they rely to have always a valid environment in flash.
It comes from a time where we were booting from parallel NOR flasah (or other boot ROM where you directly started executing code) and where we wanted to make sure that we can read the console baud rate very, very soon. Today, many boards don't have environment access in the SPL, and nobody complains that these run with a fixed baud rate only.
Did anybody ever ask who actually _needs_ this compiled in default environment?
In a secure boot environment, you cannot allow to load the environment from an untrusted source. We need a default environment in this case.
But there are better ways than compiling it in via C headers, of course!
Simon
Rather, quite all boards....see CONFIG_EXTRA_ENV in include/configs/* and the number of patches sent just to fix something in it.
UMHO the default environment should be like an extra image in u-boot, like an header added by mkimage, because there is no changes in code and behavior. Not to mention that CONFIG_EXTRA_ENV is really error prone with the requirement to put each line as a C string with trailing "\0".
We could provide other, more efficient configurations if we drop a few of the old requirements (which are not met by many recent boards anyway).
All recent boards, specially the one that switched to distro_, use the default environment. Thing is that there is no need to add an extra environment and the board can boot without any intervention by operator (or by factory) simply after flashing the bootloader.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
IMO this is just a matter of taste. Both ways should be possible.
Best regards, Stefano Babic
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de =====================================================================
On 21.11.18 11:14, Simon Goldschmidt wrote:
- What prevents the creation of such a dev package that only builds the needed lib?
I think Simon wants to prevent to rebuild u-boot.
No, I want to prevent recompiling the u-boot-tools package (fw_setenv) and Swupdate after changing U-Boot's default environment.
In fact, we are running a Debian-based system and I would like to run upstream Debian u-boot-tools. And of course running an upstream swupdate package would be preferred as well.
+1, same situation and requirements on our side. We work around this in [1], but we also have boards that we can feed from upstream Debian - except if we wanted SWupdate.
Jan
[1] https://github.com/ilbers/isar/tree/master/meta/recipes-bsp/u-boot
Hi Simon,
On 21/11/18 11:14, Simon Goldschmidt wrote:
On Wed, Nov 21, 2018 at 10:33 AM Stefano Babic sbabic@denx.de wrote:
Hi Wolfgang,
On 21/11/18 09:31, Wolfgang Denk wrote:
Dear Simon,
In message 90014ca9-64d8-8e38-a6c0-4c06e991c92c@gmail.com you wrote:
However, this approach creates several difficulties:
As usual, "difficulties" are made for solving.
I mean, none of these are technically challenging in any way.
- Distros like Debian cannot provide a devel package for SWUpdate like u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent.
Thta's fine.
If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
One possible way to fix this is not to use a compiled in default environment for the fw_env tools (or only use this as a very last resort), but instead provide a configuration option to read the default settings from a file. In this case the user could provide his own, board specific default environment.
I tried this, but it is very difficult to maintain this in sync, specially in projects where the number of developers is > 1 (all projects). If the file does not correspond to the linked environment, strange behaviors are expected.
Much better will be if u-boot generates in its build process this file. It becomes just another artifact like SPL or u-boot.img and we are sure that it is in sync.
- If we have a board with U-Boot already preinstalled and want to compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
- Why does building the U-Boot package not also create these libraries, either directly or as a separate package?
Issue is that there is not a u-boot-dev package
- What prevents the creation of such a dev package that only builds the needed lib?
I think Simon wants to prevent to rebuild u-boot.
No, I want to prevent recompiling the u-boot-tools package (fw_setenv) and Swupdate after changing U-Boot's default environment.
Right, I understand this.
In fact, we are running a Debian-based system and I would like to run upstream Debian u-boot-tools. And of course running an upstream swupdate package would be preferred as well.
I am searching / waiting for a mentor:
https://mentors.debian.net/package/swupdate
Package should be updated as well, but I'll push a new release for the end of the month.
One way to fix this would be to let fw_setenv somehow write an environment that doesn't need to know the default environment and is loaded in a "add" way, not replacing the whole environment as we normally do on load.
True, but this has side effect and limitation. You cannot change a variable in the deafult environment if you need it because you do not know it. If some changes are needed, even if for very small things like activating a gpio before ooting, you need to update the bootloader.
I have also thought at this but I dislike the drawbacks.
And security wise, I think linking in things as static libs in Swupdate is not a good decision as you can end up with 2 different versions of the same sources in your target.
This is a different topic - but until the library was used in SWUpdate, there was not another user, and if SWUpdate is the only user of it, static linking is fine.
Moving to shared library should be done in U-Boot project, then. Some changes are then required, at least how the environment is locked (it is not clean as it is done now - locking should be done by the library and not by the caller).
I can understand calling external binaries is not a good solution as well.
Right, I explain this in the other e-mail.
Using shared libs would be best. That would mean u-boot-tools fw_setenv would use a shared library to access the environment and swupdate could use it, too.
The approach is fine with me.
- U-Boot does not provide any default means to install a development library. Thus anything we modify on-top might break with the next version.
Why don't you add this, then? Patches are welcome!
+1
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread. I do think the current situation should be improved, making fw_setenv independent of the target that is built.
To be honest, I think this is not a clever approach. This whole concept of the compiled in default environment is antediluvian.
Fully agree - I am glad if there will be an improvement.
To be clear: the issue just raises if a board runs with default environment. User Space cannot know this environment because it is linked in the bootloader - the current way is to link the same environment in tools/env (that means, u-boot-fw-tools in OE).
There is no issue at all for boards that are not using default environment at all, that means they rely to have always a valid environment in flash.
It comes from a time where we were booting from parallel NOR flasah (or other boot ROM where you directly started executing code) and where we wanted to make sure that we can read the console baud rate very, very soon. Today, many boards don't have environment access in the SPL, and nobody complains that these run with a fixed baud rate only.
Did anybody ever ask who actually _needs_ this compiled in default environment?
In a secure boot environment, you cannot allow to load the environment from an untrusted source. We need a default environment in this case.
Or you make that changes are trusted.
But there are better ways than compiling it in via C headers, of course!
Exactly - another possible way is to use a configuration file, run a sort of "mkenvimage", bound the result to u-boot and use the configuration file as input for the libubootenv.[a|so] library.
Best regards, Stefano
Simon
Rather, quite all boards....see CONFIG_EXTRA_ENV in include/configs/* and the number of patches sent just to fix something in it.
UMHO the default environment should be like an extra image in u-boot, like an header added by mkimage, because there is no changes in code and behavior. Not to mention that CONFIG_EXTRA_ENV is really error prone with the requirement to put each line as a C string with trailing "\0".
We could provide other, more efficient configurations if we drop a few of the old requirements (which are not met by many recent boards anyway).
All recent boards, specially the one that switched to distro_, use the default environment. Thing is that there is no need to add an extra environment and the board can boot without any intervention by operator (or by factory) simply after flashing the bootloader.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
IMO this is just a matter of taste. Both ways should be possible.
Best regards, Stefano Babic
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de =====================================================================
Dear Stefano,
In message 9efc8990-118c-d5b9-802d-8190db2326d3@denx.de you wrote:
True, but this has side effect and limitation. You cannot change a variable in the deafult environment if you need it because you do not know it. If some changes are needed, even if for very small things like activating a gpio before ooting, you need to update the bootloader.
Just my words. (Mis) using the default environment for such purposes is broken by design.
Let's get rid of this!
Moving to shared library should be done in U-Boot project, then. Some changes are then required, at least how the environment is locked (it is not clean as it is done now - locking should be done by the library and not by the caller).
Speaking of security... shared libraries open a number of new attack vectors, too...
In a secure boot environment, you cannot allow to load the environment from an untrusted source. We need a default environment in this case.
Or you make that changes are trusted.
Right, when we sign (and check the signatures) of all other images, then why not do the very same for some environment image?
That would even be _better_ as currently there is no, absolutely no check if the builtin default environment is in any way consistent.
Best regards,
Wolfgang Denk
On Wed, Nov 21, 2018 at 12:45 PM Wolfgang Denk wd@denx.de wrote:
Dear Stefano,
In message 9efc8990-118c-d5b9-802d-8190db2326d3@denx.de you wrote:
True, but this has side effect and limitation. You cannot change a variable in the deafult environment if you need it because you do not know it. If some changes are needed, even if for very small things like activating a gpio before ooting, you need to update the bootloader.
Just my words. (Mis) using the default environment for such purposes is broken by design.
Let's get rid of this!
Moving to shared library should be done in U-Boot project, then. Some changes are then required, at least how the environment is locked (it is not clean as it is done now - locking should be done by the library and not by the caller).
Speaking of security... shared libraries open a number of new attack vectors, too...
In a secure boot environment, you cannot allow to load the environment from an untrusted source. We need a default environment in this case.
Or you make that changes are trusted.
Right, when we sign (and check the signatures) of all other images, then why not do the very same for some environment image?
You normally cannot sign the environment in the target when saving it when using private/public keys.
We are using a signed U-Boot image that apart from the default environment only needs to load MAC addresses. I cannot do this via a loaded environment (signed or unsigned) as the MAC addresses are stored in production and I cannot rely on production always having an up-to-date environment to embed their MAC addresses when programming. To use environment loading here, I would have to implement a whitelist that only loads the MAC addresses from the saved environment. That sounds a bit hacked, too.
So when it comes to secure boot, I do think there's a use case for not loading an environment. I don't currently mind how this environment is initialized. And maybe I don't yet get what you are talking about when trying to get rid of the default environment. I do need U-Boot to run with a predefined environment without loading it.
Regards, Simon
That would even be _better_ as currently there is no, absolutely no check if the builtin default environment is in any way consistent.
Dear Simon,
In message CAAh8qszhQARksHGQkUMk_QEyA8j6s7p_gnNCGQbZfAi_SK0W5A@mail.gmail.com you wrote:
Right, when we sign (and check the signatures) of all other images, then why not do the very same for some environment image?
You normally cannot sign the environment in the target when saving it when using private/public keys.
You mix things here. I was never talking about the currently used (and modified) envrionment "when saving it".
Discussion was about the default environment, and I suggested to replace this with a (signed) image used to initialize the normal environment. ther ehas never been a request or suggestion to create this signature on the target - do it in the very same way as you sign all other images for the system.
We are using a signed U-Boot image that apart from the default environment only needs to load MAC addresses. I cannot do this via a loaded environment (signed or unsigned) as the MAC addresses are stored in production and I cannot rely on production always having an up-to-date environment to embed their MAC addresses when programming.
Yes you can. If you like, you can have the U-Boot image and the environment image separate and signed separately, or you can create some form of metaimage (say, as FIT) with individual signatures, or you can simly concatelante both and use a common signature. There is a zillion of ways to do it.
To use environment loading here, I would have to implement a whitelist that only loads the MAC addresses from the saved environment. That sounds a bit hacked, too.
Why are you making things so complicated? You can always do just "env import ethaddr". No need to imeplement enything.
So when it comes to secure boot, I do think there's a use case for not loading an environment.
But you are loading it in any case. Whether from a binary object placed by the linker somewhwere in your data segment or from a [signed or at least checksummed] image somewhere else does not make any difference security-wise. It is only a minimal technical difference, i. e. using a different loading mechanism.
I don't currently mind how this environment is initialized. And maybe I don't yet get what you are talking about when trying to get rid of the default environment. I do need U-Boot to run with a predefined environment without loading it.
See my previous explantion about the 3 copies of the environment. All I suggest is to replace the binary blob by somethin gthat is not statically linked into the U-Boot image, so it can be shared for example with the fw_env tools.
Best regards,
Wolfgang Denk
Hi Wolfgang,
On 21/11/18 12:45, Wolfgang Denk wrote:
Dear Stefano,
In message 9efc8990-118c-d5b9-802d-8190db2326d3@denx.de you wrote:
True, but this has side effect and limitation. You cannot change a variable in the deafult environment if you need it because you do not know it. If some changes are needed, even if for very small things like activating a gpio before ooting, you need to update the bootloader.
Just my words. (Mis) using the default environment for such purposes is broken by design.
Let's get rid of this!
Moving to shared library should be done in U-Boot project, then. Some changes are then required, at least how the environment is locked (it is not clean as it is done now - locking should be done by the library and not by the caller).
Speaking of security... shared libraries open a number of new attack vectors, too...
This is the reason why these libraries are in meta-swupdate built as static libraries. Up now I had no requests that someone else wants to use them.
In a secure boot environment, you cannot allow to load the environment from an untrusted source. We need a default environment in this case.
Or you make that changes are trusted.
Right, when we sign (and check the signatures) of all other images, then why not do the very same for some environment image?
The weird thing is with "saveenv" - if we just read the env, it is fine, but if we want to change it, we need to sign, and this requires a private key on target.
That would even be _better_ as currently there is no, absolutely no check if the builtin default environment is in any way consistent.
This is not true. If the environment is linked to u-boot, it is signed together with u-boot and its consistency is automatically verified.
Best regards, Stefano
Dear Stefano,
In message 7089ef62-ed0f-87f4-e979-8c18a6ae4b62@denx.de you wrote:
Right, when we sign (and check the signatures) of all other images, then why not do the very same for some environment image?
The weird thing is with "saveenv" - if we just read the env, it is fine, but if we want to change it, we need to sign, and this requires a private key on target.
Agreed, but this is a totaly different issue.
The separate (potentially singed0 environment image is only the replacement for the current "default environment", which is not used for "env save". In the same way, there is no need to modfy the signed image.
But yes, it might be desirable to protect the working environment against malicious manipulation - but this should be discussed in a separate thread.
That would even be _better_ as currently there is no, absolutely no check if the builtin default environment is in any way consistent.
This is not true. If the environment is linked to u-boot, it is signed together with u-boot and its consistency is automatically verified.
Only if you use signed images. With plain U-Boot, there is not even a checksum for it...
Best regards,
Wolfgang Denk
Am Mi., 21. Nov. 2018, 15:27 hat Wolfgang Denk wd@denx.de geschrieben:
Dear Stefano,
In message 7089ef62-ed0f-87f4-e979-8c18a6ae4b62@denx.de you wrote:
Right, when we sign (and check the signatures) of all other images, then why not do the very same for some environment image?
The weird thing is with "saveenv" - if we just read the env, it is fine, but if we want to change it, we need to sign, and this requires a private key on target.
Agreed, but this is a totaly different issue.
The separate (potentially singed0 environment image is only the replacement for the current "default environment", which is not used for "env save". In the same way, there is no need to modfy the signed image.
But yes, it might be desirable to protect the working environment against malicious manipulation - but this should be discussed in a separate thread.
That would even be _better_ as currently there is no, absolutely no check if the builtin default environment is in any way consistent.
This is not true. If the environment is linked to u-boot, it is signed together with u-boot and its consistency is automatically verified.
Only if you use signed images. With plain U-Boot, there is not even a checksum for it...
When SPL loads U-Boot from a legacy image, isn't there a CRC involved over the full image including the environment?
Simon
On 21/11/18 15:37, Simon Goldschmidt wrote:
Am Mi., 21. Nov. 2018, 15:27 hat Wolfgang Denk <wd@denx.de mailto:wd@denx.de> geschrieben:
Dear Stefano, In message <7089ef62-ed0f-87f4-e979-8c18a6ae4b62@denx.de <mailto:7089ef62-ed0f-87f4-e979-8c18a6ae4b62@denx.de>> you wrote: > > > Right, when we sign (and check the signatures) of all other images, > > then why not do the very same for some environment image? > > The weird thing is with "saveenv" - if we just read the env, it is fine, > but if we want to change it, we need to sign, and this requires a > private key on target. Agreed, but this is a totaly different issue. The separate (potentially singed0 environment image is only the replacement for the current "default environment", which is not used for "env save". In the same way, there is no need to modfy the signed image. But yes, it might be desirable to protect the working environment against malicious manipulation - but this should be discussed in a separate thread. > > That would even be _better_ as currently there is no, absolutely no > > check if the builtin default environment is in any way consistent. > > This is not true. If the environment is linked to u-boot, it is signed > together with u-boot and its consistency is automatically verified. Only if you use signed images. With plain U-Boot, there is not even a checksum for it...When SPL loads U-Boot from a legacy image, isn't there a CRC involved over the full image including the environment?
I think Marek is talking about raw u-boot, not in case mkimage has put an header at the beginning. See CONFIG_SPL_RAW_IMAGE_SUPPORT and spl_parse_image_header(). The image is simply loaded without checks.
Best regards, Stefano
Dear Simon,
In message CAAh8qsyopV-HftdCWMbMu+fdcpqWWQtBnZDDyqFyv6iC1eoCDA@mail.gmail.com you wrote:
Only if you use signed images. With plain U-Boot, there is not even a checksum for it...
When SPL loads U-Boot from a legacy image, isn't there a CRC involved over the full image including the environment?
Yes, but when - for example - the compiled in default environment gets corrupted while loading from the storage device or while writing it to RAM, there will be zero check when reading it.
Best regards,
Wolfgang Denk
On 21.11.2018 16:41, Wolfgang Denk wrote:
Dear Simon,
In message CAAh8qsyopV-HftdCWMbMu+fdcpqWWQtBnZDDyqFyv6iC1eoCDA@mail.gmail.com you wrote:
Only if you use signed images. With plain U-Boot, there is not even a checksum for it...
When SPL loads U-Boot from a legacy image, isn't there a CRC involved over the full image including the environment?
Yes, but when - for example - the compiled in default environment gets corrupted while loading from the storage device or while writing it to RAM, there will be zero check when reading it.
Hmm, on my board, SPL loads U-Boot from NOR to SDRAM, then checks the uimage CRC. I'd say this is enough checks. That might not hold for all boards though.
But I still think keeping the default environment in a different place would be good.
Could we put U-Boot into a fit image and put the environment in a subimage? But it might increase SPL code size when a fit image has to be parsed...
Regards, Simon
Dear Simon,
In message e7751e81-610b-9cdd-e14d-04fc5b7a297f@gmail.com you wrote:
Hmm, on my board, SPL loads U-Boot from NOR to SDRAM, then checks the uimage CRC. I'd say this is enough checks. That might not hold for all boards though.
If you check the image in RAM, then yes. I've also seen implementations where it gets checked on the fly (for boot time reasons), so write corruption would not be covered.
Could we put U-Boot into a fit image and put the environment in a subimage? But it might increase SPL code size when a fit image has to be parsed...
Yes, of course this can be done. You would need such code anyway in SPL if you gor for Falcon mode, where you would load a (signed) FIT image with the Linux kernel and DTB (and maybe even a ramdisk image).
But as mentioned - this depends on the use case and on available resources. It may not be possible on many more resource-constricted boards - there, other mechanisms should be possible.
Best regards,
Wolfgang Denk
On 21.11.2018 16:48, Simon Goldschmidt wrote:
On 21.11.2018 16:41, Wolfgang Denk wrote:
Dear Simon,
In message CAAh8qsyopV-HftdCWMbMu+fdcpqWWQtBnZDDyqFyv6iC1eoCDA@mail.gmail.com you wrote:
Only if you use signed images. With plain U-Boot, there is not even a checksum for it...
When SPL loads U-Boot from a legacy image, isn't there a CRC involved over the full image including the environment?
Yes, but when - for example - the compiled in default environment gets corrupted while loading from the storage device or while writing it to RAM, there will be zero check when reading it.
Hmm, on my board, SPL loads U-Boot from NOR to SDRAM, then checks the uimage CRC. I'd say this is enough checks. That might not hold for all boards though.
Oops, I'll have to take that back. I just checked and SPL *does* boot the uImage U-Boot also if I deliberately corrupt the CRC of the image. So you're right, it seems to be not protected at all!
Is that expected behaviour or a bug?
Simon
Dear Simon,
In message 97f8b0ec-103b-e9b7-42ed-e1922f6d489a@gmail.com you wrote:
Oops, I'll have to take that back. I just checked and SPL *does* boot the uImage U-Boot also if I deliberately corrupt the CRC of the image. So you're right, it seems to be not protected at all!
Is that expected behaviour or a bug?
I don't know your board.
It may be intentional, for example if there are size restrictions for the SPL wich don't allow to include the CRC32 code...
Best regards,
Wolfgang Denk
Wolfgang,
Am Do., 22. Nov. 2018, 15:23 hat Wolfgang Denk wd@denx.de geschrieben:
Dear Simon,
In message 97f8b0ec-103b-e9b7-42ed-e1922f6d489a@gmail.com you wrote:
Oops, I'll have to take that back. I just checked and SPL *does* boot the uImage U-Boot also if I deliberately corrupt the CRC of the image. So you're right, it seems to be not protected at all!
Is that expected behaviour or a bug?
I don't know your board.
It may be intentional, for example if there are size restrictions for the SPL wich don't allow to include the CRC32 code...
I don't think this is intentional. I discussed this in another thread with Marek by now. There is no reference to the CRC code in the files in common/spl or the funtions called by it.
Searching the whole code, I cannot see anything that checks the uImage CRC from standard SPL. And I don't see board specific controls to do this check, did I miss something?
Regards, Simon
Dear Simon,
In message CAAh8qswJ-hf_FCvTdOggpS4MO6pThKLefjYxcquaBuhKfdQO2w@mail.gmail.com you wrote:
I don't know your board.
It may be intentional, for example if there are size restrictions for the SPL wich don't allow to include the CRC32 code...
I don't think this is intentional. I discussed this in another thread with Marek by now. There is no reference to the CRC code in the files in common/spl or the funtions called by it.
Depending on hardware capabilities, this may be indeed intentional, for example to keep the size of the SPL below a certain limit or to allow for other, more critical features instead.
Searching the whole code, I cannot see anything that checks the uImage CRC from standard SPL. And I don't see board specific controls to do this check, did I miss something?
This may well be possible. But then, this is usually stage two already - the ROM bootloader did not check any SPL checksum either...
Best regards,
Wolfgang Denk
Dear Wolfgang,
On 22.11.2018 17:00, Wolfgang Denk wrote:
Dear Simon,
In message CAAh8qswJ-hf_FCvTdOggpS4MO6pThKLefjYxcquaBuhKfdQO2w@mail.gmail.com you wrote:
I don't know your board.
It may be intentional, for example if there are size restrictions for the SPL wich don't allow to include the CRC32 code...
I don't think this is intentional. I discussed this in another thread with Marek by now. There is no reference to the CRC code in the files in common/spl or the funtions called by it.
Depending on hardware capabilities, this may be indeed intentional, for example to keep the size of the SPL below a certain limit or to allow for other, more critical features instead.
Searching the whole code, I cannot see anything that checks the uImage CRC from standard SPL. And I don't see board specific controls to do this check, did I miss something?
This may well be possible. But then, this is usually stage two already - the ROM bootloader did not check any SPL checksum either...
Well, the thing is: on my platform the boot rom *does* check SPL crc :-)
I wanted to post a patch for this, but from reading your response, this should probably be configurable. Especially as it doesn't only increase size but also boot time.
Thanks for your input on this!
Regards, Simon
Dear Stefano,
In message 6b07725e-51ba-7309-1506-e7211487b947@denx.de you wrote:
One possible way to fix this is not to use a compiled in default environment for the fw_env tools (or only use this as a very last resort), but instead provide a configuration option to read the default settings from a file. In this case the user could provide his own, board specific default environment.
I tried this, but it is very difficult to maintain this in sync, specially in projects where the number of developers is > 1 (all projects). If the file does not correspond to the linked environment, strange behaviors are expected.
Much better will be if u-boot generates in its build process this file. It becomes just another artifact like SPL or u-boot.img and we are sure that it is in sync.
I tend to disagree. As mentioned, the original intent of the default environment was only (!) to make sure that a working baudrate setting for the console is available on boards where the environment has never been initialized yet (fresh our of production) or, but only as additional benefit, that have lost their environment somehow.
Usually it is not good for any practical purposes - it is missing any board specific settings like MAC adress, IP addresses, serial numbers etc.
This was when booting from parallel NOR.
The whole paradigm was dropped when first implementations of SPL became necessary to boot from NAND - they all had harcoded baud rates, as they could not access the environment (often due to size limitations).
My question is: why do we need the default environment today?
It just consumes a couple of kB size, and does not bring any measurable benefits.
- Why does building the U-Boot package not also create these libraries, either directly or as a separate package?
Issue is that there is not a u-boot-dev package
Then why not create one?
- What prevents the creation of such a dev package that only builds the needed lib?
I think Simon wants to prevent to rebuild u-boot.
You don't have to rebuild U-Boot - building the lib should be sufficient.
To be clear: the issue just raises if a board runs with default environment. User Space cannot know this environment because it is linked in the bootloader - the current way is to link the same environment in tools/env (that means, u-boot-fw-tools in OE).
And I ask again: what exactly would we lose if we drop the whole default environment? I mean, what would we lose that we have not already given up elsewhere with nobody ever complaining about?
There is no issue at all for boards that are not using default environment at all, that means they rely to have always a valid environment in flash.
I would be seriously surprised if a single board in real life uses the default environment, and only that, for normale operation.
Did anybody ever ask who actually _needs_ this compiled in default environment?
Rather, quite all boards....see CONFIG_EXTRA_ENV in include/configs/* and the number of patches sent just to fix something in it.
You misunderstand. This was not primarily intended for the _default_ environment, but to _initialize_ the real environment. In these olden days this would usually be placed insome smaller sectors of the parallel NOR flash, and special (hand-tuned) linker scripts would make sure that the respective data would be properly placed in the U-Boot image.
At these times, a U-Boot image would usually contaim 3 copies of the same data: two redundant copies of the normal, persistent environment, embedded somehwere at "magic offsets" in the image, plus the default environment (somehwere in the [read-only] data segment, basically at an unknown address).
There would be a few boards that did not use embedded environments, so other ways wewre selected to initialize it.
But the default environment was never ever intended to be used for this purpose.
UMHO the default environment should be like an extra image in u-boot, like an header added by mkimage, because there is no changes in code and behavior. Not to mention that CONFIG_EXTRA_ENV is really error prone with the requirement to put each line as a C string with trailing "\0".
It was 1:1 coding at the time, done by a handful of experts, when the number of contributors was still single-digit.
I fully afree that it would be much nicer if we could just place it as plain text (ideally even commented!) on some other file.
But then: there is no technical reason not to implement this. Volunteers welcome!
All recent boards, specially the one that switched to distro_, use the default environment. Thing is that there is no need to add an extra environment and the board can boot without any intervention by operator (or by factory) simply after flashing the bootloader.
This is a broken design, and should be fixed. We don't need three copies of the environment. We should get rid of the "default" environment and replace it with a better way to initialize the real environment.
And yes, I really hate all these distro settings, because they are inelegant and unflexible and the slightest change requires to rebuild and reinstall the whole U-Boot image, when all you want to change is just a variable setting. This is CRAP!
IIUC, it should for example be possible to perform the initialization of the normale environment by running the equivalent of "env import" from some image (uImage or FIT image or even a plain text file [though I'd not recommend the latter as it includes zero consistency checking].
I mean: instead of complaining about the limitations of an antediluvian construct and investing precious energy in creating more and more ugly workarounds, we should think about ways to replace this with a more flexible and more powerful method.
Best regards,
Wolfgang Denk
Hi Wolfgang,
On 21/11/18 12:38, Wolfgang Denk wrote:
Dear Stefano,
In message 6b07725e-51ba-7309-1506-e7211487b947@denx.de you wrote:
One possible way to fix this is not to use a compiled in default environment for the fw_env tools (or only use this as a very last resort), but instead provide a configuration option to read the default settings from a file. In this case the user could provide his own, board specific default environment.
I tried this, but it is very difficult to maintain this in sync, specially in projects where the number of developers is > 1 (all projects). If the file does not correspond to the linked environment, strange behaviors are expected.
Much better will be if u-boot generates in its build process this file. It becomes just another artifact like SPL or u-boot.img and we are sure that it is in sync.
I tend to disagree. As mentioned, the original intent of the default environment was only (!) to make sure that a working baudrate setting for the console is available on boards where the environment has never been initialized yet (fresh our of production) or, but only as additional benefit, that have lost their environment somehow.
Current intent is to provide a "factory reset" environment and the device must be able to be functional even if with limiutation, at least it should be possible to be restored.
Usually it is not good for any practical purposes - it is missing any board specific settings like MAC adress, IP addresses, serial numbers etc.
This is also another aspect that it is difficult to be factorized. Many projects do not use the env for serial numbers or MAC, but they add an additional storage with a custom interface.
Storing MAC changed, at least for some SOC as i.MX6, because it is stored into the SOC (fuses).
This was when booting from parallel NOR.
The whole paradigm was dropped when first implementations of SPL became necessary to boot from NAND - they all had harcoded baud rates, as they could not access the environment (often due to size limitations).
My question is: why do we need the default environment today?
The "default" environemnt is the "factory reset" environment.
It just consumes a couple of kB size,
I guess more on some boards...
and does not bring any measurable benefits.
- Why does building the U-Boot package not also create these libraries, either directly or as a separate package?
Issue is that there is not a u-boot-dev package
Then why not create one?
Where ? Which project ? You men OE ?
I built and include a sort of u-boot-dev in meta-swupdate, and I have never heard of someone else who wants to use it up now.
- What prevents the creation of such a dev package that only builds the needed lib?
I think Simon wants to prevent to rebuild u-boot.
You don't have to rebuild U-Boot - building the lib should be sufficient.
If there is a configuration file that can be imported by the lib and it remains in sync with U-Boot, if U-Boot still sues linked environment.
To be clear: the issue just raises if a board runs with default environment. User Space cannot know this environment because it is linked in the bootloader - the current way is to link the same environment in tools/env (that means, u-boot-fw-tools in OE).
And I ask again: what exactly would we lose if we drop the whole default environment?
Board is not able to run after flashing the bootloader. This is become worse in the last time due to secure boot. In fact, if this "factory environment" is linked to U-Boot, it is signed together with U-Boot and can be trusted (this is the original topic in the thread).
If we use in U-Boot a sort of env import, we have to be sure that we can trust it.
I mean, what would we lose that we have not already given up elsewhere with nobody ever complaining about?
I see a lot of complains..
The feature we give with it is an additional redundancy because board comes up even if environment in flash is damaged.
There is no issue at all for boards that are not using default environment at all, that means they rely to have always a valid environment in flash.
I would be seriously surprised if a single board in real life uses the default environment, and only that, for normale operation.
I guess you are really surprised !
There are plenty of boards - "default environment" means "factory reset environment".
Did anybody ever ask who actually _needs_ this compiled in default environment?
Rather, quite all boards....see CONFIG_EXTRA_ENV in include/configs/* and the number of patches sent just to fix something in it.
You misunderstand.
I guess...
This was not primarily intended for the _default_ environment, but to _initialize_ the real environment.
This was maybe the initial goal, but if I look in current code there is just the symbol "default_environment" in the ELF. It contains the default as you mind + the "real" initialization environment from CONFIG_EXTRA_ENV.
In these olden days this would usually be placed insome smaller sectors of the parallel NOR flash, and special (hand-tuned) linker scripts would make sure that the respective data would be properly placed in the U-Boot image.
It could be, but I am talking about what I see in current code.
At these times, a U-Boot image would usually contaim 3 copies of the same data: two redundant copies of the normal, persistent environment, embedded somehwere at "magic offsets" in the image, plus the default environment (somehwere in the [read-only] data segment, basically at an unknown address).
I do not see the last one in data - there are two copies (with CONFIG_ENV_REDUND set, of course) in flash + one (called "default_environment") in .text segment. And yes, it is linked, and address changes.
There would be a few boards that did not use embedded environments, so other ways wewre selected to initialize it.
But the default environment was never ever intended to be used for this purpose.
Ok - but the fact is how it is used nowadays. Just see the number of patches sent to ML just to change something in CONFIG_EXTRA_ENV.
UMHO the default environment should be like an extra image in u-boot, like an header added by mkimage, because there is no changes in code and behavior. Not to mention that CONFIG_EXTRA_ENV is really error prone with the requirement to put each line as a C string with trailing "\0".
It was 1:1 coding at the time, done by a handful of experts, when the number of contributors was still single-digit.
Right
I fully afree that it would be much nicer if we could just place it as plain text (ideally even commented!) on some other file.
+1
But then: there is no technical reason not to implement this. Volunteers welcome!
+1
but we need some concept how to do: do we link the generated environment as today ? Or do we put somewhere else, for example at the end of U-Boot (and SPL, too, as some boards can read environment in SPL).
All recent boards, specially the one that switched to distro_, use the default environment. Thing is that there is no need to add an extra environment and the board can boot without any intervention by operator (or by factory) simply after flashing the bootloader.
This is a broken design, and should be fixed. We don't need three copies of the environment. We should get rid of the "default" environment and replace it with a better way to initialize the real environment.
We have still3 copies of the environment. Fine if we find a common way and not board specific to load the initial environment.
And yes, I really hate all these distro settings, because they are inelegant and unflexible and the slightest change requires to rebuild and reinstall the whole U-Boot image, when all you want to change is just a variable setting. This is CRAP!
There are different goals. Distros want to have exactly the same interface for all boards and same set of scripts to boot. If ideally there is just one u-boot at least for the same set of architecture, distros could have just one package.
On the other side, products / devices prefer to have a specific environment to speed up the boot process (distro setting performs scanning) or to provide custom feature.
IIUC, it should for example be possible to perform the initialization of the normale environment by running the equivalent of "env import" from some image (uImage or FIT image or even a plain text file
A signed fit image with the environment should be possible. Anyway, the same (not signed) we have running mkenvimage on an ASCII file and then store somewhere.
[though I'd not recommend the latter as it includes zero consistency checking].
I mean: instead of complaining about the limitations of an antediluvian construct and investing precious energy in creating more and more ugly workarounds, we should think about ways to replace this with a more flexible and more powerful method.
Agree on this, we should find an accepted and community approved method..
Best regards, Stefano
Dear Stefano,
In message 151c30b1-d383-d212-a611-519150a80e98@denx.de you wrote:
I tend to disagree. As mentioned, the original intent of the default environment was only (!) to make sure that a working baudrate setting for the console is available on boards where the environment has never been initialized yet (fresh our of production) or, but only as additional benefit, that have lost their environment somehow.
Current intent is to provide a "factory reset" environment and the device must be able to be functional even if with limiutation, at least it should be possible to be restored.
I understand the intend. But instead of chosing an approoriate method (creating a separate enviornment block to be loaded with "env import" which has been created for exactly such purposes as user profiles or factory reset) they decided to to the Wong Thing.
This should be fixed.
My question is: why do we need the default environment today?
The "default" environemnt is the "factory reset" environment.
You avoid a straight and honest answer here.
It is _used_ for this purpoose, or rather _misused_.
My question is: whyu do we _need_ the default environment today?
I think we should get rid of it, and yes, obviously this means that we should put the current (mis-) users from their heads to their feet.
- Why does building the U-Boot package not also create these libraries, either directly or as a separate package?
Issue is that there is not a u-boot-dev package
Then why not create one?
Where ? Which project ? You men OE ?
You said there was no such image. I don't know which project you were referring to...
You don't have to rebuild U-Boot - building the lib should be sufficient.
If there is a configuration file that can be imported by the lib and it remains in sync with U-Boot, if U-Boot still sues linked environment.
Sorry, I don't understand what you want to tell me here.
And I ask again: what exactly would we lose if we drop the whole default environment?
Board is not able to run after flashing the bootloader. This is become worse in the last time due to secure boot. In fact, if this "factory environment" is linked to U-Boot, it is signed together with U-Boot and can be trusted (this is the original topic in the thread).
This is what people made out of it. But it's not the only possible way, and definitely not the best way.
You ignore my arguments.
Instead of using a binary blob inserted somewhere by the linker, we could use a more suitable data structure, probably even separate and with it's own signature so it could be used (and verified) by external tools as well.
And get rid of this current implementation.
If we use in U-Boot a sort of env import, we have to be sure that we can trust it.
Any other form of implementation is probably better and more secure. At themoment the binary blob just gets loaded, without any checking, not even for corruption.
I see a lot of complains..
I don't.
The feature we give with it is an additional redundancy because board comes up even if environment in flash is damaged.
If you split off the initialization, you could even have more than one copy of the initial / factory settings if you like.
I would be seriously surprised if a single board in real life uses the default environment, and only that, for normale operation.
I guess you are really surprised !
There are plenty of boards - "default environment" means "factory reset environment".
s/means/has been misused/
This is not a feature, it is part of the problem that should be solved!
This was maybe the initial goal, but if I look in current code there is just the symbol "default_environment" in the ELF. It contains the default as you mind + the "real" initialization environment from CONFIG_EXTRA_ENV.
This has always been like that. Yes of course it is a convenient way to create4 customized initial settings. But the method how it is implemented does not scale, and we are running in limitations.
So let's replace it with soemthing more powerful and more flexible.
At these times, a U-Boot image would usually contaim 3 copies of the same data: two redundant copies of the normal, persistent environment, embedded somehwere at "magic offsets" in the image, plus the default environment (somehwere in the [read-only] data segment, basically at an unknown address).
I do not see the last one in data - there are two copies (with CONFIG_ENV_REDUND set, of course) in flash + one (called "default_environment") in .text segment. And yes, it is linked, and address changes.
If you were using an "embedded environment" it was 3 copies. But for initialization, only one would be needed.
but we need some concept how to do: do we link the generated environment as today ? Or do we put somewhere else, for example at the end of U-Boot (and SPL, too, as some boards can read environment in SPL).
If we don't use a binary blob any more linking few advantages; the only one I can think of that it is easy to get the start address in the code - it's just a variable initialized by the linker. the disadvantage is that this address is more or less random unless you play tricks in the linker script, and it;s difficult to access from outside.
To me it seems more desirable to have a separate image, so we can still decide what to do with it: attach (or otherwise combine it) with the U-Boot or SPL binary - maybe using the linker or "cat" or "dd" or FIT images or whatever seems useful in the specific context. I don't think we have to decide for one specific implementation - we should allow for flexbility here.
Then we can also use other (external) tools like fw_env to access this area easily.
We have still3 copies of the environment. Fine if we find a common way and not board specific to load the initial environment.
3, or even more. This is good, as nobody will suffere from an increase of the memory footprint.
The difference is that an ancient crutch gets replaced by a more powerful and flexible mnechanism - one where you can potentially have not only one "default" (or factory) environment, but more (think of "user profiles"), which can be changed (even securely, i. e. with signature checking) without having to recompile and re-install U-Boot.
And yes, I really hate all these distro settings, because they are inelegant and unflexible and the slightest change requires to rebuild and reinstall the whole U-Boot image, when all you want to change is just a variable setting. This is CRAP!
There are different goals. Distros want to have exactly the same interface for all boards and same set of scripts to boot. If ideally there is just one u-boot at least for the same set of architecture, distros could have just one package.
I don't think these are so different goals. These are the result of the current limitations and that people tend to think small - only for their own poroject and within the current limits.
As is, each and every distribution will have their own U-Boot binary. And switching the distro will mean you have to update U-Boot.
If we splitt off the defualt environment, distros could use the same binary (at least in theory) - but in any case switching distros should just mean you have to add the new environment settings - and when I say "add" here I mean exactly that. Ideally there would be no need to replace / remove an already existing (known to be working) environment from another distro. We could keep it to allow switching between distros without reinstall. We could use it as a fall-back if anything goes wrong. And so on...
On the other side, products / devices prefer to have a specific environment to speed up the boot process (distro setting performs scanning) or to provide custom feature.
Right, but my proposal just adds more lexibility, it does not take away anything you have right now.
A signed fit image with the environment should be possible. Anyway, the same (not signed) we have running mkenvimage on an ASCII file and then store somewhere.
Right. Or, if you are satisfied with the status quo for the default environment (no checking at all) you could just use a plain text file.
Best regards,
Wolfgang Denk
Am Mi., 21. Nov. 2018, 15:23 hat Wolfgang Denk wd@denx.de geschrieben:
Dear Stefano,
In message 151c30b1-d383-d212-a611-519150a80e98@denx.de you wrote:
I tend to disagree. As mentioned, the original intent of the default environment was only (!) to make sure that a working baudrate setting for the console is available on boards where the environment has never been initialized yet (fresh our of production) or, but only as additional benefit, that have lost their environment somehow.
Current intent is to provide a "factory reset" environment and the device must be able to be functional even if with limiutation, at least it should be possible to be restored.
I understand the intend. But instead of chosing an approoriate method (creating a separate enviornment block to be loaded with "env import" which has been created for exactly such purposes as user profiles or factory reset) they decided to to the Wong Thing.
This should be fixed.
My question is: why do we need the default environment today?
The "default" environemnt is the "factory reset" environment.
You avoid a straight and honest answer here.
It is _used_ for this purpoose, or rather _misused_.
My question is: whyu do we _need_ the default environment today?
I think we should get rid of it, and yes, obviously this means that we should put the current (mis-) users from their heads to their feet.
- Why does building the U-Boot package not also create these libraries, either directly or as a separate package?
Issue is that there is not a u-boot-dev package
Then why not create one?
Where ? Which project ? You men OE ?
You said there was no such image. I don't know which project you were referring to...
You don't have to rebuild U-Boot - building the lib should be sufficient.
If there is a configuration file that can be imported by the lib and it remains in sync with U-Boot, if U-Boot still sues linked environment.
Sorry, I don't understand what you want to tell me here.
And I ask again: what exactly would we lose if we drop the whole default environment?
Board is not able to run after flashing the bootloader. This is become worse in the last time due to secure boot. In fact, if this "factory environment" is linked to U-Boot, it is signed together with U-Boot and can be trusted (this is the original topic in the thread).
This is what people made out of it. But it's not the only possible way, and definitely not the best way.
You ignore my arguments.
Instead of using a binary blob inserted somewhere by the linker, we could use a more suitable data structure, probably even separate and with it's own signature so it could be used (and verified) by external tools as well.
And get rid of this current implementation.
If we use in U-Boot a sort of env import, we have to be sure that we can trust it.
Any other form of implementation is probably better and more secure. At themoment the binary blob just gets loaded, without any checking, not even for corruption.
I see a lot of complains..
I don't.
The feature we give with it is an additional redundancy because board comes up even if environment in flash is damaged.
If you split off the initialization, you could even have more than one copy of the initial / factory settings if you like.
I would be seriously surprised if a single board in real life uses the default environment, and only that, for normale operation.
I guess you are really surprised !
There are plenty of boards - "default environment" means "factory reset environment".
s/means/has been misused/
This is not a feature, it is part of the problem that should be solved!
This was maybe the initial goal, but if I look in current code there is just the symbol "default_environment" in the ELF. It contains the default as you mind + the "real" initialization environment from CONFIG_EXTRA_ENV.
This has always been like that. Yes of course it is a convenient way to create4 customized initial settings. But the method how it is implemented does not scale, and we are running in limitations.
So let's replace it with soemthing more powerful and more flexible.
At these times, a U-Boot image would usually contaim 3 copies of the same data: two redundant copies of the normal, persistent environment, embedded somehwere at "magic offsets" in the image, plus the default environment (somehwere in the [read-only] data segment, basically at an unknown address).
I do not see the last one in data - there are two copies (with CONFIG_ENV_REDUND set, of course) in flash + one (called "default_environment") in .text segment. And yes, it is linked, and address changes.
If you were using an "embedded environment" it was 3 copies. But for initialization, only one would be needed.
but we need some concept how to do: do we link the generated environment as today ? Or do we put somewhere else, for example at the end of U-Boot (and SPL, too, as some boards can read environment in SPL).
If we don't use a binary blob any more linking few advantages; the only one I can think of that it is easy to get the start address in the code - it's just a variable initialized by the linker. the disadvantage is that this address is more or less random unless you play tricks in the linker script, and it;s difficult to access from outside.
To me it seems more desirable to have a separate image, so we can still decide what to do with it: attach (or otherwise combine it) with the U-Boot or SPL binary - maybe using the linker or "cat" or "dd" or FIT images or whatever seems useful in the specific context. I don't think we have to decide for one specific implementation - we should allow for flexbility here.
Then we can also use other (external) tools like fw_env to access this area easily.
That would make sense, yes. Seems like I really did mix things up. I think going this way would solve things for the distro packages as well as keep usage scenarios like mine running.
Regards, Simon
We have still3 copies of the environment. Fine if we find a common way and not board specific to load the initial environment.
3, or even more. This is good, as nobody will suffere from an increase of the memory footprint.
The difference is that an ancient crutch gets replaced by a more powerful and flexible mnechanism - one where you can potentially have not only one "default" (or factory) environment, but more (think of "user profiles"), which can be changed (even securely, i. e. with signature checking) without having to recompile and re-install U-Boot.
And yes, I really hate all these distro settings, because they are inelegant and unflexible and the slightest change requires to rebuild and reinstall the whole U-Boot image, when all you want to change is just a variable setting. This is CRAP!
There are different goals. Distros want to have exactly the same interface for all boards and same set of scripts to boot. If ideally there is just one u-boot at least for the same set of architecture, distros could have just one package.
I don't think these are so different goals. These are the result of the current limitations and that people tend to think small - only for their own poroject and within the current limits.
As is, each and every distribution will have their own U-Boot binary. And switching the distro will mean you have to update U-Boot.
If we splitt off the defualt environment, distros could use the same binary (at least in theory) - but in any case switching distros should just mean you have to add the new environment settings - and when I say "add" here I mean exactly that. Ideally there would be no need to replace / remove an already existing (known to be working) environment from another distro. We could keep it to allow switching between distros without reinstall. We could use it as a fall-back if anything goes wrong. And so on...
On the other side, products / devices prefer to have a specific environment to speed up the boot process (distro setting performs scanning) or to provide custom feature.
Right, but my proposal just adds more lexibility, it does not take away anything you have right now.
A signed fit image with the environment should be possible. Anyway, the same (not signed) we have running mkenvimage on an ASCII file and then store somewhere.
Right. Or, if you are satisfied with the status quo for the default environment (no checking at all) you could just use a plain text file.
Best regards,
Wolfgang Denk
-- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de "I like your game but we have to change the rules."
Hi Simon,
On 20/11/18 22:11, Simon Goldschmidt wrote:
On 04.09.2018 12:30, Andreas Reichel wrote:
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
- Distros like Debian cannot provide a devel package for SWUpdate like
u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
- If we have a board with U-Boot already preinstalled and want to
compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
- U-Boot does not provide any default means to install a development
library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread.
I did some tests, but I have found the way quite fragile. We should need at least markers to identify begin and end of the environment because size differs for each board.
I do think the current situation should be improved, making fw_setenv independent of the target that is built.
A simple way is to let fw_setenv to get the default env via a parameter (file), but this is also error prone if u-boot is updated and the file with default env not.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
I do not prefer this - SWUpdate should be as much as possible self contained (also for security reasons). Linking libubootenv.a is like linking any other library. The weirdness is just with the default environment, that should be addressed.
Best regards, Stefano Babic
On 21.11.18 10:19, Stefano Babic wrote:
Hi Simon,
On 20/11/18 22:11, Simon Goldschmidt wrote:
On 04.09.2018 12:30, Andreas Reichel wrote:
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
- Distros like Debian cannot provide a devel package for SWUpdate like
u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
- If we have a board with U-Boot already preinstalled and want to
compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
- U-Boot does not provide any default means to install a development
library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread.
I did some tests, but I have found the way quite fragile. We should need at least markers to identify begin and end of the environment because size differs for each board.
I do think the current situation should be improved, making fw_setenv independent of the target that is built.
A simple way is to let fw_setenv to get the default env via a parameter (file), but this is also error prone if u-boot is updated and the file with default env not.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
I do not prefer this - SWUpdate should be as much as possible self contained (also for security reasons). Linking libubootenv.a is like linking any other library. The weirdness is just with the default environment, that should be addressed.
I'm not sure if this is still on the radar as requirement, but it would be great if we could kick all the board dependencies out of that library, make them config files. That would make the lib compatible with common binary distros like Debian.
Jan
On Wed, Nov 21, 2018 at 10:19 AM Stefano Babic sbabic@denx.de wrote:
Hi Simon,
On 20/11/18 22:11, Simon Goldschmidt wrote:
On 04.09.2018 12:30, Andreas Reichel wrote:
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
Distros like Debian cannot provide a devel package for SWUpdate like u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
If we have a board with U-Boot already preinstalled and want to compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we
had to compile U-Boot again, although it is already installed since we lack a dev package.
- U-Boot does not provide any default means to install a development library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread.
I did some tests, but I have found the way quite fragile. We should need at least markers to identify begin and end of the environment because size differs for each board.
I do think the current situation should be improved, making fw_setenv independent of the target that is built.
A simple way is to let fw_setenv to get the default env via a parameter (file), but this is also error prone if u-boot is updated and the file with default env not.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
I do not prefer this - SWUpdate should be as much as possible self contained (also for security reasons). Linking libubootenv.a is like linking any other library. The weirdness is just with the default environment, that should be addressed.
For exactly these security reasons, I think it would be better to have the code only once in your target. And this is not only true for the u-boot env lib but also for mtd support and maybe for the webserver.
If you have the same versions twice, you need to track and fix both versions if there should be issues.
I know these things are not available as libs right now, even if that would be preferable. Therefore I think calling external binaries could be better. And I don't yet understand why calling external binaries would be a security risk, maybe you can explain this to me?
Regards, Simon
On 21/11/18 11:21, Simon Goldschmidt wrote:
On Wed, Nov 21, 2018 at 10:19 AM Stefano Babic sbabic@denx.de wrote:
Hi Simon,
On 20/11/18 22:11, Simon Goldschmidt wrote:
On 04.09.2018 12:30, Andreas Reichel wrote:
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
Distros like Debian cannot provide a devel package for SWUpdate like u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
If we have a board with U-Boot already preinstalled and want to compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we
had to compile U-Boot again, although it is already installed since we lack a dev package.
- U-Boot does not provide any default means to install a development library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread.
I did some tests, but I have found the way quite fragile. We should need at least markers to identify begin and end of the environment because size differs for each board.
I do think the current situation should be improved, making fw_setenv independent of the target that is built.
A simple way is to let fw_setenv to get the default env via a parameter (file), but this is also error prone if u-boot is updated and the file with default env not.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
I do not prefer this - SWUpdate should be as much as possible self contained (also for security reasons). Linking libubootenv.a is like linking any other library. The weirdness is just with the default environment, that should be addressed.
For exactly these security reasons, I think it would be better to have the code only once in your target. And this is not only true for the u-boot env lib but also for mtd support and maybe for the webserver.
Webserver ?
I guess you are then about libmtd.a and libubi.a. There is a ticket to have a mtd-utils-dev package: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911821
If you have the same versions twice, you need to track and fix both versions if there should be issues.
Currently, both mtd-utils and u-boot do not export a library. The only user in OE is SWUpdate that has own rules to build it.
Of course, this could be changed (ad it is desired).
I know these things are not available as libs right now, even if that would be preferable. Therefore I think calling external binaries could be better.
No.
And I don't yet understand why calling external binaries would be a security risk, maybe you can explain this to me?
If you check in SWUpdate, you see that code does not call system() at all - the only exception is for shell scripts.
The first attack I can imagine (sure that we can find several different if we think enough about it) is a leak in your application that let change /bin/sh or /usr/bin/fw_setenv on the filesystem. Even if the leak is outside SWUpdate, an attacker can use it to avoid an update, run malicious code or install something else.
Not only, if SWUpdate remains self-contained, you can jail it in a chroot environment - less dependencies SWUpdate has, much better.
Best regards, Stefano Babic
On Wed, Nov 21, 2018 at 12:05 PM Stefano Babic sbabic@denx.de wrote:
On 21/11/18 11:21, Simon Goldschmidt wrote:
On Wed, Nov 21, 2018 at 10:19 AM Stefano Babic sbabic@denx.de wrote:
Hi Simon,
On 20/11/18 22:11, Simon Goldschmidt wrote:
On 04.09.2018 12:30, Andreas Reichel wrote:
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
Distros like Debian cannot provide a devel package for SWUpdate like u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
If we have a board with U-Boot already preinstalled and want to compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we
had to compile U-Boot again, although it is already installed since we lack a dev package.
- U-Boot does not provide any default means to install a development library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread.
I did some tests, but I have found the way quite fragile. We should need at least markers to identify begin and end of the environment because size differs for each board.
I do think the current situation should be improved, making fw_setenv independent of the target that is built.
A simple way is to let fw_setenv to get the default env via a parameter (file), but this is also error prone if u-boot is updated and the file with default env not.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
I do not prefer this - SWUpdate should be as much as possible self contained (also for security reasons). Linking libubootenv.a is like linking any other library. The weirdness is just with the default environment, that should be addressed.
For exactly these security reasons, I think it would be better to have the code only once in your target. And this is not only true for the u-boot env lib but also for mtd support and maybe for the webserver.
Webserver ?
Mongoose. Right now, I'm talking about the swupdate 2017.11 branch. Maybe there are changes regarding webserver since then. I will update to a newer branch soon (before relase, that is :-)
I guess you are then about libmtd.a and libubi.a. There is a ticket to have a mtd-utils-dev package:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911821If you have the same versions twice, you need to track and fix both versions if there should be issues.
Currently, both mtd-utils and u-boot do not export a library. The only user in OE is SWUpdate that has own rules to build it.
Of course, this could be changed (ad it is desired).
Yes, I'm talking about libmtd.a. It's good to know there's a plan to change that.
Thanks for your answers!
Regards, Simon
I know these things are not available as libs right now, even if that would be preferable. Therefore I think calling external binaries could be better.
No.
And I don't yet understand why calling external binaries would be a security risk, maybe you can explain this to me?
If you check in SWUpdate, you see that code does not call system() at all - the only exception is for shell scripts.
The first attack I can imagine (sure that we can find several different if we think enough about it) is a leak in your application that let change /bin/sh or /usr/bin/fw_setenv on the filesystem. Even if the leak is outside SWUpdate, an attacker can use it to avoid an update, run malicious code or install something else.
Not only, if SWUpdate remains self-contained, you can jail it in a chroot environment - less dependencies SWUpdate has, much better.
Best regards, Stefano Babic
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de =====================================================================
Hi Simon,
On 21/11/18 12:13, Simon Goldschmidt wrote:
On Wed, Nov 21, 2018 at 12:05 PM Stefano Babic sbabic@denx.de wrote:
On 21/11/18 11:21, Simon Goldschmidt wrote:
On Wed, Nov 21, 2018 at 10:19 AM Stefano Babic sbabic@denx.de wrote:
Hi Simon,
On 20/11/18 22:11, Simon Goldschmidt wrote:
On 04.09.2018 12:30, Andreas Reichel wrote:
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
Distros like Debian cannot provide a devel package for SWUpdate like u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
If we have a board with U-Boot already preinstalled and want to compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we
had to compile U-Boot again, although it is already installed since we lack a dev package.
- U-Boot does not provide any default means to install a development library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread.
I did some tests, but I have found the way quite fragile. We should need at least markers to identify begin and end of the environment because size differs for each board.
I do think the current situation should be improved, making fw_setenv independent of the target that is built.
A simple way is to let fw_setenv to get the default env via a parameter (file), but this is also error prone if u-boot is updated and the file with default env not.
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
I do not prefer this - SWUpdate should be as much as possible self contained (also for security reasons). Linking libubootenv.a is like linking any other library. The weirdness is just with the default environment, that should be addressed.
For exactly these security reasons, I think it would be better to have the code only once in your target. And this is not only true for the u-boot env lib but also for mtd support and maybe for the webserver.
Webserver ?
Mongoose.
SWupdate has a modified version of the Webserver (yes, code comes from Mongoose project) to support streaming. It is a different situation as libmtd or libubootenv
Right now, I'm talking about the swupdate 2017.11 branch. Maybe there are changes regarding webserver since then. I will update to a newer branch soon (before relase, that is :-)
There are big changes. Code was synchronized wit 6.11 version of the Webserver and old static website was replaced by a responsive Web application.
I guess you are then about libmtd.a and libubi.a. There is a ticket to have a mtd-utils-dev package:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911821If you have the same versions twice, you need to track and fix both versions if there should be issues.
Currently, both mtd-utils and u-boot do not export a library. The only user in OE is SWUpdate that has own rules to build it.
Of course, this could be changed (ad it is desired).
Yes, I'm talking about libmtd.a. It's good to know there's a plan to change that.
Plan depends on other projects, anyway...
Thanks for your answers!
Best regards, Stefano
Regards, Simon
I know these things are not available as libs right now, even if that would be preferable. Therefore I think calling external binaries could be better.
No.
And I don't yet understand why calling external binaries would be a security risk, maybe you can explain this to me?
If you check in SWUpdate, you see that code does not call system() at all - the only exception is for shell scripts.
The first attack I can imagine (sure that we can find several different if we think enough about it) is a leak in your application that let change /bin/sh or /usr/bin/fw_setenv on the filesystem. Even if the leak is outside SWUpdate, an attacker can use it to avoid an update, run malicious code or install something else.
Not only, if SWUpdate remains self-contained, you can jail it in a chroot environment - less dependencies SWUpdate has, much better.
Best regards, Stefano Babic
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de =====================================================================
Hi Simon,
reviving an old (dead) thread:
On 20/11/18 22:11, Simon Goldschmidt wrote:
On 04.09.2018 12:30, Andreas Reichel wrote:
Hi all,
as Stefano Babic was so friendly and pointed out a few things already, we come the following problematic points:
For SWupdate to access U-Boot's environment, it uses code from U-Boot. Before 2015, fw_env.c was copied - hence out of version control, afterwards and since then, a lib.a produced by U-Boot has been used and renamed to libubootenv.a to link against.
However, this approach creates several difficulties:
- Distros like Debian cannot provide a devel package for SWUpdate like
u-boot-dev, since U-Boot has its default environment code compiled board-dependently and Debian needed it board-independent. If a board-independent libubootenv.a was provided without a specific default environment, the first update process would destroy the U-Boot environment if it had had bad CRC before. (Thanks Stefano for this good point).
- If we have a board with U-Boot already preinstalled and want to
compile SWUpdate for it, we need the sources of this very U-Boot to get the propper lib. For example in a debian-like build system we had to compile U-Boot again, although it is already installed since we lack a dev package.
- U-Boot does not provide any default means to install a development
library. Thus anything we modify on-top might break with the next version.
First proposal by Stefano and me would be to somehow split the default environment from the library to have a board-independent component and board specific data that is passed to U-Boot and SWUpdate somehow.
Goal is to factor out U-Boot environment support for other software like SWUpdate and not patching and hacking like its the case with recipes as in openembedded atm.
Has there been any progress in the SWupdate/fw_setenv integration? I thought I remember reading something about letting fw_setenv extract the environment from the U-Boot binary in the target, but I cannot find the thread. I do think the current situation should be improved, making fw_setenv independent of the target that is built.
This is done - I took some suggestions (thanks Marek, you, Wolfgang, etc.) and I create an independent (shared) library under LGPL-2.1. Details at https://groups.google.com/forum/#!topic/swupdate/2wcIyfexOro
And as this thread is on the swupdate group as well: I would prefer calling an external fw_setenv tool instead of linking in the static library libubootenv.a...
This is also done - the "new" fw_setenv takes as argument the initial environment as file and it is then independent from the target.
Best regards, Stefano
participants (7)
-
Andreas Reichel -
Jan Kiszka -
Marek Vasut -
Otavio Salvador -
Simon Goldschmidt -
Stefano Babic -
Wolfgang Denk