[U-Boot-Users] embedding environment by hand
first a little background ... the Blackfin processor has a custom object format for booting code (called a "loader file" or "LDR" for short). this custom object format is what actually gets burned into say parallel or spi or nand flash. i have to hope that the Blackfin processor isnt alone in having a custom object format ;), so hopefully the changes i propose here will be useful to others.
the exact object format is irrelevant here, but the point is that while the environment is actually embedded inside of the LDR image, it cannot be embedded with the standard method of creating the data array "environment" in environment.c and making sure the environment.o object gets linked at an exact offset. this is because we take the u-boot.bin and run it through another utility to get u-boot.ldr. the offsets in u-boot.bin do not correlate in any way to the offsets in u-boot.ldr nor can they as the LDR format may different drastically by processor.
the utility that generates the u-boot.ldr has the ability to reserve space inside of the final image. so in the board configuration file, we say the environment lives at an offset of 0x4000 (to correspond to one of the flash sectors) of size say 0x2000. the utility is then told to generate space at an offset in u-boot.ldr at offset 0x4000 of size 0x2000. now the u-boot.ldr can be burned into flash without a problem and saving of the environment is OK.
however, the initial u-boot.ldr is generated without the default environment programmed into it. so what i'd propose is to introduce a new define "ENV_IS_EMBEDDED_CUSTOM" and add a new flag to the envcrc helper. this would allow the environment.o linked into u-boot.bin to be empty and we can use envcrc helper to extract the environment blob so that the blob can be passed to extrenal utilities for embedding. the patch attached accomplishes both of these things. -mike
In message 200710101825.30341.vapier@gentoo.org you wrote:
first a little background ... the Blackfin processor has a custom object format for booting code (called a "loader file" or "LDR" for short). this custom object format is what actually gets burned into say parallel or spi or nand flash. i have to hope that the Blackfin processor isnt alone in having a custom object format ;), so hopefully the changes i propose here will be> useful to others.
Never heard about such a thing. All the rest of the world is using ELF as long as images need a structure that must be processable by tools, and raw binary blobs for anything else.
Don't you use any GNU toolchain there? Isn't there any support for the GNU linker? Can't you use standard tools?
the exact object format is irrelevant here, but the point is that while the environment is actually embedded inside of the LDR image, it cannot be embedded with the standard method of creating the data array "environment" in environment.c and making sure the environment.o object gets linked at an exact offset. this is because we take the u-boot.bin and run it through another utility to get u-boot.ldr. the offsets in u-boot.bin do not correlate in any way to the offsets in u-boot.ldr nor can they as the LDR format may different drastically by processor.
Ummm... this sounds like linking to me, and if your linker tool performs such shifting and reordering, it also has to adjust the code that references such shifted or reordered memory. If it doesn't, I don't understand what it does at all except from corrupting perfectly good images ;-)
the utility that generates the u-boot.ldr has the ability to reserve space inside of the final image. so in the board configuration file, we say the environment lives at an offset of 0x4000 (to correspond to one of the flash sectors) of size say 0x2000. the utility is then told to generate space at an offset in u-boot.ldr at offset 0x4000 of size 0x2000. now the u-boot.ldr can be burned into flash without a problem and saving of the environment is OK.
That's what we normally do in the linker scripts. Can't you use a normal linker, then?
however, the initial u-boot.ldr is generated without the default environment programmed into it. so what i'd propose is to introduce a new
Why is this the case?
define "ENV_IS_EMBEDDED_CUSTOM" and add a new flag to the envcrc helper. this would allow the environment.o linked into u-boot.bin to be empty and we can use envcrc helper to extract the environment blob so that the blob can be passed to extrenal utilities for embedding. the patch attached accomplishes both of these things.
I cannot see why that would be necessary. If you cannot initialize the embedded environment, just handle it like anybody else who not uses an initialized embedded (i. e. for exampole a separated) environment. Why would you need extra code?
--- common/environment.c (revision 950) +++ common/environment.c (working copy)
This makes no sense to me.
Best regards,
Wolfgang Denk
On Wednesday 10 October 2007, Wolfgang Denk wrote:
In message 200710101825.30341.vapier@gentoo.org you wrote:
first a little background ... the Blackfin processor has a custom object format for booting code (called a "loader file" or "LDR" for short). this custom object format is what actually gets burned into say parallel or spi or nand flash. i have to hope that the Blackfin processor isnt alone in having a custom object format ;), so hopefully the changes i propose here will be> useful to others.
Never heard about such a thing. All the rest of the world is using ELF as long as images need a structure that must be processable by tools, and raw binary blobs for anything else.
we're talking about the format below that ... "u-boot" is a perfectly standard ELF. however, the processor itself does not understand ELF. so at power on, a few pins tell the processor where to start executing ... parallel, spi, uart, whatever. the processor will start fetching data from the defined place and load up the image into memory.
Don't you use any GNU toolchain there? Isn't there any support for the GNU linker? Can't you use standard tools?
the Blackfin port is fully integrated into the GNU toolchain and doing everything the "correct" way.
the exact object format is irrelevant here, but the point is that while the environment is actually embedded inside of the LDR image, it cannot be embedded with the standard method of creating the data array "environment" in environment.c and making sure the environment.o object gets linked at an exact offset. this is because we take the u-boot.bin and run it through another utility to get u-boot.ldr. the offsets in u-boot.bin do not correlate in any way to the offsets in u-boot.ldr nor can they as the LDR format may different drastically by processor.
Ummm... this sounds like linking to me, and if your linker tool performs such shifting and reordering, it also has to adjust the code that references such shifted or reordered memory. If it doesn't, I don't understand what it does at all except from corrupting perfectly good images ;-)
i could update BFD and define a new object format so that we could do: objcopy -O LDR u-boot u-boot.ldr but the end result would be exactly the same -- the offset into the binary blow "u-boot.ldr" would not correspond to the offset into the binary blow "u-boot.bin".
with these comments in mind, i think you can re-evaluate the previous e-mail ;)
--- common/environment.c (revision 950) +++ common/environment.c (working copy)
This makes no sense to me.
./tools/envcrc --binary > environment.bin
and then this bin file can be given to any other tool so that it can be embedded as need be -mike
In message 200710101918.48152.vapier@gentoo.org you wrote:
we're talking about the format below that ... "u-boot" is a perfectly standard ELF. however, the processor itself does not understand ELF. so at power on, a few pins tell the processor where to start executing ... parallel, spi, uart, whatever. the processor will start fetching data from the defined place and load up the image into memory.
But to do so, the "image" (= u-boot.bin ?) should not need to be touched, or why should it?
I mean, I can convert U-Boot and download it as S-RECs or as an Intel Hex file or whatever - this changes just the format, but not the content.
I'm afraid I don't understand what you're trying to say.
i could update BFD and define a new object format so that we could do: objcopy -O LDR u-boot u-boot.ldr but the end result would be exactly the same -- the offset into the binary blow "u-boot.ldr" would not correspond to the offset into the binary blow "u-boot.bin".
I don't understand a single word.
This makes no sense to me.
./tools/envcrc --binary > environment.bin
and then this bin file can be given to any other tool so that it can be embedded as need be
Still doesn't. Sorry.
Best regards,
Wolfgang Denk
On Wednesday 10 October 2007, Wolfgang Denk wrote:
In message 200710101918.48152.vapier@gentoo.org you wrote:
we're talking about the format below that ... "u-boot" is a perfectly standard ELF. however, the processor itself does not understand ELF. so at power on, a few pins tell the processor where to start executing ... parallel, spi, uart, whatever. the processor will start fetching data from the defined place and load up the image into memory.
But to do so, the "image" (= u-boot.bin ?) should not need to be touched, or why should it?
it isnt being touched. the LDR image is being touched. you can think of it as something like: objcopy -O binary u-boot u-boot.bin ( dd if=u-boot.bin count=10 ./tools/envcrc --binary dd if=u-boot.bin skip=10 ) > u-boot.ldr
I mean, I can convert U-Boot and download it as S-RECs or as an Intel Hex file or whatever - this changes just the format, but not the content.
I'm afraid I don't understand what you're trying to say.
because u-boot.bin is not burned into the flash, the u-boot.ldr file is. we want to embed the environment into u-boot.ldr because that is the file that gets burned into flash and because only the first few sectors of the flash are of the "small" variety. so we cannot have the environment live in a different location in flash or it'd be a huge waste.
consider the layout of the flash: first sector: 0x0 - 0x2000 second sector: 0x2000 - 0x4000 third sector: 0x4000 - 0x6000 ... n sector: 0x80000 - 0x90000 n+1 sector: 0x90000 - 0xa0000 ...
we have to make sure that the environment lives at 0x2000 bytes into u-boot.ldr because u-boot.ldr is burned into the flash. using the standard u-boot linker script will only make sure that the environment lives at 0x2000 bytes into u-boot.bin. but the process of converting u-boot.ldr into u-boot.bin adds a variable amount of overhead -- it cannot be precalculated and stored in the board header file so that the linker script makes sure that (0x2000 - ldr format overhead) into u-boot.bin lines up with the 0x2000 offset into u-boot.ldr.
instead, we simply tell the utility that generates u-boot.ldr "create some space at offset 0x2000 of size 0x4000" so that when the u-boot.ldr is actually burned into flash, these offsets correspond to the small sector in the flash reserved for the environment.
however, we still need to populate that initial space we made in the u-boot.ldr, so the patch i proposed allows extracting the binary version of the environment and passing it to the utility which creates the u-boot.ldr. -mike
On Wed 10 Oct 2007 19:28, Wolfgang Denk pondered:
I'm afraid I don't understand what you're trying to say.
The LDR file format, is an ELF-like container that the Blackfin on-chip BootROM understands/requires on new chips. Rather than putting an ELF relocator into the bootrom, and forcing everyone to use ELF files, ADI (not me) made up their own file format. If you are interested, I can point you to the spec, but my guess is you don't really care.
The Blackfin (on newer devices) has no way to do execute in place, or load the u-boot.bin file from flash - I was told that this was done to enable/ensure secure boot. (The first instruction fetch on a reset is always the internal ROM), depending on the settings of external pins & internal registers (OTP), it can load a bootstream (from NAND, NOR, SPI Flash, I2C EEPROM, or UART), potentially validate a checksum, or signature, and either halt, or continue to run/load.
To boot from the newer parts - since we can't use the u-boot.bin file anymore, we are required to create the ADI made up LDR file format.
What Mike has done is to create a tool/utility that takes the standard u-boot (elf) file, and re-packages it into the LDR container. This is distributed with the Blackfin Toolchain rpm/deb/emerge/src under the GPL.
Does that make sense?
Things are complicated by the fact that this LDR format is not static - and continues to evolve with every new part that comes out, as the person who writes the BootROM decides to add new features, that require new switches and bit fields in the LDR stream.
If so, back to Mike's question:
We are having problems embedding the U-Boot environment into this ldr file.
The utility that repackages the u-boot (elf) into the u-boot.ldr has the ability to reserve space inside of the LDR image. For example - we can tell it to reserve/leave blank, the area from 0x4000 to 0x6000.
In the board configuration file, we say the environment lives at an offset of 0x4000 (to correspond to one of the flash sectors) of size say 0x2000. the utility is then told to generate space at an offset in u-boot.ldr at offset 0x4000 of size 0x2000. now the u-boot.ldr can be burned into flash without a problem and saving of the environment is OK.
This seems to work OK.
However, the initial u-boot (elf) is generated without the default environment programmed into it.
Our proposal is to introduce: - a new define "ENV_IS_EMBEDDED_CUSTOM" and - add a new flag to the envcrc helper.
this would allow the environment.o linked into u-boot.bin to be empty/blank and we can use envcrc helper to extract the environment blob so that the blob can be passed to external utilities for embedding.
Dear Robin,
in message 200710150059.20355.rgetz@blackfin.uclinux.org you wrote:
To boot from the newer parts - since we can't use the u-boot.bin file anymore, we are required to create the ADI made up LDR file format.
What Mike has done is to create a tool/utility that takes the standard u-boot (elf) file, and re-packages it into the LDR container. This is distributed with the Blackfin Toolchain rpm/deb/emerge/src under the GPL.
Does that make sense?
Thanks for the explanations - it does.
Things are complicated by the fact that this LDR format is not static - and continues to evolve with every new part that comes out, as the person who writes the BootROM decides to add new features, that require new switches and bit fields in the LDR stream.
But it doesn't rip a binary image apart, or does it? If it does - then how do you actually link U-Boot?
We are having problems embedding the U-Boot environment into this ldr file.
Well, then don't embed it - this is not necessary. There are many board configurations without embedded environment. Embedding the environment makes only sense on certain boards which use boot sector type flash chips where you (1) have to write the U-Boot image to a specific location to match the hardware-defined reset entry point, and (2) you want to use one or more of the smaller boot sectors for the environment, but these are now located right within the U-Boot image.
As far as I understand your decription, you also have small boot sectors you want to use for the environment, but no specific requirement for the location of U-Boot.
Then just reserve the space for the environment sectors, adjust the U-Boot configuration to use these, and be done with it, i. .e leave the envrionment as a separate (not embedded within the image) aread and uninitializedz.
The utility that repackages the u-boot (elf) into the u-boot.ldr has the ability to reserve space inside of the LDR image. For example - we can tell it to reserve/leave blank, the area from 0x4000 to 0x6000.
OK.
In the board configuration file, we say the environment lives at an offset of 0x4000 (to correspond to one of the flash sectors) of size say 0x2000. the utility is then told to generate space at an offset in u-boot.ldr at offset 0x4000 of size 0x2000. now the u-boot.ldr can be burned into flash without a problem and saving of the environment is OK.
This seems to work OK.
Indeed. Sounds OK.
However, the initial u-boot (elf) is generated without the default environment programmed into it.
Stop. There is probably some confusion here.
U-Boot always has a default envrionment compiled in. This is somewhere in the data section and only U-Boot knows where to find it. It gets used if none of the coinfigured environment sectors has a valid checksum.
Note that the default environment has *nothing* to do with the environment stored in the configured flash sector(s). It is an *additional* copy.
Situation is:
* Boards with an embedded environment have *initialized* environment sectors embedded within the image. This may be considered an advantage (because you have an initialized, valid environment in flash when you first install U-Boot on a board) or a disadvantage (because any update of the U-Boot image in flash will overwrite your current environment settings, unless you take spacial care to restore these after programming the U-Boot image to flash).
* Boards with separate envrionment work a bit differently: after intial installation on a virgin system the separate flash sectors are still empty (without valid checksum) and thus U-Boot prints a warning message and uses the compiled in environment. The external flash sectors get initialized only when you run a "saveenv" command. This may be considered an advantage (because your current environment will not be overwritten when you iuntsall a new version of U-Boot) or a disadvantage (because after initial installation of U-Boot no valid environment is found in flash so the warning gets printed).
Whether you prefer one or the other is mostly a matter of taste. In general, seaprate environments are easier to handle and somewhat moe robust.
Our proposal is to introduce:
- a new define "ENV_IS_EMBEDDED_CUSTOM" and
- add a new flag to the envcrc helper.
I see no need for this.
this would allow the environment.o linked into u-boot.bin to be empty/blank and we can use envcrc helper to extract the environment blob so that the blob can be passed to external utilities for embedding.
If you don't want to embed the environment (or cannot do it), then just don't do it. Configure as described above, and use it as a separate, detached environment like all the other boards using such a configuration.
[Maybe we could meet at #u-boot and discuss this online if you feel I'm too stubborn to understand the obvious..]
Best regards,
Wolfgang Denk
On Monday 15 October 2007, Wolfgang Denk wrote:
in message 200710150059.20355.rgetz@blackfin.uclinux.org you wrote:
Things are complicated by the fact that this LDR format is not static - and continues to evolve with every new part that comes out, as the person who writes the BootROM decides to add new features, that require new switches and bit fields in the LDR stream.
But it doesn't rip a binary image apart, or does it? If it does - then how do you actually link U-Boot?
it doesnt. the .bin -> .ldr conversion happens after everything. we already have a utility in u-boot mainline that does this (bin2ldr), ive just implemented a more generic version in our svn tree since.
We are having problems embedding the U-Boot environment into this ldr file.
Well, then don't embed it - this is not necessary. There are many board configurations without embedded environment. Embedding the environment makes only sense on certain boards which use boot sector type flash chips where you (1) have to write the U-Boot image to a specific location to match the hardware-defined reset entry point, and (2) you want to use one or more of the smaller boot sectors for the environment, but these are now located right within the U-Boot image.
As far as I understand your decription, you also have small boot sectors you want to use for the environment, but no specific requirement for the location of U-Boot.
there is a specific requirement for the location of U-Boot. it *must* be at the very beginning of flash (where all the boot sectors are). the bootrom on the Blackfin does not support starting at anywhere else, it will only start reading from the very beginning.
However, the initial u-boot (elf) is generated without the default environment programmed into it.
Stop. There is probably some confusion here.
U-Boot always has a default envrionment compiled in. This is somewhere in the data section and only U-Boot knows where to find it. It gets used if none of the coinfigured environment sectors has a valid checksum.
Note that the default environment has *nothing* to do with the environment stored in the configured flash sector(s). It is an *additional* copy.
i am fully aware of this, but that does not change what i want to do. by this argument, there's no reason to have the tools/envcrc utility in the first place. if it isnt designed specifically for embedding the proper crc-ed version into the u-boot image, then it's a useless utility.
this would allow the environment.o linked into u-boot.bin to be empty/blank and we can use envcrc helper to extract the environment blob so that the blob can be passed to external utilities for embedding.
If you don't want to embed the environment (or cannot do it), then just don't do it. Configure as described above, and use it as a separate, detached environment like all the other boards using such a configuration.
except that we *do* want to do it and we *can* as outlined already. -mike
participants (3)
-
Mike Frysinger -
Robin Getz -
Wolfgang Denk