Re: [U-Boot] Where to put a large bootloader-supplied device tree on ARM ?
[adding u-boot list]
On 07/12/2012 01:52 AM, Mitch Bradley wrote:
On 7/8/2012 6:30 PM, Nicolas Pitre wrote:
On Fri, 6 Jul 2012, Mitch Bradley wrote:
On 7/6/2012 3:23 PM, David VomLehn (dvomlehn) wrote:
The kernel *must* go where it is linked, but the FDT contains only relative references and is thus free to go anywhere. The same is true of ramdisks, which are usually placed after the kernel.
The kernel must go where it is linked *only* if you are using the 'Image' output. When using 'zImage' you can put the kernel anywhere in memory, or in the first 128MB of RAM if CONFIG_AUTO_ZRELADDR is used.
Right, but the kernel image is compressed, so after decompression it expands into the area just after it. Also, the .bss segment is in that vicinity.
To be exact, the compressed kernel moves itself out of the region where the decompressed kernel will end up before doing the decompression, but only if necessary. So it is a good idea to load zImage away from the decompressed kernel area to avoid this extra move and save some fraction of a second on boot time.
There's some code in arch/arm/boot/compressed/head.S to relocate device tree blobs, but it requires CONFIG_ARM_APPENDED_DTB which is not recommended - arch/arm/Kconfig recommends using the documented boot protocol istead .
This is in case a DTB is appended to zImage. When the DTB is detected, the moving of zImage out of the decompressed area must take care of moving the DTB as well.
Documentation/arm/Booting says to put the dtb "in a region of memory where the kernel decompressor will not overwrite it", further recommending the first 16KiB.
As noted, the first 16KiB loses if the dtb is too large. And "where the kernel decompressor will not overwrite it" says what won't work, not what will. It appears that the decompressor works out its addresses dynamically, so there's no hard prescription even for what to avoid.
A good rule of thumb is to take the size of the decompressed kernel and multiply this by 3. Rounding up is also fine. So for example if your arch/arm/boot/Image is 5MB, then putting anciliary data such as a ramdisk or a large DTB from 16MB into RAM or above should be fine.
For now, I'm putting the initrd at the end of memory and the dtb below that. That seems to work, but I'm unsure whether or not I'm just "getting lucky".
That's also perfectly fine.
Alas, that worked for machines with 512 MiB of main memory, but failed on 1 GiB machines. My guess is that, when the initrd and dtb are near the top of a 1 GiB memory, the virtual address gets too near the top of the kernel's 1 GiB of virtual space (which starts at 0xc0000000), perhaps colliding with the VMALLOC space.
Putting them just below the 128 MiB boundary seems to work.
Interesting. I think this is also a problem on u-boot just waiting to happen. u-boot locates itself at the end of RAM and likes to copy the fdt and initrd to just below that. Any machine with 1G+ is going to hit this. I avoided it because I limited u-boot to 512MB on highbank.
Rob
Nicolas
devicetree-discuss mailing list devicetree-discuss@lists.ozlabs.org https://lists.ozlabs.org/listinfo/devicetree-discuss
Hi Rob,
On Thu, 12 Jul 2012 15:34:17 -0500, Rob Herring robherring2@gmail.com wrote:
[adding u-boot list]
On 07/12/2012 01:52 AM, Mitch Bradley wrote:
On 7/8/2012 6:30 PM, Nicolas Pitre wrote:
On Fri, 6 Jul 2012, Mitch Bradley wrote:
On 7/6/2012 3:23 PM, David VomLehn (dvomlehn) wrote:
The kernel *must* go where it is linked, but the FDT contains only relative references and is thus free to go anywhere. The same is true of ramdisks, which are usually placed after the kernel.
The kernel must go where it is linked *only* if you are using the 'Image' output. When using 'zImage' you can put the kernel anywhere in memory, or in the first 128MB of RAM if CONFIG_AUTO_ZRELADDR is used.
Right, but the kernel image is compressed, so after decompression it expands into the area just after it. Also, the .bss segment is in that vicinity.
To be exact, the compressed kernel moves itself out of the region where the decompressed kernel will end up before doing the decompression, but only if necessary. So it is a good idea to load zImage away from the decompressed kernel area to avoid this extra move and save some fraction of a second on boot time.
There's some code in arch/arm/boot/compressed/head.S to relocate device tree blobs, but it requires CONFIG_ARM_APPENDED_DTB which is not recommended - arch/arm/Kconfig recommends using the documented boot protocol istead .
This is in case a DTB is appended to zImage. When the DTB is detected, the moving of zImage out of the decompressed area must take care of moving the DTB as well.
Documentation/arm/Booting says to put the dtb "in a region of memory where the kernel decompressor will not overwrite it", further recommending the first 16KiB.
As noted, the first 16KiB loses if the dtb is too large. And "where the kernel decompressor will not overwrite it" says what won't work, not what will. It appears that the decompressor works out its addresses dynamically, so there's no hard prescription even for what to avoid.
A good rule of thumb is to take the size of the decompressed kernel and multiply this by 3. Rounding up is also fine. So for example if your arch/arm/boot/Image is 5MB, then putting anciliary data such as a ramdisk or a large DTB from 16MB into RAM or above should be fine.
For now, I'm putting the initrd at the end of memory and the dtb below that. That seems to work, but I'm unsure whether or not I'm just "getting lucky".
That's also perfectly fine.
Alas, that worked for machines with 512 MiB of main memory, but failed on 1 GiB machines. My guess is that, when the initrd and dtb are near the top of a 1 GiB memory, the virtual address gets too near the top of the kernel's 1 GiB of virtual space (which starts at 0xc0000000), perhaps colliding with the VMALLOC space.
Putting them just below the 128 MiB boundary seems to work.
Interesting. I think this is also a problem on u-boot just waiting to happen. u-boot locates itself at the end of RAM and likes to copy the fdt and initrd to just below that. Any machine with 1G+ is going to hit this. I avoided it because I limited u-boot to 512MB on highbank.
If I'm not mistaken, yes U-Boot loads itself as high as it can, and I don't know about the FDT, but no, U-Boot does not "like" to load initrd "just below that": it loads initrd where the boot commands tell it to, and the boot commands are written by board developers. Nothing in U-Boot forces initrd to be loaded as high as possible.
That leaves the question of the FDT, though -- I'm not familiar enough with it (yet) to tell if it is always located just under U-Boot or if its placement is controllable by board commands.
Amicalement,
Dear Albert ARIBAUD,
In message 20120712233801.0411daa7@lilith you wrote:
If I'm not mistaken, yes U-Boot loads itself as high as it can, and I don't know about the FDT, but no, U-Boot does not "like" to load initrd "just below that": it loads initrd where the boot commands tell it to, and the boot commands are written by board developers. Nothing in U-Boot forces initrd to be loaded as high as possible.
That leaves the question of the FDT, though -- I'm not familiar enough with it (yet) to tell if it is always located just under U-Boot or if its placement is controllable by board commands.
You can always just set the "fdt_high" and "initrd_high" evironment variables to restrict the positioning of initrd and FDT in RAM; see the README for details.
Best regards,
Wolfgang Denk
On 07/12/2012 04:47 PM, Wolfgang Denk wrote:
Dear Albert ARIBAUD,
In message 20120712233801.0411daa7@lilith you wrote:
If I'm not mistaken, yes U-Boot loads itself as high as it can, and I don't know about the FDT, but no, U-Boot does not "like" to load initrd "just below that": it loads initrd where the boot commands tell it to, and the boot commands are written by board developers. Nothing in U-Boot forces initrd to be loaded as high as possible.
u-boot loads the initrd where you tell it, then bootm relocates it for some reason.
That leaves the question of the FDT, though -- I'm not familiar enough with it (yet) to tell if it is always located just under U-Boot or if its placement is controllable by board commands.
You can always just set the "fdt_high" and "initrd_high" evironment variables to restrict the positioning of initrd and FDT in RAM; see the README for details.
Yes, I'm aware of all this, but this is not the default behavior and the default behavior will not work in this case. It also has another bug related to the relocation that I fixed:
http://www.mail-archive.com/u-boot@lists.denx.de/msg86475.html
Rob
Hi Rob,
On Thu, 12 Jul 2012 20:28:22 -0500, Rob Herring robherring2@gmail.com wrote:
On 07/12/2012 04:47 PM, Wolfgang Denk wrote:
Dear Albert ARIBAUD,
In message 20120712233801.0411daa7@lilith you wrote:
If I'm not mistaken, yes U-Boot loads itself as high as it can, and I don't know about the FDT, but no, U-Boot does not "like" to load initrd "just below that": it loads initrd where the boot commands tell it to, and the boot commands are written by board developers. Nothing in U-Boot forces initrd to be loaded as high as possible.
u-boot loads the initrd where you tell it, then bootm relocates it for some reason.
That leaves the question of the FDT, though -- I'm not familiar enough with it (yet) to tell if it is always located just under U-Boot or if its placement is controllable by board commands.
You can always just set the "fdt_high" and "initrd_high" evironment variables to restrict the positioning of initrd and FDT in RAM; see the README for details.
Yes, I'm aware of all this, but this is not the default behavior and the default behavior will not work in this case. It also has another bug related to the relocation that I fixed:
http://www.mail-archive.com/u-boot@lists.denx.de/msg86475.html
Rob
As for the default behavior, this becomes a board config issue, not a U-Boot issue.
Regarding the patch, as a bugfix it will go in u-boot-arm/master for 2012.07.
With all this, do we have the overall issue covered?
Amicalement,
participants (3)
-
Albert ARIBAUD -
Rob Herring -
Wolfgang Denk