[U-Boot] Default initrd and fdt load behaviour on ARM (Was: Re: [linux-sunxi] Re: Not able to boot ramdisk on cubietruck)
Adding u-boot list since I think this is a general issue/question.
On Sun, 2014-03-09 at 20:00 +0100, Olliver Schinagl wrote:
On 03/09/14 16:18, tyler.baker@linaro.org wrote:
On Sunday, March 9, 2014 8:06:27 AM UTC-7, tyler...@linaro.org wrote:
Hello,
I am trying to boot the cubietruck with a minimal ramdisk. However, it seems to hang at "Starting kernel..." whenever I pass bootz a ramdisk load address.
[...]
Turl help me solve this in IRC. Needed to setenv initrd_high '0xffffffff' in case anyone else runs into this.
The http://linux-sunxi.org/Mainline_Kernel_Howto did mention this ;) but thanks for helping remind people!
Actually it mentions fdt_high but not initrd_high.
But what is the reason for the default behaviour of bootz doing things which do not conform to linux/Documentation/arm/Booting and therefore is not going to boot?
The docs recommend that DTB and initrd go just after the 128MB boundary. If either are loaded "too high" then the kernel will crash on boot, often in a tricky to diagnose way (I've chased it down more than once). It's especially annoying when one has carefully loaded everything at a suitable address in the first place ;-)
Should the global default be changed to either 0xffffffff (no relocation) or to start-of-ram+256MB (which should satisfy the kernels requirements without needing logic changes to handle "just after the 128MB boundary" as a concept).
Or is it intended that each board should opt-in to a sensible default via their default environment?
Does bootm suffer the same? I suspect it does, at least for fdt (since initrd has a load addr in the u-boot header).
Ian.
On Tue, Mar 11, 2014 at 10:38:05AM +0000, Ian Campbell wrote:
Adding u-boot list since I think this is a general issue/question.
On Sun, 2014-03-09 at 20:00 +0100, Olliver Schinagl wrote:
On 03/09/14 16:18, tyler.baker@linaro.org wrote:
On Sunday, March 9, 2014 8:06:27 AM UTC-7, tyler...@linaro.org wrote:
Hello,
I am trying to boot the cubietruck with a minimal ramdisk. However, it seems to hang at "Starting kernel..." whenever I pass bootz a ramdisk load address.
[...]
Turl help me solve this in IRC. Needed to setenv initrd_high '0xffffffff' in case anyone else runs into this.
The http://linux-sunxi.org/Mainline_Kernel_Howto did mention this ;) but thanks for helping remind people!
Actually it mentions fdt_high but not initrd_high.
But what is the reason for the default behaviour of bootz doing things which do not conform to linux/Documentation/arm/Booting and therefore is not going to boot?
Because U-Boot is more than just ARM, and ARM boards not enforcing / getting the correct behavour via fdt_high / initrd_high or bootm_low / bootm_mapsize / bootm_size / CONFIG_SYS_BOOTMAPSZ.
The docs recommend that DTB and initrd go just after the 128MB boundary. If either are loaded "too high" then the kernel will crash on boot, often in a tricky to diagnose way (I've chased it down more than once). It's especially annoying when one has carefully loaded everything at a suitable address in the first place ;-)
I have as well, sadly.
Should the global default be changed to either 0xffffffff (no relocation) or to start-of-ram+256MB (which should satisfy the kernels requirements without needing logic changes to handle "just after the 128MB boundary" as a concept).
Or is it intended that each board should opt-in to a sensible default via their default environment?
Does bootm suffer the same? I suspect it does, at least for fdt (since initrd has a load addr in the u-boot header).
SoCs should set a sane and correct set of values, using either of the available mechanisms. I think Tegra gets this all correct via CONFIG_SYS_BOOTMAPSZ and I've been meaning to whack the various TI parts as part of the generic distro work.
And keep in mind that when we set fdt/initrd_high to a non-0xffffffff value we're setting a maximum that's still checked vs how much we have, so setting base+512MB is probably a best default for cases where we may run on boards with varying amounts of DDR.
On Tue, 2014-03-11 at 09:48 -0400, Tom Rini wrote:
On Tue, Mar 11, 2014 at 10:38:05AM +0000, Ian Campbell wrote:
But what is the reason for the default behaviour of bootz doing things which do not conform to linux/Documentation/arm/Booting and therefore is not going to boot?
Because U-Boot is more than just ARM, and ARM boards not enforcing / getting the correct behavour via fdt_high / initrd_high or bootm_low / bootm_mapsize / bootm_size / CONFIG_SYS_BOOTMAPSZ.
So you consider this to be a board level problem, rather than an arch level one? (IOW we can't just fix this for all arm boards at once)
[...]
Should the global default be changed to either 0xffffffff (no relocation) or to start-of-ram+256MB (which should satisfy the kernels requirements without needing logic changes to handle "just after the 128MB boundary" as a concept).
Or is it intended that each board should opt-in to a sensible default via their default environment?
Does bootm suffer the same? I suspect it does, at least for fdt (since initrd has a load addr in the u-boot header).
SoCs should set a sane and correct set of values, using either of the available mechanisms. I think Tegra gets this all correct via CONFIG_SYS_BOOTMAPSZ and I've been meaning to whack the various TI parts as part of the generic distro work.
I took a look at BOOTMAPSZ on sunxi and it is using (16 << 20) which seems sensible, if a bit small, so perhaps something else is going wrong.
Tegra uses 256MB, might be worth giving that a go I guess.
And keep in mind that when we set fdt/initrd_high to a non-0xffffffff value we're setting a maximum that's still checked vs how much we have, so setting base+512MB is probably a best default for cases where we may run on boards with varying amounts of DDR.
I'm not sure I follow -- why is +512MB best?
Thanks, Ian.
On Tue, Mar 11, 2014 at 03:21:54PM +0000, Ian Campbell wrote:
On Tue, 2014-03-11 at 09:48 -0400, Tom Rini wrote:
On Tue, Mar 11, 2014 at 10:38:05AM +0000, Ian Campbell wrote:
But what is the reason for the default behaviour of bootz doing things which do not conform to linux/Documentation/arm/Booting and therefore is not going to boot?
Because U-Boot is more than just ARM, and ARM boards not enforcing / getting the correct behavour via fdt_high / initrd_high or bootm_low / bootm_mapsize / bootm_size / CONFIG_SYS_BOOTMAPSZ.
So you consider this to be a board level problem, rather than an arch level one? (IOW we can't just fix this for all arm boards at once)
I'm not 100% sure. An issue is that lowmem isn't always 512MB, it may be 768MB. That also may not really matter for this, I'm not sure. But part of the problem is that AFAICS no one provides a 100% correct example today. With some testing around now, for TI platforms (DDR starts at 0x80000000 it would be: /* * We setup defaults based on constraints from the Linux kernel, which * should also be safe elsewhere. We have the default load at 32MB into * DDR (for the kernel), FDT above 128MB (the maximum location for the * end of the kernel), and the ramdisk 512KB above that (allowing for * hopefully never seen large trees). We say all of this must be within * the first 512MB as that will always be within the kernel lowmem and * thus visible via bootm_size. */ #define CONFIG_EXTRA_ENV_SETTINGS \ - "loadaddr=0x80200000\0" \ - "fdtaddr=0x80F80000\0" \ - "fdt_high=0xffffffff\0" \ + "loadaddr=0x82000000\0" \ + "fdtaddr=0x88000000\0" \ + "rdaddr=0x88080000\0" \ + "bootm_size=0x20000000\0" \
All of that said, I could be convinced that we could try and make common/image.c::getenv_bootm_size() smarter about defaults but we need to take a lot of care there, and allow people to opt-out as that's common code and how much lowmem we have is configurable a lot.
[...]
Should the global default be changed to either 0xffffffff (no relocation) or to start-of-ram+256MB (which should satisfy the kernels requirements without needing logic changes to handle "just after the 128MB boundary" as a concept).
Or is it intended that each board should opt-in to a sensible default via their default environment?
Does bootm suffer the same? I suspect it does, at least for fdt (since initrd has a load addr in the u-boot header).
SoCs should set a sane and correct set of values, using either of the available mechanisms. I think Tegra gets this all correct via CONFIG_SYS_BOOTMAPSZ and I've been meaning to whack the various TI parts as part of the generic distro work.
I took a look at BOOTMAPSZ on sunxi and it is using (16 << 20) which seems sensible, if a bit small, so perhaps something else is going wrong.
Tegra uses 256MB, might be worth giving that a go I guess.
Doing some playing around, BOOTMAPSZ isn't a complete solution, you need bootm_low also set. In fact, I found the best results with just doing botom_low but there may be cases where you also want BOOTMAPSZ.
And keep in mind that when we set fdt/initrd_high to a non-0xffffffff value we're setting a maximum that's still checked vs how much we have, so setting base+512MB is probably a best default for cases where we may run on boards with varying amounts of DDR.
I'm not sure I follow -- why is +512MB best?
I'm a fan of setting things to the maximum allowed, to allow for future growth. This would also allow for pretty giant ramdisks to be passed which is unlikely but not impossible (I need to debug X and I can't use nfs/sd/etc for root, let me assemble a big ramdisk with everything I need). Getting back to why this would be left up to the SoC/board level, if for some reason there's a reason to not go up quite that high, they can tweak it.
On Tue, 2014-03-11 at 13:17 -0400, Tom Rini wrote:
On Tue, Mar 11, 2014 at 03:21:54PM +0000, Ian Campbell wrote:
On Tue, 2014-03-11 at 09:48 -0400, Tom Rini wrote:
On Tue, Mar 11, 2014 at 10:38:05AM +0000, Ian Campbell wrote:
But what is the reason for the default behaviour of bootz doing things which do not conform to linux/Documentation/arm/Booting and therefore is not going to boot?
Because U-Boot is more than just ARM, and ARM boards not enforcing / getting the correct behavour via fdt_high / initrd_high or bootm_low / bootm_mapsize / bootm_size / CONFIG_SYS_BOOTMAPSZ.
So you consider this to be a board level problem, rather than an arch level one? (IOW we can't just fix this for all arm boards at once)
I'm not 100% sure. An issue is that lowmem isn't always 512MB, it may be 768MB.
I think in reality it can vary substantially between the two (or even outside it) based on the kernel's .config (so outside u-boot's control).
That also may not really matter for this, I'm not sure. But part of the problem is that AFAICS no one provides a 100% correct example today. With some testing around now, for TI platforms (DDR starts at 0x80000000 it would be: /*
- We setup defaults based on constraints from the Linux kernel, which
- should also be safe elsewhere. We have the default load at 32MB into
- DDR (for the kernel), FDT above 128MB (the maximum location for the
- end of the kernel), and the ramdisk 512KB above that (allowing for
- hopefully never seen large trees). We say all of this must be within
- the first 512MB as that will always be within the kernel lowmem and
- thus visible via bootm_size.
*/ #define CONFIG_EXTRA_ENV_SETTINGS \
"loadaddr=0x80200000\0" \"fdtaddr=0x80F80000\0" \"fdt_high=0xffffffff\0" \
"loadaddr=0x82000000\0" \"fdtaddr=0x88000000\0" \"rdaddr=0x88080000\0" \"bootm_size=0x20000000\0" \All of that said, I could be convinced that we could try and make common/image.c::getenv_bootm_size() smarter about defaults but we need to take a lot of care there, and allow people to opt-out as that's common code and how much lowmem we have is configurable a lot.
[...]
Should the global default be changed to either 0xffffffff (no relocation) or to start-of-ram+256MB (which should satisfy the kernels requirements without needing logic changes to handle "just after the 128MB boundary" as a concept).
Or is it intended that each board should opt-in to a sensible default via their default environment?
Does bootm suffer the same? I suspect it does, at least for fdt (since initrd has a load addr in the u-boot header).
SoCs should set a sane and correct set of values, using either of the available mechanisms. I think Tegra gets this all correct via CONFIG_SYS_BOOTMAPSZ and I've been meaning to whack the various TI parts as part of the generic distro work.
I took a look at BOOTMAPSZ on sunxi and it is using (16 << 20) which seems sensible, if a bit small, so perhaps something else is going wrong.
Tegra uses 256MB, might be worth giving that a go I guess.
Doing some playing around, BOOTMAPSZ isn't a complete solution, you need bootm_low also set. In fact, I found the best results with just doing botom_low but there may be cases where you also want BOOTMAPSZ.
I'll put fiddling with this for sunxi on my todo list. Although I kind of hope someone else will beat me to it ;-)
And keep in mind that when we set fdt/initrd_high to a non-0xffffffff value we're setting a maximum that's still checked vs how much we have, so setting base+512MB is probably a best default for cases where we may run on boards with varying amounts of DDR.
I'm not sure I follow -- why is +512MB best?
I'm a fan of setting things to the maximum allowed, to allow for future growth. This would also allow for pretty giant ramdisks to be passed which is unlikely but not impossible (I need to debug X and I can't use nfs/sd/etc for root, let me assemble a big ramdisk with everything I need).
That makes sense.
On the other hand the "just above 128MB" advice is even better since that will Just Work and allow images to stretch to the end of RAM if necessary. But I suppose that is a far bigger rework on the u-boot side of things, to flip from "maximum address" allocation to "starting from" allocation. I don't think I want to get into that...
Getting back to why this would be left up to the SoC/board level, if for some reason there's a reason to not go up quite that high, they can tweak it.
Ian.
On Wed, Mar 12, 2014 at 01:04:38PM +0000, Ian Campbell wrote:
On Tue, 2014-03-11 at 13:17 -0400, Tom Rini wrote:
On Tue, Mar 11, 2014 at 03:21:54PM +0000, Ian Campbell wrote:
On Tue, 2014-03-11 at 09:48 -0400, Tom Rini wrote:
On Tue, Mar 11, 2014 at 10:38:05AM +0000, Ian Campbell wrote:
But what is the reason for the default behaviour of bootz doing things which do not conform to linux/Documentation/arm/Booting and therefore is not going to boot?
Because U-Boot is more than just ARM, and ARM boards not enforcing / getting the correct behavour via fdt_high / initrd_high or bootm_low / bootm_mapsize / bootm_size / CONFIG_SYS_BOOTMAPSZ.
So you consider this to be a board level problem, rather than an arch level one? (IOW we can't just fix this for all arm boards at once)
I'm not 100% sure. An issue is that lowmem isn't always 512MB, it may be 768MB.
I think in reality it can vary substantially between the two (or even outside it) based on the kernel's .config (so outside u-boot's control).
Yeah, I think you're right. Which is why this is a board/SoC level problem to provide "good enough" defaults.
That also may not really matter for this, I'm not sure. But part of the problem is that AFAICS no one provides a 100% correct example today. With some testing around now, for TI platforms (DDR starts at 0x80000000 it would be: /*
- We setup defaults based on constraints from the Linux kernel, which
- should also be safe elsewhere. We have the default load at 32MB into
- DDR (for the kernel), FDT above 128MB (the maximum location for the
- end of the kernel), and the ramdisk 512KB above that (allowing for
- hopefully never seen large trees). We say all of this must be within
- the first 512MB as that will always be within the kernel lowmem and
- thus visible via bootm_size.
*/ #define CONFIG_EXTRA_ENV_SETTINGS \
"loadaddr=0x80200000\0" \"fdtaddr=0x80F80000\0" \"fdt_high=0xffffffff\0" \
"loadaddr=0x82000000\0" \"fdtaddr=0x88000000\0" \"rdaddr=0x88080000\0" \"bootm_size=0x20000000\0" \All of that said, I could be convinced that we could try and make common/image.c::getenv_bootm_size() smarter about defaults but we need to take a lot of care there, and allow people to opt-out as that's common code and how much lowmem we have is configurable a lot.
[...]
Should the global default be changed to either 0xffffffff (no relocation) or to start-of-ram+256MB (which should satisfy the kernels requirements without needing logic changes to handle "just after the 128MB boundary" as a concept).
Or is it intended that each board should opt-in to a sensible default via their default environment?
Does bootm suffer the same? I suspect it does, at least for fdt (since initrd has a load addr in the u-boot header).
SoCs should set a sane and correct set of values, using either of the available mechanisms. I think Tegra gets this all correct via CONFIG_SYS_BOOTMAPSZ and I've been meaning to whack the various TI parts as part of the generic distro work.
I took a look at BOOTMAPSZ on sunxi and it is using (16 << 20) which seems sensible, if a bit small, so perhaps something else is going wrong.
Tegra uses 256MB, might be worth giving that a go I guess.
Doing some playing around, BOOTMAPSZ isn't a complete solution, you need bootm_low also set. In fact, I found the best results with just doing botom_low but there may be cases where you also want BOOTMAPSZ.
I'll put fiddling with this for sunxi on my todo list. Although I kind of hope someone else will beat me to it ;-)
Having whacked at it for the TI parts, I strongly suspect sunxi can also get away with defaulting to including bootm_low (which yes, also covers bootz) set to 512MB or 256MB and possibly also...
And keep in mind that when we set fdt/initrd_high to a non-0xffffffff value we're setting a maximum that's still checked vs how much we have, so setting base+512MB is probably a best default for cases where we may run on boards with varying amounts of DDR.
I'm not sure I follow -- why is +512MB best?
I'm a fan of setting things to the maximum allowed, to allow for future growth. This would also allow for pretty giant ramdisks to be passed which is unlikely but not impossible (I need to debug X and I can't use nfs/sd/etc for root, let me assemble a big ramdisk with everything I need).
That makes sense.
On the other hand the "just above 128MB" advice is even better since that will Just Work and allow images to stretch to the end of RAM if necessary. But I suppose that is a far bigger rework on the u-boot side of things, to flip from "maximum address" allocation to "starting from" allocation. I don't think I want to get into that...
Nope, you've got it backwards. If set set bootm_low to 512MB then we allow for ramdisks to grow to the end of usable memory (and since it's overridable if you have a case with a kernel of 1GB lowmem and a 768MB ramdisk, you can adjust and be fine). The starting point for these should be just above 128MB (unless you know you have to work with 128MB or smaller boards) since that is a real limit on usable ARM kernels, without major rework.
But, "starting from" allocation would just be loading from low and setting fdt/initrd_high to 0xffffffff so we don't move it.
participants (2)
-
Ian Campbell -
Tom Rini