[U-Boot] [RFC] ARM: Start using fdt_high for relocation
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting fdt_high (in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Hi Tom,
On Fri, 6 Dec 2013 10:48:50 -0500, Tom Rini trini@ti.com wrote:
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting fdt_high (in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Amicalement,
On Fri, Dec 06, 2013 at 05:59:37PM +0100, Albert ARIBAUD wrote:
Hi Tom,
On Fri, 6 Dec 2013 10:48:50 -0500, Tom Rini trini@ti.com wrote:
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting fdt_high (in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Sure. Lets take am335x_evm builds (so Beaglebone Black/White, etc). If you start enabling all of the tracing options in the kernel (function tracing, graphs, etc), you get an uncompressed kernel and BSS that will use up the first ~16MiB of DDR. We default to placing the DT at about 15MiB into memory. So the kernel runs, clears BSS and eats the DT. System now hangs, and depending on debug options set you may or may not see anything at all from the kernel. U-Boot couldn't detect this failure because we don't know how big the kernel BSS is, only how big the zImage is (and where it is) and how big the fdt is and where it is. No overlaps, go ahead and run.
Hi Tom,
On Fri, 6 Dec 2013 12:18:13 -0500, Tom Rini trini@ti.com wrote:
On Fri, Dec 06, 2013 at 05:59:37PM +0100, Albert ARIBAUD wrote:
Hi Tom,
On Fri, 6 Dec 2013 10:48:50 -0500, Tom Rini trini@ti.com wrote:
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting fdt_high (in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Sure. Lets take am335x_evm builds (so Beaglebone Black/White, etc). If you start enabling all of the tracing options in the kernel (function tracing, graphs, etc), you get an uncompressed kernel and BSS that will use up the first ~16MiB of DDR. We default to placing the DT at about 15MiB into memory. So the kernel runs, clears BSS and eats the DT. System now hangs, and depending on debug options set you may or may not see anything at all from the kernel. U-Boot couldn't detect this failure because we don't know how big the kernel BSS is, only how big the zImage is (and where it is) and how big the fdt is and where it is. No overlaps, go ahead and run.
Thanks.
The only issue I have with the RFC is that the +512 MiB value will only work with targets which have more than 512 MiB DDR, right? But since you're suggesting this should be set in configs, you are only suggesting +512 MiB, and any target could actually specify a lower value as long as it's greater than or eqal to 16 MiB. Correct?
Amicalement,
On Fri, Dec 06, 2013 at 08:28:04PM +0100, Albert ARIBAUD wrote:
Hi Tom,
On Fri, 6 Dec 2013 12:18:13 -0500, Tom Rini trini@ti.com wrote:
On Fri, Dec 06, 2013 at 05:59:37PM +0100, Albert ARIBAUD wrote:
Hi Tom,
On Fri, 6 Dec 2013 10:48:50 -0500, Tom Rini trini@ti.com wrote:
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting fdt_high (in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Sure. Lets take am335x_evm builds (so Beaglebone Black/White, etc). If you start enabling all of the tracing options in the kernel (function tracing, graphs, etc), you get an uncompressed kernel and BSS that will use up the first ~16MiB of DDR. We default to placing the DT at about 15MiB into memory. So the kernel runs, clears BSS and eats the DT. System now hangs, and depending on debug options set you may or may not see anything at all from the kernel. U-Boot couldn't detect this failure because we don't know how big the kernel BSS is, only how big the zImage is (and where it is) and how big the fdt is and where it is. No overlaps, go ahead and run.
Thanks.
The only issue I have with the RFC is that the +512 MiB value will only work with targets which have more than 512 MiB DDR, right? But since you're suggesting this should be set in configs, you are only suggesting +512 MiB, and any target could actually specify a lower value as long as it's greater than or eqal to 16 MiB. Correct?
fdt_high is only an upper bound on what we may relocate to (setting aside the magic value of 0xffffffff which means no relocation). I just confirmed this too: U-Boot# bdi ... boot_params = 0x80000100 DRAM bank = 0x00000000 -> start = 0x80000000 -> size = 0x40000000 ... U-Boot# setenv fdt_high 0xe0000000 ... U-Boot# bootz $loadaddr - $fdtaddr Kernel image @ 0x80200000 [ 0x000000 - 0x3e5068 ] ## Flattened Device Tree blob at 80f80000 Booting using the fdt blob at 0x80f80000 Loading Device Tree to bf62a000, end bf630d9a ... OK
Of course, 0xbf62a000 is a bad address in that it won't be seen by the kernel, but it was in the higher-than-we-have-memory-at limit I set.
On 12/06/2013 01:31 PM, Tom Rini wrote:
On Fri, Dec 06, 2013 at 08:28:04PM +0100, Albert ARIBAUD wrote:
Hi Tom,
On Fri, 6 Dec 2013 12:18:13 -0500, Tom Rini trini@ti.com wrote:
On Fri, Dec 06, 2013 at 05:59:37PM +0100, Albert ARIBAUD wrote:
Hi Tom,
On Fri, 6 Dec 2013 10:48:50 -0500, Tom Rini trini@ti.com wrote:
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting fdt_high (in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Sure. Lets take am335x_evm builds (so Beaglebone Black/White, etc). If you start enabling all of the tracing options in the kernel (function tracing, graphs, etc), you get an uncompressed kernel and BSS that will use up the first ~16MiB of DDR. We default to placing the DT at about 15MiB into memory. So the kernel runs, clears BSS and eats the DT. System now hangs, and depending on debug options set you may or may not see anything at all from the kernel. U-Boot couldn't detect this failure because we don't know how big the kernel BSS is, only how big the zImage is (and where it is) and how big the fdt is and where it is. No overlaps, go ahead and run.
Thanks.
The only issue I have with the RFC is that the +512 MiB value will only work with targets which have more than 512 MiB DDR, right? But since you're suggesting this should be set in configs, you are only suggesting +512 MiB, and any target could actually specify a lower value as long as it's greater than or eqal to 16 MiB. Correct?
fdt_high is only an upper bound on what we may relocate to (setting aside the magic value of 0xffffffff which means no relocation). I just confirmed this too: U-Boot# bdi ... boot_params = 0x80000100 DRAM bank = 0x00000000 -> start = 0x80000000 -> size = 0x40000000 ... U-Boot# setenv fdt_high 0xe0000000 ... U-Boot# bootz $loadaddr - $fdtaddr Kernel image @ 0x80200000 [ 0x000000 - 0x3e5068 ] ## Flattened Device Tree blob at 80f80000 Booting using the fdt blob at 0x80f80000 Loading Device Tree to bf62a000, end bf630d9a ... OK
Of course, 0xbf62a000 is a bad address in that it won't be seen by the kernel, but it was in the higher-than-we-have-memory-at limit I set.
I haven't really followed this thread since I just noticed it tangentially, but on Tegra...
Since commit 7f1b767aea94 "ARM: tegra: define CONFIG_SYS_BOOTMAPSZ" all Tegra boards define BOOTMAPSZ to influence where the DTB gets relocated. I wonder what's the benefit of using BOOTMAPSZ vs. defining a default value for $fdt_high?
And couldn't you just move $fdt_addr_r high enough in RAM so even if the DTB wasn't relocated it'd never overlap BSS? Tegra's $kernel_addr_r and $fdt_addr_r are likely set up that way already, although I didn't check.
On Fri, Dec 06, 2013 at 01:44:22PM -0700, Stephen Warren wrote:
On 12/06/2013 01:31 PM, Tom Rini wrote:
On Fri, Dec 06, 2013 at 08:28:04PM +0100, Albert ARIBAUD wrote:
Hi Tom,
On Fri, 6 Dec 2013 12:18:13 -0500, Tom Rini trini@ti.com wrote:
On Fri, Dec 06, 2013 at 05:59:37PM +0100, Albert ARIBAUD wrote:
Hi Tom,
On Fri, 6 Dec 2013 10:48:50 -0500, Tom Rini trini@ti.com wrote:
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting fdt_high (in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Sure. Lets take am335x_evm builds (so Beaglebone Black/White, etc). If you start enabling all of the tracing options in the kernel (function tracing, graphs, etc), you get an uncompressed kernel and BSS that will use up the first ~16MiB of DDR. We default to placing the DT at about 15MiB into memory. So the kernel runs, clears BSS and eats the DT. System now hangs, and depending on debug options set you may or may not see anything at all from the kernel. U-Boot couldn't detect this failure because we don't know how big the kernel BSS is, only how big the zImage is (and where it is) and how big the fdt is and where it is. No overlaps, go ahead and run.
Thanks.
The only issue I have with the RFC is that the +512 MiB value will only work with targets which have more than 512 MiB DDR, right? But since you're suggesting this should be set in configs, you are only suggesting +512 MiB, and any target could actually specify a lower value as long as it's greater than or eqal to 16 MiB. Correct?
fdt_high is only an upper bound on what we may relocate to (setting aside the magic value of 0xffffffff which means no relocation). I just confirmed this too: U-Boot# bdi ... boot_params = 0x80000100 DRAM bank = 0x00000000 -> start = 0x80000000 -> size = 0x40000000 ... U-Boot# setenv fdt_high 0xe0000000 ... U-Boot# bootz $loadaddr - $fdtaddr Kernel image @ 0x80200000 [ 0x000000 - 0x3e5068 ] ## Flattened Device Tree blob at 80f80000 Booting using the fdt blob at 0x80f80000 Loading Device Tree to bf62a000, end bf630d9a ... OK
Of course, 0xbf62a000 is a bad address in that it won't be seen by the kernel, but it was in the higher-than-we-have-memory-at limit I set.
I haven't really followed this thread since I just noticed it tangentially, but on Tegra...
Since commit 7f1b767aea94 "ARM: tegra: define CONFIG_SYS_BOOTMAPSZ" all Tegra boards define BOOTMAPSZ to influence where the DTB gets relocated. I wonder what's the benefit of using BOOTMAPSZ vs. defining a default value for $fdt_high?
I believe they both get the same thing done but SYS_BOOTMAPSZ also influences relocation of ramdisks (and in these cases we disable ramdisk relocation with initrd_high=0xffffffff).
And couldn't you just move $fdt_addr_r high enough in RAM so even if the DTB wasn't relocated it'd never overlap BSS? Tegra's $kernel_addr_r and $fdt_addr_r are likely set up that way already, although I didn't check.
There's two problems here. The first problem is that we have between 256MiB and 1GiB of DDR on the platform, but we could just design for the smallest case. The second problem is, what's big enough? You've got 32MiB (tegra30) which I would hope is enough (and I suggested as much in Dennis' thread) for kernel + BSS, but how big is a multi platform kernel with a few big features going to get? Or do we say that's an unreasonable out of box use case?
On 12/06/2013 02:04 PM, Tom Rini wrote: ...
There's two problems here. The first problem is that we have between 256MiB and 1GiB of DDR on the platform, but we could just design for the smallest case. The second problem is, what's big enough? You've got 32MiB (tegra30) which I would hope is enough (and I suggested as much in Dennis' thread) for kernel + BSS, but how big is a multi platform kernel with a few big features going to get? Or do we say that's an unreasonable out of box use case?
Is there any limit on .data/.bss size like there is .text (due to the limited range of relative jump encoding), or would data accesses just fall back to a relative load of the absolute address, and hence be unbounded.
FWIW, I see the following sizes currently:
tegra_defconfig
.text 005862c4 .data 0005eb68 .bss 00055bf0
multi_v7_defconfig
.text 004a5560 .data 00092600 .bss 00046014
(I think multi_v7_defconfig doesn't yet have that many drivers enabled, and when it does presumably they'd be modules)
... so BSS isn't terribly large at the moment.
Dear Tom,
In message 20131206154850.GW420@bill-the-cat you wrote:
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting fdt_high (in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Maybe I'm missing something - but what about systems with less than 1 GB RAM? Actually only a small percentage of devices we see here have 512 MB or more...
Best regards,
Wolfgang Denk
participants (4)
-
Albert ARIBAUD -
Stephen Warren -
Tom Rini -
Wolfgang Denk