[PATCH] arm: bootm: Disable LMB reservation for command line and board info on arm64
On arm64, board info is not applicable and kernel command line patched into the DT, so the LMB reservation here makes no sense anymore. On legacy arm32, this might still be necessary on systems which do not use DT or use legacy ATAGS. Disable this LMB reservation on arm64.
This also permits Linux DT to specify reserved memory node at address close to the end of DRAM bank, i.e. overlaping with U-Boot location. Since after boot, U-Boot will be no more, this is OK.
Signed-off-by: Marek Vasut marek.vasut+renesas@gmail.com Cc: Hai Pham hai.pham.ud@renesas.com Cc: Simon Goldschmidt simon.k.r.goldschmidt@gmail.com Cc: Stephen Warren swarren@nvidia.com Cc: Tom Rini trini@konsulko.com --- arch/arm/lib/bootm.c | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c index f60ee3a7e6..23b99a541c 100644 --- a/arch/arm/lib/bootm.c +++ b/arch/arm/lib/bootm.c @@ -43,6 +43,7 @@ DECLARE_GLOBAL_DATA_PTR;
static struct tag *params;
+#ifndef CONFIG_ARM64 static ulong get_sp(void) { ulong ret; @@ -86,6 +87,7 @@ void arch_lmb_reserve(struct lmb *lmb) break; } } +#endif
__weak void board_quiesce_devices(void) {
On Sat, May 29, 2021 at 01:34:32PM +0200, Marek Vasut wrote:
On arm64, board info is not applicable and kernel command line patched into the DT, so the LMB reservation here makes no sense anymore. On legacy arm32, this might still be necessary on systems which do not use DT or use legacy ATAGS. Disable this LMB reservation on arm64.
This also permits Linux DT to specify reserved memory node at address close to the end of DRAM bank, i.e. overlaping with U-Boot location. Since after boot, U-Boot will be no more, this is OK.
Signed-off-by: Marek Vasut marek.vasut+renesas@gmail.com Cc: Hai Pham hai.pham.ud@renesas.com Cc: Simon Goldschmidt simon.k.r.goldschmidt@gmail.com Cc: Stephen Warren swarren@nvidia.com Cc: Tom Rini trini@konsulko.com
Applied to u-boot/master, thanks!
On 29.05.21 13:34, Marek Vasut wrote:
On arm64, board info is not applicable and kernel command line patched into the DT, so the LMB reservation here makes no sense anymore. On legacy arm32, this might still be necessary on systems which do not use DT or use legacy ATAGS. Disable this LMB reservation on arm64.
This also permits Linux DT to specify reserved memory node at address close to the end of DRAM bank, i.e. overlaping with U-Boot location. Since after boot, U-Boot will be no more, this is OK.
Signed-off-by: Marek Vasut marek.vasut+renesas@gmail.com Cc: Hai Pham hai.pham.ud@renesas.com Cc: Simon Goldschmidt simon.k.r.goldschmidt@gmail.com Cc: Stephen Warren swarren@nvidia.com Cc: Tom Rini trini@konsulko.com
arch/arm/lib/bootm.c | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c index f60ee3a7e6..23b99a541c 100644 --- a/arch/arm/lib/bootm.c +++ b/arch/arm/lib/bootm.c @@ -43,6 +43,7 @@ DECLARE_GLOBAL_DATA_PTR;
static struct tag *params;
+#ifndef CONFIG_ARM64 static ulong get_sp(void) { ulong ret; @@ -86,6 +87,7 @@ void arch_lmb_reserve(struct lmb *lmb) break; } } +#endif
__weak void board_quiesce_devices(void) {
This causes troubles for [1], but I have no clue yet, what is happening. Without the patch, we start like this:
Found U-Boot script /boot/boot.scr 889 bytes read in 21 ms (41 KiB/s) ## Executing script at 83000000 Loading /usr/lib/linux-image-4.19.94/ti/k3-am6548-iot2050-advanced.dtb... 78306 bytes read in 25 ms (3 MiB/s) Loading /boot/vmlinux-4.19.94... 13537288 bytes read in 107 ms (120.7 MiB/s) ## Flattened Device Tree blob at 82000000 Booting using the fdt blob at 0x82000000 Loading Device Tree to 00000000fdefa000, end 00000000fdf101e1 ... OK
Starting kernel ...
With the patch applied, I'm getting stuck like this:
Found U-Boot script /boot/boot.scr 889 bytes read in 21 ms (41 KiB/s) ## Executing script at 83000000 Loading /usr/lib/linux-image-4.19.94/ti/k3-am6548-iot2050-advanced.dtb... 78306 bytes read in 25 ms (3 MiB/s) Loading /boot/vmlinux-4.19.94... 13537288 bytes read in 109 ms (118.4 MiB/s) ## Flattened Device Tree blob at 82000000 Booting using the fdt blob at 0x82000000 Loading Device Tree to 00000000fffe9000, end 00000000fffff1e1 ...
Obviously, the DT target adress changed, possibly to an unsupported/reserved address. But I do not understand the mechanics behind all this yet. Any hints welcome on what goes wrong here and whether something needs to be adjusted in our board settings.
Thanks, Jan
[1] https://patchwork.ozlabs.org/project/uboot/list/?series=248449&state=*
On 7/20/21 11:08 AM, Jan Kiszka wrote: [...]
diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c index f60ee3a7e6..23b99a541c 100644 --- a/arch/arm/lib/bootm.c +++ b/arch/arm/lib/bootm.c @@ -43,6 +43,7 @@ DECLARE_GLOBAL_DATA_PTR;
static struct tag *params;
+#ifndef CONFIG_ARM64 static ulong get_sp(void) { ulong ret; @@ -86,6 +87,7 @@ void arch_lmb_reserve(struct lmb *lmb) break; } } +#endif
__weak void board_quiesce_devices(void) {
This causes troubles for [1], but I have no clue yet, what is happening. Without the patch, we start like this:
Found U-Boot script /boot/boot.scr 889 bytes read in 21 ms (41 KiB/s) ## Executing script at 83000000 Loading /usr/lib/linux-image-4.19.94/ti/k3-am6548-iot2050-advanced.dtb... 78306 bytes read in 25 ms (3 MiB/s) Loading /boot/vmlinux-4.19.94... 13537288 bytes read in 107 ms (120.7 MiB/s) ## Flattened Device Tree blob at 82000000 Booting using the fdt blob at 0x82000000 Loading Device Tree to 00000000fdefa000, end 00000000fdf101e1 ... OK
Starting kernel ...
With the patch applied, I'm getting stuck like this:
Found U-Boot script /boot/boot.scr 889 bytes read in 21 ms (41 KiB/s) ## Executing script at 83000000 Loading /usr/lib/linux-image-4.19.94/ti/k3-am6548-iot2050-advanced.dtb... 78306 bytes read in 25 ms (3 MiB/s) Loading /boot/vmlinux-4.19.94... 13537288 bytes read in 109 ms (118.4 MiB/s) ## Flattened Device Tree blob at 82000000 Booting using the fdt blob at 0x82000000 Loading Device Tree to 00000000fffe9000, end 00000000fffff1e1 ...
Obviously, the DT target adress changed, possibly to an unsupported/reserved address. But I do not understand the mechanics behind all this yet. Any hints welcome on what goes wrong here and whether something needs to be adjusted in our board settings.
Can you share the output of bdinfo on this board ?
On 21.07.21 01:34, Marek Vasut wrote:
On 7/20/21 11:08 AM, Jan Kiszka wrote: [...]
diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c index f60ee3a7e6..23b99a541c 100644 --- a/arch/arm/lib/bootm.c +++ b/arch/arm/lib/bootm.c @@ -43,6 +43,7 @@ DECLARE_GLOBAL_DATA_PTR; static struct tag *params; +#ifndef CONFIG_ARM64 static ulong get_sp(void) { ulong ret; @@ -86,6 +87,7 @@ void arch_lmb_reserve(struct lmb *lmb) break; } } +#endif __weak void board_quiesce_devices(void) {
This causes troubles for [1], but I have no clue yet, what is happening. Without the patch, we start like this:
Found U-Boot script /boot/boot.scr 889 bytes read in 21 ms (41 KiB/s) ## Executing script at 83000000 Loading /usr/lib/linux-image-4.19.94/ti/k3-am6548-iot2050-advanced.dtb... 78306 bytes read in 25 ms (3 MiB/s) Loading /boot/vmlinux-4.19.94... 13537288 bytes read in 107 ms (120.7 MiB/s) ## Flattened Device Tree blob at 82000000 Booting using the fdt blob at 0x82000000 Loading Device Tree to 00000000fdefa000, end 00000000fdf101e1 ... OK
Starting kernel ...
With the patch applied, I'm getting stuck like this:
Found U-Boot script /boot/boot.scr 889 bytes read in 21 ms (41 KiB/s) ## Executing script at 83000000 Loading /usr/lib/linux-image-4.19.94/ti/k3-am6548-iot2050-advanced.dtb... 78306 bytes read in 25 ms (3 MiB/s) Loading /boot/vmlinux-4.19.94... 13537288 bytes read in 109 ms (118.4 MiB/s) ## Flattened Device Tree blob at 82000000 Booting using the fdt blob at 0x82000000 Loading Device Tree to 00000000fffe9000, end 00000000fffff1e1 ...
Obviously, the DT target adress changed, possibly to an unsupported/reserved address. But I do not understand the mechanics behind all this yet. Any hints welcome on what goes wrong here and whether something needs to be adjusted in our board settings.
Can you share the output of bdinfo on this board ?
Sure (with your commit reverted for now):
IOT2050> bdinfo boot_params = 0x0000000000000000 DRAM bank = 0x0000000000000000 -> start = 0x0000000080000000 -> size = 0x0000000080000000 flashstart = 0x0000000000000000 flashsize = 0x0000000000000000 flashoffset = 0x0000000000000000 baudrate = 115200 bps relocaddr = 0x00000000fff2d000 reloc off = 0x000000007f72d000 Build = 64-bit current eth = unknown ethaddr = 70:77:77:77:57:70 IP addr = <NULL> fdt_blob = 0x00000000fdef9f10 new_fdt = 0x00000000fdef9f10 fdt_size = 0x0000000000012ea0 lmb_dump_all: memory.cnt = 0x1 memory[0] [0x80000000-0xffffffff], 0x80000000 bytes flags: 0 reserved.cnt = 0x2 reserved[0] [0x9e800000-0xa21fffff], 0x03a00000 bytes flags: 4 reserved[1] [0xfdef86c0-0xffffffff], 0x02107940 bytes flags: 0 arch_number = 0x0000000000000000 TLB addr = 0x00000000ffff0000 irq_sp = 0x00000000fdef9b20 sp start = 0x00000000fdef9b20 Early malloc usage: 23c8 / 8000
There is some "TLB" block apparently overlapping when we move the DT. What's this? Looking at the code, not a TLB with the usual meaning but rather the page table U-Boot uses for itself, right?
Were we just lucky so far with side effects of the LMB reservation on this platform (and I suspect that it affect k3 in general, not only our board)?
Thanks, Jan
On 21.07.21 08:08, Jan Kiszka wrote:
On 21.07.21 01:34, Marek Vasut wrote:
On 7/20/21 11:08 AM, Jan Kiszka wrote: [...]
diff --git a/arch/arm/lib/bootm.c b/arch/arm/lib/bootm.c index f60ee3a7e6..23b99a541c 100644 --- a/arch/arm/lib/bootm.c +++ b/arch/arm/lib/bootm.c @@ -43,6 +43,7 @@ DECLARE_GLOBAL_DATA_PTR; static struct tag *params; +#ifndef CONFIG_ARM64 static ulong get_sp(void) { ulong ret; @@ -86,6 +87,7 @@ void arch_lmb_reserve(struct lmb *lmb) break; } } +#endif __weak void board_quiesce_devices(void) {
This causes troubles for [1], but I have no clue yet, what is happening. Without the patch, we start like this:
Found U-Boot script /boot/boot.scr 889 bytes read in 21 ms (41 KiB/s) ## Executing script at 83000000 Loading /usr/lib/linux-image-4.19.94/ti/k3-am6548-iot2050-advanced.dtb... 78306 bytes read in 25 ms (3 MiB/s) Loading /boot/vmlinux-4.19.94... 13537288 bytes read in 107 ms (120.7 MiB/s) ## Flattened Device Tree blob at 82000000 Booting using the fdt blob at 0x82000000 Loading Device Tree to 00000000fdefa000, end 00000000fdf101e1 ... OK
Starting kernel ...
With the patch applied, I'm getting stuck like this:
Found U-Boot script /boot/boot.scr 889 bytes read in 21 ms (41 KiB/s) ## Executing script at 83000000 Loading /usr/lib/linux-image-4.19.94/ti/k3-am6548-iot2050-advanced.dtb... 78306 bytes read in 25 ms (3 MiB/s) Loading /boot/vmlinux-4.19.94... 13537288 bytes read in 109 ms (118.4 MiB/s) ## Flattened Device Tree blob at 82000000 Booting using the fdt blob at 0x82000000 Loading Device Tree to 00000000fffe9000, end 00000000fffff1e1 ...
Obviously, the DT target adress changed, possibly to an unsupported/reserved address. But I do not understand the mechanics behind all this yet. Any hints welcome on what goes wrong here and whether something needs to be adjusted in our board settings.
Can you share the output of bdinfo on this board ?
Sure (with your commit reverted for now):
IOT2050> bdinfo boot_params = 0x0000000000000000 DRAM bank = 0x0000000000000000 -> start = 0x0000000080000000 -> size = 0x0000000080000000 flashstart = 0x0000000000000000 flashsize = 0x0000000000000000 flashoffset = 0x0000000000000000 baudrate = 115200 bps relocaddr = 0x00000000fff2d000 reloc off = 0x000000007f72d000 Build = 64-bit current eth = unknown ethaddr = 70:77:77:77:57:70 IP addr = <NULL> fdt_blob = 0x00000000fdef9f10 new_fdt = 0x00000000fdef9f10 fdt_size = 0x0000000000012ea0 lmb_dump_all: memory.cnt = 0x1 memory[0] [0x80000000-0xffffffff], 0x80000000 bytes flags: 0 reserved.cnt = 0x2 reserved[0] [0x9e800000-0xa21fffff], 0x03a00000 bytes flags: 4 reserved[1] [0xfdef86c0-0xffffffff], 0x02107940 bytes flags: 0 arch_number = 0x0000000000000000 TLB addr = 0x00000000ffff0000 irq_sp = 0x00000000fdef9b20 sp start = 0x00000000fdef9b20 Early malloc usage: 23c8 / 8000
There is some "TLB" block apparently overlapping when we move the DT. What's this? Looking at the code, not a TLB with the usual meaning but rather the page table U-Boot uses for itself, right?
Were we just lucky so far with side effects of the LMB reservation on this platform (and I suspect that it affect k3 in general, not only our board)?
Just rebased over master, kept Marek's change, and rerun bdinfo:
IOT2050> bdinfo boot_params = 0x0000000000000000 DRAM bank = 0x0000000000000000 -> start = 0x0000000080000000 -> size = 0x0000000040000000 flashstart = 0x0000000000000000 flashsize = 0x0000000000000000 flashoffset = 0x0000000000000000 baudrate = 115200 bps relocaddr = 0x00000000bff41000 reloc off = 0x000000003f741000 Build = 64-bit fdt_blob = 0x00000000bdf0d750 new_fdt = 0x00000000bdf0d750 fdt_size = 0x0000000000013660 lmb_dump_all: memory.cnt = 0x1 memory[0] [0x80000000-0xbfffffff], 0x40000000 bytes flags: 0 reserved.cnt = 0x1 reserved[0] [0x9e800000-0xa21fffff], 0x03a00000 bytes flags: 4 arch_number = 0x0000000000000000 TLB addr = 0x00000000bfff0000 irq_sp = 0x00000000bdf0d360 sp start = 0x00000000bdf0d360 Early malloc usage: 23c8 / 8000 IOT2050> boot switch to partitions #0, OK mmc0 is current device Scanning mmc 0:1... Found U-Boot script /boot/boot.scr 911 bytes read in 28 ms (31.3 KiB/s) ## Executing script at 83000000 Loading /usr/lib/linux-image-4.19.94/ti/k3-am6528-iot2050-basic.dtb... 77409 bytes read in 34 ms (2.2 MiB/s) Loading /boot/vmlinux-4.19.94... 13537288 bytes read in 587 ms (22 MiB/s) ## Flattened Device Tree blob at 82000000 Booting using the fdt blob at 0x82000000 Loading Device Tree to 00000000bffea000, end 00000000bffffe60 ...
From that, I would definitely say that the fdt is loaded into the TLB
area, and that is likely the page table still in use at this stage.
Lokesh, Marek is overly busy and asked me to debug this further. But I would need some indication if my interpretations are right and some hint what is supposed to protect the TLB range from being used by other things. And if that protection would be generic or board-specific.
I've checked the AM65x-SR1.0 EVM I have here, and I can see there as well that mmu_setup() of U-Boot proper works with the same address range the that boot_relocate_fdt() will later overwrite. It looks to me like TLBs (the actual ones) are protecting us in some cases from the consequences, at least on the EVM, not on our boards. Soon after the overwriting, dcache and mmu will be turned off anyway for Linux.
Thanks, Jan
participants (4)
-
Jan Kiszka -
Marek Vasut -
Marek Vasut -
Tom Rini