Hello everyone,
I'm working on porting a mainline U-Boot on a custom board based on a SPEAr600. Right now, I'm facing an issue related with fitImage booting.
Before starting on the fitImage issue, let me state that the zImage and the custom DTB concatenated in a uImage have the load address and entry point be respectively 0x0200.0000 and 0x0200.0040. The fitImage load and entry are both 0x0200.0000. The uImage loaded at the address 0x0080.0000 is booted fine by U-Boot, the fitImage at the same address, not.
The datasheet of the SoC states that the external SDRAM is located between 0x0000.0000 and 0x3fff.ffff. The size of the fitImage is currently 0x0038.9484 bytes, the uImage 0x0038.9007.
The fitImage booting abruptly stops right after "Loading Device Tree to 0778a000, end 0778e7d6 ... OK" with the following trace: "prefetch abort pc : [<f188af44>] lr : [<07fca8c7>] reloc pc : [<ed7f8f44>] lr : [<03f388c7>] sp : 0778fb78 ip : fffc4c98 fp : 03f00020 r10: deadbeef r9 : 0778ff00 r8 : 07f922a0 r7 : 00ff0000 r6 : 000047d7 r5 : 00004700 r4 : 0778a1b8 r3 : 00000008 r2 : 000016c4 r1 : 0778a1b8 r0 : 0778a1b8 Flags: nZCv IRQs off FIQs off Mode SVC_32 Code: data abort pc : [<07f931c6>] lr : [<07fb2dd1>] reloc pc : [<03f011c6>] lr : [<03f20dd1>] sp : 0778fa80 ip : 00000114 fp : 03f00020 r10: deadbeef r9 : 0778ff00 r8 : 07f922a0 r7 : 60000000 r6 : 00000698 r5 : f188af44 r4 : fffffffc r3 : fffffff0 r2 : 0778ff00 r1 : 00000020 r0 : 00000006 Flags: NzCv IRQs on FIQs on Mode SVC_32 Code: 23036be5 439d4818 f885f038 42642404 (595a00a3) Resetting CPU ...
resetting ... System is going to reboot ..."
I've narrowed down the issue to a memmove being called with destination pointer being the source pointer. If I print a debug message before and after this memcpy[1], the first debug message is printed but not the second one. If I add a condition for (dest == src) return dest in the very beginning of memmove, the fitImage boots just fine. This test seemed to have been removed by commit cb0eae8cf8aaca76910dee4c7eb536d0814d1bd2[12], and by reverting this commit, I was able to successfuly boot Linux with a fitImage.
Here is the log without debug messages WITHOUT the dest == src check in memmove:
U-Boot 2018.11-00007-g0b16f8424a-dirty (Dec 05 2018 - 09:23:31 +0100)-SPEAr
CPU: SPEAr600 DRAM: 128 MiB Flash: 512 KiB NAND: 128 MiB Loading Environment from Flash... OK Net: dwmac.e0800000 Hit SPACE in 3 seconds to stop autoboot. => dhcp; tftp fitImage; bootm Speed: 100, full duplex BOOTP broadcast 1 DHCP client bound to address 192.168.0.169 (3 ms) Speed: 100, full duplex Using dwmac.e0800000 device TFTP from server 192.168.0.13; our IP address is 192.168.0.169 Filename 'fitImage'. Load address: 0x800000 Loading: ################################################################# ################################################################# ################################################################# ########################################################## 5.5 MiB/s done Bytes transferred = 3708036 (389484 hex) ## Loading kernel from FIT Image at 00800000 ... Using 'conf@default' configuration Trying 'kernel@0' kernel subimage Description: Linux Type: Kernel Image Compression: uncompressed Data Start: 0x008000b0 Data Size: 3700720 Bytes = 3.5 MiB Architecture: ARM OS: Linux Load Address: 0x02000000 Entry Point: 0x02000000 Hash algo: sha1 Hash value: efe249f573647bad3ce87c9c4b244986a90234db ## Loading fdt from FIT Image at 00800000 ... Using 'conf@default' configuration Trying 'fdt@0' fdt subimage Description: Device Tree Type: Flat Device Tree Compression: uncompressed Data Start: 0x00b87984 Data Size: 6103 Bytes = 6 KiB Architecture: ARM Hash algo: sha1 Hash value: 53089fa031e727f094e6bf9ad4e93f4e95b7fba3 Booting using the fdt blob at 0xb87984 Loading Kernel Image ... OK Loading Device Tree to 0778a000, end 0778e7d6 ... OK prefetch abort pc : [<f188af44>] lr : [<07fca8c7>] reloc pc : [<ed7f8f44>] lr : [<03f388c7>] sp : 0778fb78 ip : fffc4c98 fp : 03f00020 r10: deadbeef r9 : 0778ff00 r8 : 07f922a0 r7 : 00ff0000 r6 : 000047d7 r5 : 00004700 r4 : 0778a1b8 r3 : 00000008 r2 : 000016c4 r1 : 0778a1b8 r0 : 0778a1b8 Flags: nZCv IRQs off FIQs off Mode SVC_32 Code: data abort pc : [<07f931c6>] lr : [<07fb2dd1>] reloc pc : [<03f011c6>] lr : [<03f20dd1>] sp : 0778fa80 ip : 00000114 fp : 03f00020 r10: deadbeef r9 : 0778ff00 r8 : 07f922a0 r7 : 60000000 r6 : 00000698 r5 : f188af44 r4 : fffffffc r3 : fffffff0 r2 : 0778ff00 r1 : 00000020 r0 : 00000006 Flags: NzCv IRQs on FIQs on Mode SVC_32 Code: 23036be5 439d4818 f885f038 42642404 (595a00a3) Resetting CPU ...
resetting ... System is going to reboot ...
With debug messages (_DEBUG = 1 and a few printf("%s: %d\n", __func__, __LINE__);):
=> dhcp; tftp fitImage; bootm Trying dwmac.e0800000 Speed: 100, full duplex BOOTP broadcast 1 DHCPHandler: got packet: (src=67, dst=68, len=300) state: 3 Filtering pkt = 0 DHCPHandler: got DHCP packet: (src=67, dst=68, len=300) state: 3 DHCP: state=SELECTING bp_file: "" TRANSITIONING TO REQUESTING STATE dhcp_send_request_packet: Sending DHCPREQUEST Transmitting DHCPREQUEST packet: len = 342 DHCPHandler: got packet: (src=67, dst=68, len=300) state: 4 Filtering pkt = 0 DHCPHandler: got DHCP packet: (src=67, dst=68, len=300) state: 4 DHCP State: REQUESTING net_boot_file_name: DHCP client bound to address 192.168.0.169 (21 ms) Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 Trying dwmac.e0800000 Speed: 100, full duplex TFTP blocksize = 1468, timeout = 5000 ms Using dwmac.e0800000 device TFTP from server 192.168.0.13; our IP address is 192.168.0.169 Filename 'fitImage'. Load address: 0x800000 Loading: send option "timeout 5" Got OACK: timeout 5 Blocksize ack: 1468, 1468 ################################################################# ################################################################# ################################################################# ########################################################## 5.3 MiB/s done Bytes transferred = 3708036 (389484 hex) Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 ## Current stack ends at 0x0778dd04 * kernel: default image load address = 0x00800000 ## Loading kernel from FIT Image at 00800000 ... No configuration specified, trying default... Found default configuration: 'conf@default' Using 'conf@default' configuration Trying 'kernel@0' kernel subimage Description: Linux Type: Kernel Image Compression: uncompressed Data Start: 0x008000b0 Data Size: 3700720 Bytes = 3.5 MiB Architecture: ARM OS: Linux Load Address: 0x02000000 Entry Point: 0x02000000 Hash node: 'hash@0' Hash algo: sha1 Hash value: efe249f573647bad3ce87c9c4b244986a90234db Hash len: 20 kernel data at 0x008000b0, len = 0x003877f0 (3700720) * ramdisk: using config 'conf@default' from image at 0x00800000 * ramdisk: no 'ramdisk' in config * fdt: using config 'conf@default' from image at 0x00800000 ## Checking for 'FDT'/'FDT Image' at 00800000 ## Loading fdt from FIT Image at 00800000 ... Using 'conf@default' configuration Trying 'fdt@0' fdt subimage Description: Device Tree Type: Flat Device Tree Compression: uncompressed Data Start: 0x00b87984 Data Size: 6103 Bytes = 6 KiB Architecture: ARM Can't get 'load' property from FIT 0x00800000, node: offset 3701020, name fdt@0 (FDT_ERR_NOTFOUND) Hash node: 'hash@0' Hash algo: sha1 Hash value: 53089fa031e727f094e6bf9ad4e93f4e95b7fba3 Hash len: 20 Can't get 'load' property from FIT 0x00800000, node: offset 3701020, name fdt@0 (FDT_ERR_NOTFOUND) fit_uname=fdt@0, fit_uname_config=conf@default Booting using the fdt blob at 0xb87984 of_flat_tree at 0x00b87984 size 0x000017d7 Initial value for argc=3 Final value for argc=3 Loading Kernel Image ... OK kernel loaded at 0x02000000, end = 0x023877f0 ## initrd_high = 0x08000000, copy_to_ram = 1 ramdisk load start = 0x00000000, ramdisk load end = 0x00000000 Initial value for argc=3 Final value for argc=3 Initial value for argc=3 Final value for argc=3 using: FDT ## device tree at 00b87984 ... 00b8915a (len=18391 [0x47D7]) Loading Device Tree to 07788000, end 0778c7d6 ... OK Initial value for argc=3 Final value for argc=3 image_setup_linux: 1511 fdt_setprop: 300 fdt_setprop_placeholder: 283 fdt_splice_: 108 fdt_splice_struct_: 132 fdt_splice_: 108 image_setup_libfdt: 480 arch_fixup_fdt: 54 fdt_fixup_memory_banks: 440 fdt_setprop: 300 fdt_setprop_placeholder: 283 fdt_resize_property_: 215 fdt_splice_struct_: 132 fdt_splice_: 108 memmove: 547 dest=0x77881b8 src=0x77881b8 size=0x16c4 undefined instruction pc : [<07fc9534>] lr : [<07fc9537>] reloc pc : [<03f39534>] lr : [<03f39537>] sp : 0778db58 ip : fffc3f40 fp : 03f00020 r10: deadbeef r9 : 0778df00 r8 : 07f902a0 r7 : 00ff0000 r6 : 077881b8 r5 : 077881b8 r4 : 000016c4 r3 : 07fc952d r2 : 000016c4 r1 : 077881b8 r0 : 077881b8 Flags: nZCv IRQs off FIQs off Mode SVC_32 Code: 48094908 fd17f000 00310022 f0030028 (0028faf1) Resetting CPU ...
resetting ... System is going to reboot ...
Basically, the error path is: image_setup_linux[2]->image_setup_libfdt[3]->arch_fixup_fdt[4]-> fdt_fixup_memory_banks[5]->fdt_setprop[6]->fdt_setprop_placeholder[7]-> fdt_resize_property_[8]->fdt_splice_struct_[9]->fdt_splice_[10]-> memmove[11]->memcpy
The bootloader build is marked as dirty but I'm only adding files (board C and header files, board defconfig, etc...) for board support, no modifications otherwise, it's based on 2018.11.
It's weird to me that it's happening with this SoC because it's based on ARM926ejs which is widely used I assume. Shouldn't have anyone already encountered the bug? Or is nobody actually booting a fitImage and had the luck to never call memcpy with src == dest anywhere else in the code?
Let me know if I can be of any help for debugging.
Thanks, Quentin
[1] https://elixir.bootlin.com/u-boot/latest/source/lib/string.c#L547 [2] https://elixir.bootlin.com/u-boot/latest/source/common/image.c#L1511 [3] https://elixir.bootlin.com/u-boot/latest/source/common/image-fdt.c#L480 [4] https://elixir.bootlin.com/u-boot/latest/source/arch/arm/lib/bootm-fdt.c#L54 [5] https://elixir.bootlin.com/u-boot/latest/source/common/fdt_support.c#L440 [6] https://elixir.bootlin.com/u-boot/latest/source/scripts/dtc/libfdt/fdt_rw.c#... [7] https://elixir.bootlin.com/u-boot/latest/source/scripts/dtc/libfdt/fdt_rw.c#... [8] https://elixir.bootlin.com/u-boot/latest/source/scripts/dtc/libfdt/fdt_rw.c#... [9] https://elixir.bootlin.com/u-boot/latest/source/scripts/dtc/libfdt/fdt_rw.c#... [10] https://elixir.bootlin.com/u-boot/latest/source/scripts/dtc/libfdt/fdt_rw.c#... [11] https://elixir.bootlin.com/u-boot/latest/source/lib/string.c#L547 [12] http://git.denx.de/?p=u-boot.git;a=commit;h=cb0eae8cf8aaca76910dee4c7eb536d0...