[U-Boot] ROCK64 fails to boot using U-Boot TPL
I've been unable so far to boot NetBSD on my PINE64 ROCK64 (v2.0, Rockchip RK3328) using U-Boot built with its own TPL, the default since commit ff3dd0a474.
The same NetBSD image boots fine using the binary TPL supplied by Rockchip.
The exact failure varies but it always seems memory-related. Typically the NetBSD kernel fails right after starting with a panic in its virtual-memory system, uvm.
Also significant may be these messages sometimes output by U-Boot at startup, which I haven't seen when the Rockchip TPL is used:
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
I've pasted the output from a typical failed boot below and have submitted a problem report (with a bit more detail) to the NetBSD project:
https://gnats.netbsd.org/54557
Does anyone know why this might be happening? Is there perhaps some additional setup the U-Boot TPL expects from the OS to finish configuring the RK3328's memory controller that's currently missing from NetBSD?
How I can diagnose this further?
----------
U-Boot TPL 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:11:29) LPDDR3 Trying to boot from BOOTROM Returning to boot ROM...
U-Boot SPL 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:11:29 -0400) Trying to boot from MMC2 Card did not respond to voltage select! spl: mmc init failed with error: -95 Trying to boot from MMC1 NOTICE: BL31: v2.1(release):v2.1-678-g2fc6ffc4-dirty NOTICE: BL31: Built : 12:10:07, Sep 12 2019 ERROR: over or zero region, nr=4187432, max=10 NOTICE: BL31:Rockchip release version: v1.2
U-Boot 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:12:31 -0400)
Model: Pine64 Rock64 DRAM: 4 GiB MMC: rksdmmc@ff500000: 1, rksdmmc@ff520000: 0 Loading Environment from MMC... *** Warning - bad CRC, using default environment
In: serial@ff130000 Out: serial@ff130000 Err: serial@ff130000 Model: Pine64 Rock64 Net: Warning: ethernet@ff540000 (eth0) using random MAC address - da:e0:78:b1:50:d2 eth0: ethernet@ff540000 Hit any key to stop autoboot: 0 Card did not respond to voltage select! switch to partitions #0, OK mmc1 is current device Scanning mmc 1:1... 53513 bytes read in 9 ms (5.7 MiB/s) Found EFI removable media binary efi/boot/bootaa64.efi libfdt fdt_check_header(): FDT_ERR_BADMAGIC Scanning disk rksdmmc@ff500000.blk... Card did not respond to voltage select! Scanning disk rksdmmc@ff520000.blk... Disk rksdmmc@ff520000.blk not ready Found 3 disks BootOrder not defined EFI boot manager: Cannot load any image 205136 bytes read in 16 ms (12.2 MiB/s) libfdt fdt_check_header(): FDT_ERR_BADMAGIC -
NetBSD/evbarm EFI Boot (aarch64), Revision 1.11 (Wed Sep 18 15:50:45
UTC 2019) (from NetBSD 9.99.12) Press return to boot now, any other key for boot prompt booting netbsd - starting in 0 seconds. 6100056+2730512+1985652+1823764+490622=0xec5b00 [ 1.0000000] NetBSD/evbarm (fdt) booting ... [ 1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, [ 1.0000000] 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, [ 1.0000000] 2018, 2019 The NetBSD Foundation, Inc. All rights reserved. [ 1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993 [ 1.0000000] The Regents of the University of California. All rights reserved.
[ 1.0000000] NetBSD 9.99.12 (GENERIC64) #16: Wed Sep 18 12:30:03 EDT 2019 [ 1.0000000] ssouth@laptop:/usr/src/sys/arch/evbarm/compile/obj/GENERIC64 [ 1.0000000] total memory = 4064 MB [ 1.0000000] avail memory = 3918 MB [ 1.0000000] panic: kernel diagnostic assertion "ucpu == VM_FREE_PAGE_TO_CPU(pg)" failed: file "/usr/src/sys/uvm/uvm_page.c", line 852 [ 1.0000000] cpu0: Begin traceback... [ 1.0000000] trace fp ffffffc000bec8d0 [ 1.0000000] fp ffffffc000bec8f0 vpanic() at ffffffc0004924c0 netbsd:vpanic+0x198 [ 1.0000000] fp ffffffc000bec950 kern_assert() at ffffffc0005cdf24 netbsd:kern_assert+0x5c [ 1.0000000] fp ffffffc000bec9e0 uvm_pagealloc_pgfl() at ffffffc00040f388 netbsd:uvm_pagealloc_pgfl+0x5c8 [ 1.0000000] fp ffffffc000beca40 uvm_pagealloc_strat() at ffffffc000410034 netbsd:uvm_pagealloc_strat+0x18c [ 1.0000000] fp ffffffc000becad0 uvm_km_kmem_alloc() at ffffffc000403b30 netbsd:uvm_km_kmem_alloc+0x60 [ 1.0000000] fp ffffffc000becb40 pool_page_alloc() at ffffffc00048b0c0 netbsd:pool_page_alloc+0x28 [ 1.0000000] fp ffffffc000becb60 pool_grow() at ffffffc00048d294 netbsd:pool_grow+0x35c [ 1.0000000] fp ffffffc000becbd0 pool_get() at ffffffc00048c5e8 netbsd:pool_get+0xa0 [ 1.0000000] fp ffffffc000becc30 pool_cache_get_slow() at ffffffc00048e8a4 netbsd:pool_cache_get_slow+0x1dc [ 1.0000000] fp ffffffc000becc80 pool_cache_get_paddr() at ffffffc0004901e8 netbsd:pool_cache_get_paddr+0x258 [ 1.0000000] fp ffffffc000becce0 kmem_intr_alloc() at ffffffc0004859f0 netbsd:kmem_intr_alloc+0x60 [ 1.0000000] fp ffffffc000becd10 kmem_intr_zalloc() at ffffffc000485b20 netbsd:kmem_intr_zalloc+0x10 [ 1.0000000] fp ffffffc000becd30 kmem_zalloc() at ffffffc000485dc4 netbsd:kmem_zalloc+0x84 [ 1.0000000] fp ffffffc000becd50 module_builtin_add() at ffffffc0004432a0 netbsd:module_builtin_add+0x98 [ 1.0000000] fp ffffffc000becdf0 module_init() at ffffffc0004437f8 netbsd:module_init+0xf0 [ 1.0000000] fp ffffffc000bece50 main() at ffffffc0005d0b08 netbsd:main+0x108 [ 1.0000000] fp 0000000000000000 aarch64_start() at ffffffc00000183c netbsd:aarch64_start+0x103c [ 1.0000000] cpu0: End traceback... Stopped in pid 0.1 (system) at netbsd:cpu_Debugger+0x4: ret db{0}>
19.09.2019 11:56, Simon South пишет:
I've been unable so far to boot NetBSD on my PINE64 ROCK64 (v2.0, Rockchip RK3328) using U-Boot built with its own TPL, the default since commit ff3dd0a474.
The same NetBSD image boots fine using the binary TPL supplied by Rockchip.
The exact failure varies but it always seems memory-related. Typically the NetBSD kernel fails right after starting with a panic in its virtual-memory system, uvm.
Also significant may be these messages sometimes output by U-Boot at startup, which I haven't seen when the Rockchip TPL is used:
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
I've pasted the output from a typical failed boot below and have submitted a problem report (with a bit more detail) to the NetBSD project:
https://gnats.netbsd.org/54557
Does anyone know why this might be happening? Is there perhaps some additional setup the U-Boot TPL expects from the OS to finish configuring the RK3328's memory controller that's currently missing from NetBSD?
How I can diagnose this further?
Hi,
I have no idea what is going on. But are you sure that you tested the same u-boot binaries with or without binary TPL? u-boot-tpl.bin and rk3328_ddr_333MHz_v1.16.bin should be fully interchangeable while deploying the bootloader.
U-Boot TPL 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:11:29) LPDDR3 Trying to boot from BOOTROM Returning to boot ROM...
U-Boot SPL 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:11:29 -0400) Trying to boot from MMC2 Card did not respond to voltage select! spl: mmc init failed with error: -95 Trying to boot from MMC1 NOTICE: BL31: v2.1(release):v2.1-678-g2fc6ffc4-dirty NOTICE: BL31: Built : 12:10:07, Sep 12 2019 ERROR: over or zero region, nr=4187432, max=10 NOTICE: BL31:Rockchip release version: v1.2
U-Boot 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:12:31 -0400)
Model: Pine64 Rock64 DRAM: 4 GiB MMC: rksdmmc@ff500000: 1, rksdmmc@ff520000: 0 Loading Environment from MMC... *** Warning - bad CRC, using default environment
In: serial@ff130000 Out: serial@ff130000 Err: serial@ff130000 Model: Pine64 Rock64 Net: Warning: ethernet@ff540000 (eth0) using random MAC address - da:e0:78:b1:50:d2 eth0: ethernet@ff540000 Hit any key to stop autoboot: 0 Card did not respond to voltage select! switch to partitions #0, OK mmc1 is current device Scanning mmc 1:1... 53513 bytes read in 9 ms (5.7 MiB/s) Found EFI removable media binary efi/boot/bootaa64.efi libfdt fdt_check_header(): FDT_ERR_BADMAGIC Scanning disk rksdmmc@ff500000.blk... Card did not respond to voltage select! Scanning disk rksdmmc@ff520000.blk... Disk rksdmmc@ff520000.blk not ready Found 3 disks BootOrder not defined EFI boot manager: Cannot load any image 205136 bytes read in 16 ms (12.2 MiB/s) libfdt fdt_check_header(): FDT_ERR_BADMAGIC
NetBSD/evbarm EFI Boot (aarch64), Revision 1.11 (Wed Sep 18 15:50:45
UTC 2019) (from NetBSD 9.99.12) Press return to boot now, any other key for boot prompt booting netbsd - starting in 0 seconds. 6100056+2730512+1985652+1823764+490622=0xec5b00 [ 1.0000000] NetBSD/evbarm (fdt) booting ... [ 1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, [ 1.0000000] 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, [ 1.0000000] 2018, 2019 The NetBSD Foundation, Inc. All rights reserved. [ 1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993 [ 1.0000000] The Regents of the University of California. All rights reserved.
[ 1.0000000] NetBSD 9.99.12 (GENERIC64) #16: Wed Sep 18 12:30:03 EDT 2019 [ 1.0000000] ssouth@laptop:/usr/src/sys/arch/evbarm/compile/obj/GENERIC64 [ 1.0000000] total memory = 4064 MB [ 1.0000000] avail memory = 3918 MB [ 1.0000000] panic: kernel diagnostic assertion "ucpu == VM_FREE_PAGE_TO_CPU(pg)" failed: file "/usr/src/sys/uvm/uvm_page.c", line 852 [ 1.0000000] cpu0: Begin traceback... [ 1.0000000] trace fp ffffffc000bec8d0 [ 1.0000000] fp ffffffc000bec8f0 vpanic() at ffffffc0004924c0 netbsd:vpanic+0x198 [ 1.0000000] fp ffffffc000bec950 kern_assert() at ffffffc0005cdf24 netbsd:kern_assert+0x5c [ 1.0000000] fp ffffffc000bec9e0 uvm_pagealloc_pgfl() at ffffffc00040f388 netbsd:uvm_pagealloc_pgfl+0x5c8 [ 1.0000000] fp ffffffc000beca40 uvm_pagealloc_strat() at ffffffc000410034 netbsd:uvm_pagealloc_strat+0x18c [ 1.0000000] fp ffffffc000becad0 uvm_km_kmem_alloc() at ffffffc000403b30 netbsd:uvm_km_kmem_alloc+0x60 [ 1.0000000] fp ffffffc000becb40 pool_page_alloc() at ffffffc00048b0c0 netbsd:pool_page_alloc+0x28 [ 1.0000000] fp ffffffc000becb60 pool_grow() at ffffffc00048d294 netbsd:pool_grow+0x35c [ 1.0000000] fp ffffffc000becbd0 pool_get() at ffffffc00048c5e8 netbsd:pool_get+0xa0 [ 1.0000000] fp ffffffc000becc30 pool_cache_get_slow() at ffffffc00048e8a4 netbsd:pool_cache_get_slow+0x1dc [ 1.0000000] fp ffffffc000becc80 pool_cache_get_paddr() at ffffffc0004901e8 netbsd:pool_cache_get_paddr+0x258 [ 1.0000000] fp ffffffc000becce0 kmem_intr_alloc() at ffffffc0004859f0 netbsd:kmem_intr_alloc+0x60 [ 1.0000000] fp ffffffc000becd10 kmem_intr_zalloc() at ffffffc000485b20 netbsd:kmem_intr_zalloc+0x10 [ 1.0000000] fp ffffffc000becd30 kmem_zalloc() at ffffffc000485dc4 netbsd:kmem_zalloc+0x84 [ 1.0000000] fp ffffffc000becd50 module_builtin_add() at ffffffc0004432a0 netbsd:module_builtin_add+0x98 [ 1.0000000] fp ffffffc000becdf0 module_init() at ffffffc0004437f8 netbsd:module_init+0xf0 [ 1.0000000] fp ffffffc000bece50 main() at ffffffc0005d0b08 netbsd:main+0x108 [ 1.0000000] fp 0000000000000000 aarch64_start() at ffffffc00000183c netbsd:aarch64_start+0x103c [ 1.0000000] cpu0: End traceback... Stopped in pid 0.1 (system) at netbsd:cpu_Debugger+0x4: ret db{0}>
On 2019-09-19 11:48 a.m., Matwey V. Kornilov wrote:
I have no idea what is going on. But are you sure that you tested the same u-boot binaries with or without binary TPL?
I believe so. Here's how I build U-Boot with its own TPL:
make -j2 distclean; rm idbloader.img make -j2 rock64-rk3328_defconfig all u-boot.itb
With the Rockchip TPL instead, it's:
make -j2 distclean; rm idbloader.img make -j2 rock64-rk3328_defconfig all u-boot.itb tools/mkimage -n rk3328 -T rksd -d ~/src/rockchip-linux/rkbin/bin/rk33/rk3328_ddr_333MHz_v1.16.bin idbloader.img cat spl/u-boot-spl.bin >> idbloader.img
In either case I install on the microSD card with
sudo dd if=idbloader.img of=/dev/sdb seek=64 sudo dd if=u-boot.itb of=/dev/sdb seek=16384
Does this look right?
20.09.2019 3:06, Simon South пишет:
On 2019-09-19 11:48 a.m., Matwey V. Kornilov wrote:
I have no idea what is going on. But are you sure that you tested the same u-boot binaries with or without binary TPL?
I believe so. Here's how I build U-Boot with its own TPL:
make -j2 distclean; rm idbloader.img make -j2 rock64-rk3328_defconfig all u-boot.itb
With the Rockchip TPL instead, it's:
make -j2 distclean; rm idbloader.img make -j2 rock64-rk3328_defconfig all u-boot.itb tools/mkimage -n rk3328 -T rksd -d ~/src/rockchip-linux/rkbin/bin/rk33/rk3328_ddr_333MHz_v1.16.bin idbloader.img cat spl/u-boot-spl.bin >> idbloader.img
In either case I install on the microSD card with
sudo dd if=idbloader.img of=/dev/sdb seek=64 sudo dd if=u-boot.itb of=/dev/sdb seek=16384
Does this look right?
Yes. I think so.
Could you please provide us console logs for both successful and unsuccessful cases? Maybe one could see some meaning difference.
On 2019-09-20 3:39 a.m., Matwey V. Kornilov wrote:
Could you please provide us console logs for both successful and unsuccessful cases?
Attached.
The only differences I see are
- The random Ethernet address assigned in each case, and - The "FDT_ERR_BADMAGIC" error messages that are output only when the U-Boot TPL is used.
пт, 20 сент. 2019 г. в 12:30, Simon South simon@simonsouth.net:
On 2019-09-20 3:39 a.m., Matwey V. Kornilov wrote:
Could you please provide us console logs for both successful and unsuccessful cases?
Attached.
The only differences I see are
- The random Ethernet address assigned in each case, and
- The "FDT_ERR_BADMAGIC" error messages that are output only when the
U-Boot TPL is used.
Missed correct FDT file may cause booting errors like your. At least in Linux kernel world. I am not an NetBSD expert. The question is that FDT is not directly related to TPL in any way.
Do you have an idea where is correct FDT supposed to came from in NetBSD case? For Linux, the file is placed on the boot partition and retrieved by u-boot while looking for EFI application, etc. Another backup FDT is bundled with the u-boot image itself, however it never worked for me for booting Linux.
-- Simon South simon@simonsouth.net
On Fri, Sep 20, 2019 at 01:07:12PM +0300, Matwey V. Kornilov wrote:
Do you have an idea where is correct FDT supposed to came from in NetBSD case? For Linux, the file is placed on the boot partition and retrieved by u-boot while looking for EFI application, etc.
Same for NetBSD. Assuming that other parts have not been changed, Simon's experiments should have used the same .dtb for all tests.
Martin
пт, 20 сент. 2019 г. в 13:14, Martin Husemann martin@duskware.de:
On Fri, Sep 20, 2019 at 01:07:12PM +0300, Matwey V. Kornilov wrote:
Do you have an idea where is correct FDT supposed to came from in NetBSD case? For Linux, the file is placed on the boot partition and retrieved by u-boot while looking for EFI application, etc.
Same for NetBSD. Assuming that other parts have not been changed, Simon's experiments should have used the same .dtb for all tests.
Now I remember that somebody complained about large files reading from eMMC.
Simon,
Could you please use u-boot console and try to locate and load dtb file manually using `ls` and `load` commands in both scenarios?
On 2019-09-20 12:43 p.m., Matwey V. Kornilov wrote:
Could you please use u-boot console and try to locate and load dtb file manually using `ls` and `load` commands in both scenarios?
Results attached.
I'm using a regular microSD card, not eMMC, but you're right: Using the U-Boot TPL the DTB file isn't loaded correctly; it's placed in memory but the data is corrupt:
=> md ${fdt_addr_r} 01f00000: adaaaa8a ea22282a baa0aeae aa7aa3aa ....*(".......z. 01f00010: a8a200ab abaa000a a8a2aa08 aa88aafa ................ 01f00020: e92a00a3 aabcf8a8 eaaaa8fa aaaaab88 ..*............. 01f00030: aaa8a0aa 2aa8088a 22a5e3ba 8bea4abe .......*...".J.. 01f00040: 23b188aa a20aa84a 82aaaaaa 8eee8cae ...#J........... 01f00050: 92aaaa4a aaa4feaa aa8a2eae eaa38b8a J............... 01f00060: aae4b2a8 aaacaaba aaea8ab8 2aeeb82a ............*..* (...) => load mmc 1 ${fdt_addr_r} /dtb/rockchip/rk3328-rock64.dtb 53513 bytes read in 10 ms (5.1 MiB/s) => md ${fdt_addr_r} 01f00000: edfe0d50 09d10000 38000000 70ba0000 P..........8...p 01f00010: 28000000 11000000 10000000 00000000 ...(............ 01f00020: 99120000 38ba0000 00000000 00000000 .......8........ 01f00030: 00000000 00000000 01000000 00000000 ................ 01f00040: 03000000 1e000000 00000000 656e6970 ............pine 01f00050: 722c3436 366b636f 6f720034 68636b63 64,rock64.rockch 01f00060: 722c7069 3233336b 00000038 03000000 ip,rk3328....... (...) => fdt addr ${fdt_addr_r} libfdt fdt_check_header(): FDT_ERR_BADMAGIC =>
Note the magic number at the start, which should be "edfe0dd0".
When the Rockchip TPL is used the same DTB file loads without issue, and the magic number is correct.
My current theory (following Mark Kettenis' email) is that all this is due to U-Boot's not configuring the RK805 PMIC on the ROCK64. The device tree suggests it's used to regulate power to the DRAM and is expected to be set up at pre-boot:
vcc_ddr: DCDC_REG3 { regulator-name = "vcc_ddr"; regulator-always-on; regulator-boot-on; regulator-state-mem { regulator-on-in-suspend; }; };
However I see only yesterday Elaine Zhang posted patches that enable support in U-Boot for the RK805, which makes me think this isn't taking place currently. And while NetBSD includes its own RK805 driver it is likely assuming this setup has already happened ("regulator-boot-on") by the time the kernel starts and does not need to be done again.
Does this sound reasonable?
I'm going to apply Elaine's patches and see if that changes things.
пт, 20 сент. 2019 г. в 20:28, Simon South simon@simonsouth.net:
On 2019-09-20 12:43 p.m., Matwey V. Kornilov wrote:
Could you please use u-boot console and try to locate and load dtb file manually using `ls` and `load` commands in both scenarios?
Results attached.
I'm using a regular microSD card, not eMMC, but you're right: Using the U-Boot TPL the DTB file isn't loaded correctly; it's placed in memory but the data is corrupt:
=> md ${fdt_addr_r} 01f00000: adaaaa8a ea22282a baa0aeae aa7aa3aa ....*(".......z. 01f00010: a8a200ab abaa000a a8a2aa08 aa88aafa ................ 01f00020: e92a00a3 aabcf8a8 eaaaa8fa aaaaab88 ..*............. 01f00030: aaa8a0aa 2aa8088a 22a5e3ba 8bea4abe .......*...".J.. 01f00040: 23b188aa a20aa84a 82aaaaaa 8eee8cae ...#J........... 01f00050: 92aaaa4a aaa4feaa aa8a2eae eaa38b8a J............... 01f00060: aae4b2a8 aaacaaba aaea8ab8 2aeeb82a ............*..* (...) => load mmc 1 ${fdt_addr_r} /dtb/rockchip/rk3328-rock64.dtb 53513 bytes read in 10 ms (5.1 MiB/s) => md ${fdt_addr_r} 01f00000: edfe0d50 09d10000 38000000 70ba0000 P..........8...p 01f00010: 28000000 11000000 10000000 00000000 ...(............ 01f00020: 99120000 38ba0000 00000000 00000000 .......8........ 01f00030: 00000000 00000000 01000000 00000000 ................ 01f00040: 03000000 1e000000 00000000 656e6970 ............pine 01f00050: 722c3436 366b636f 6f720034 68636b63 64,rock64.rockch 01f00060: 722c7069 3233336b 00000038 03000000 ip,rk3328....... (...) => fdt addr ${fdt_addr_r} libfdt fdt_check_header(): FDT_ERR_BADMAGIC =>Note the magic number at the start, which should be "edfe0dd0".
Note, that EFI application is loaded successfully at the same time. What is the difference? In filesize?
When the Rockchip TPL is used the same DTB file loads without issue, and the magic number is correct.
My current theory (following Mark Kettenis' email) is that all this is due to U-Boot's not configuring the RK805 PMIC on the ROCK64. The device tree suggests it's used to regulate power to the DRAM and is expected to be set up at pre-boot:
vcc_ddr: DCDC_REG3 { regulator-name = "vcc_ddr"; regulator-always-on; regulator-boot-on; regulator-state-mem { regulator-on-in-suspend; }; };However I see only yesterday Elaine Zhang posted patches that enable support in U-Boot for the RK805, which makes me think this isn't taking place currently. And while NetBSD includes its own RK805 driver it is likely assuming this setup has already happened ("regulator-boot-on") by the time the kernel starts and does not need to be done again.
Does this sound reasonable?
I'm going to apply Elaine's patches and see if that changes things.
-- Simon South simon@simonsouth.net
On 2019-09-20 1:37 p.m., Matwey V. Kornilov wrote:
What is the difference? In filesize?
The EFI bootloader:
205136 bootaa64.efi
The DTB file:
53513 rk3328-rock64.dtb
It's about one quarter the size.
It is strange the EFI bootloader and the kernel both seem always to load reliably when the DTB file doesn't.
----------
=> ls mmc 1 /EFI/BOOT/ ./ ../ 205136 bootaa64.efi
1 file(s), 2 dir(s)
=> ls mmc 1 /dtb/rockchip/ ./ ../ 77017 rk3399-sapphire.dtb 81304 rk3399-sapphire-excavator.dtb 78646 rk3399-rockpro64.dtb 77846 rk3399-rock960.dtb 76155 rk3399-rock-pi-4.dtb 77701 rk3399-roc-pc.dtb 78035 rk3399-puma-haikou.dtb 78483 rk3399-nanopi-m4.dtb 78791 rk3399-nanopc-t4.dtb 86532 rk3399-gru-scarlet-kd.dtb 86507 rk3399-gru-scarlet-inx.dtb 89522 rk3399-gru-kevin.dtb 85495 rk3399-gru-bob.dtb 80312 rk3399-firefly.dtb 78519 rk3399-ficus.dtb 68842 rk3399-evb.dtb 53513 rk3328-rock64.dtb 52150 rk3328-roc-cc.dtb 51134 rk3328-evb.dtb
19 file(s), 2 dir(s)
=>
Could you please try to revert this commit and see what's happening then?
https://github.com/u-boot/u-boot/commit/16593ccd07afdbb24a937eac4ea7e16269ff...
пт, 20 сент. 2019, 20:44 Simon South simon@simonsouth.net:
On 2019-09-20 1:37 p.m., Matwey V. Kornilov wrote:
What is the difference? In filesize?
The EFI bootloader:
205136 bootaa64.efiThe DTB file:
53513 rk3328-rock64.dtbIt's about one quarter the size.
It is strange the EFI bootloader and the kernel both seem always to load reliably when the DTB file doesn't.
=> ls mmc 1 /EFI/BOOT/ ./ ../ 205136 bootaa64.efi
1 file(s), 2 dir(s)
=> ls mmc 1 /dtb/rockchip/ ./ ../ 77017 rk3399-sapphire.dtb 81304 rk3399-sapphire-excavator.dtb 78646 rk3399-rockpro64.dtb 77846 rk3399-rock960.dtb 76155 rk3399-rock-pi-4.dtb 77701 rk3399-roc-pc.dtb 78035 rk3399-puma-haikou.dtb 78483 rk3399-nanopi-m4.dtb 78791 rk3399-nanopc-t4.dtb 86532 rk3399-gru-scarlet-kd.dtb 86507 rk3399-gru-scarlet-inx.dtb 89522 rk3399-gru-kevin.dtb 85495 rk3399-gru-bob.dtb 80312 rk3399-firefly.dtb 78519 rk3399-ficus.dtb 68842 rk3399-evb.dtb 53513 rk3328-rock64.dtb 52150 rk3328-roc-cc.dtb 51134 rk3328-evb.dtb
19 file(s), 2 dir(s)
=>
-- Simon South simon@simonsouth.net
On 2019-09-21 3:47 a.m., Matwey V. Kornilov wrote:
Could you please try to revert this commit and see what's happening then?
Unfortunately undoing that change makes things worse: Either the NetBSD EFI bootloader hangs immediately on start (without any output) or it crashes with output like
"Synchronous Abort" handler, esr 0x96000044 elr: fffffffffd1ad760 lr : fffffffffd1ac030 (reloc) elr: 00000000fbeeb760 lr : 00000000fbeea030 x0 : 00000000fcf679a0 x1 : 00000000fef3e628 x2 : 0000000000000000 x3 : 00000000fbf17408 (...)
In either case I still see the "FDT_ERR_BADMAGIC" messages from U-Boot.
Applying Elaine's patches for RK805 support makes no difference.
Complete log attached.
Could you try to load dtb (and check that dtb is valid) into the other memory location? For instance into ${kernel_addr_r} which is a place for loading EFI application binary?
пт, 20 сент. 2019 г. в 20:44, Simon South simon@simonsouth.net:
On 2019-09-20 1:37 p.m., Matwey V. Kornilov wrote:
What is the difference? In filesize?
The EFI bootloader:
205136 bootaa64.efiThe DTB file:
53513 rk3328-rock64.dtbIt's about one quarter the size.
It is strange the EFI bootloader and the kernel both seem always to load reliably when the DTB file doesn't.
=> ls mmc 1 /EFI/BOOT/ ./ ../ 205136 bootaa64.efi
1 file(s), 2 dir(s)
=> ls mmc 1 /dtb/rockchip/ ./ ../ 77017 rk3399-sapphire.dtb 81304 rk3399-sapphire-excavator.dtb 78646 rk3399-rockpro64.dtb 77846 rk3399-rock960.dtb 76155 rk3399-rock-pi-4.dtb 77701 rk3399-roc-pc.dtb 78035 rk3399-puma-haikou.dtb 78483 rk3399-nanopi-m4.dtb 78791 rk3399-nanopc-t4.dtb 86532 rk3399-gru-scarlet-kd.dtb 86507 rk3399-gru-scarlet-inx.dtb 89522 rk3399-gru-kevin.dtb 85495 rk3399-gru-bob.dtb 80312 rk3399-firefly.dtb 78519 rk3399-ficus.dtb 68842 rk3399-evb.dtb 53513 rk3328-rock64.dtb 52150 rk3328-roc-cc.dtb 51134 rk3328-evb.dtb
19 file(s), 2 dir(s)
=>
-- Simon South simon@simonsouth.net
On 2019-09-21 8:29 a.m., Matwey V. Kornilov wrote:
Could you try to load dtb (and check that dtb is valid) into the other memory location?
Same result: With the U-Boot TPL this fails (the FDT header is corrupt); with the Rockchip TPL this works fine. See below.
I notice though it always seems to be only that one single bit (bit 7 of the first 32-bit word loaded) that differs between the two runs. Everything else in memory looks fine. And with the U-Boot TPL in use I can manually set a new value for that word, and it seems to stick:
=> md ${kernel_addr_r} 2 02080000: edfe0d50 09d10000 P....... => fdt addr ${kernel_addr_r} libfdt fdt_check_header(): FDT_ERR_BADMAGIC => nm ${kernel_addr_r} 02080000: edfe0d50 ? edfe0dd0 02080000: edfe0dd0 ? q => md ${kernel_addr_r} 2 02080000: edfe0dd0 09d10000 ........ => fdt addr ${kernel_addr_r} =>
I wonder if this is a clue as to what's going on.
----------
U-Boot TPL:
Hit any key to stop autoboot: 0 => print kernel_addr_r kernel_addr_r=0x02080000 => md ${kernel_addr_r} 20 02080000: adaaaa8a ea22286a baa0aeae aa7aa1aa ....j(".......z. 02080010: a8b200ab abaa000a a8a2aa08 aa88aaea ................ 02080020: e92a00a3 2aacf8aa eaaaa8fa aaaaab88 ..*....*........ 02080030: aaa8a0aa 2aa8888a 22a5e2ba 8aea4a3c .......*..."<J.. 02080040: 23b188aa a28a884a 82aaaaaa 8eee8cae ...#J........... 02080050: 92aaaa4a aaa4feaa ea8a2eae eaa38b8a J............... 02080060: aee4b2a8 aaacaaba aaea8ab8 2aee982a ............*..* 02080070: aaa0aaac aaeeab9a aeae84c8 ac2e8aaa ................ => load mmc 1 ${kernel_addr_r} /dtb/rockchip/rk3328-rock64.dtb 53513 bytes read in 10 ms (5.1 MiB/s) => md ${kernel_addr_r} 20 02080000: edfe0d50 09d10000 38000000 70ba0000 P..........8...p 02080010: 28000000 11000000 10000000 00000000 ...(............ 02080020: 99120000 38ba0000 00000000 00000000 .......8........ 02080030: 00000000 00000000 01000000 00000000 ................ 02080040: 03000000 1e000000 00000000 656e6970 ............pine 02080050: 722c3436 366b636f 6f720034 68636b63 64,rock64.rockch 02080060: 722c7069 3233336b 00000038 03000000 ip,rk3328....... 02080070: 04000000 0b000000 01000000 03000000 ................ => fdt addr ${kernel_addr_r} libfdt fdt_check_header(): FDT_ERR_BADMAGIC =>
Rockchip TPL:
Hit any key to stop autoboot: 0 => print kernel_addr_r kernel_addr_r=0x02080000 => md ${kernel_addr_r} 20 02080000: adaaaa9a ea23286a baa8aeae aa7aa3aa ....j(#.......z. 02080010: aaf200ab abaa080a a8b2aa88 aa88aafa ................ 02080020: e92a4ca3 aafcf8aa eaaaa8fa aaaaab88 .L*............. 02080030: aaa8a0aa 2aea888b e2a5ebfa abea6abe .......*.....j.. 02080040: abb188aa a29aac4a 8aaaaaaa 8eee8cae ....J........... 02080050: d2aaaace aaa4feea ea8a6ebe eaa38b8a .........n...... 02080060: aee4b2a9 aaacaaba aaea8ab8 3aeeb8aa ...............: 02080070: aaa1aaae aaeeab9e aeae8ce8 ae2e8aaa ................ => load mmc 1 ${kernel_addr_r} /dtb/rockchip/rk3328-rock64.dtb 53513 bytes read in 10 ms (5.1 MiB/s) => md ${kernel_addr_r} 20 02080000: edfe0dd0 09d10000 38000000 70ba0000 ...........8...p 02080010: 28000000 11000000 10000000 00000000 ...(............ 02080020: 99120000 38ba0000 00000000 00000000 .......8........ 02080030: 00000000 00000000 01000000 00000000 ................ 02080040: 03000000 1e000000 00000000 656e6970 ............pine 02080050: 722c3436 366b636f 6f720034 68636b63 64,rock64.rockch 02080060: 722c7069 3233336b 00000038 03000000 ip,rk3328....... 02080070: 04000000 0b000000 01000000 03000000 ................ => fdt addr ${kernel_addr_r} =>
сб, 21 сент. 2019 г. в 16:38, Simon South simon@simonsouth.net:
On 2019-09-21 8:29 a.m., Matwey V. Kornilov wrote:
Could you try to load dtb (and check that dtb is valid) into the other memory location?
Same result: With the U-Boot TPL this fails (the FDT header is corrupt); with the Rockchip TPL this works fine. See below.
I notice though it always seems to be only that one single bit (bit 7 of the first 32-bit word loaded) that differs between the two runs. Everything else in memory looks fine. And with the U-Boot TPL in use I can manually set a new value for that word, and it seems to stick:
I have no idea. Only suggestion is that something thinks that this memory position is MMIO and then it writes 7 bit for the some purpose.
=> md ${kernel_addr_r} 2 02080000: edfe0d50 09d10000 P....... => fdt addr ${kernel_addr_r} libfdt fdt_check_header(): FDT_ERR_BADMAGIC => nm ${kernel_addr_r} 02080000: edfe0d50 ? edfe0dd0 02080000: edfe0dd0 ? q => md ${kernel_addr_r} 2 02080000: edfe0dd0 09d10000 ........ => fdt addr ${kernel_addr_r} =>I wonder if this is a clue as to what's going on.
U-Boot TPL:
Hit any key to stop autoboot: 0 => print kernel_addr_r kernel_addr_r=0x02080000 => md ${kernel_addr_r} 20 02080000: adaaaa8a ea22286a baa0aeae aa7aa1aa ....j(".......z. 02080010: a8b200ab abaa000a a8a2aa08 aa88aaea ................ 02080020: e92a00a3 2aacf8aa eaaaa8fa aaaaab88 ..*....*........ 02080030: aaa8a0aa 2aa8888a 22a5e2ba 8aea4a3c .......*..."<J.. 02080040: 23b188aa a28a884a 82aaaaaa 8eee8cae ...#J........... 02080050: 92aaaa4a aaa4feaa ea8a2eae eaa38b8a J............... 02080060: aee4b2a8 aaacaaba aaea8ab8 2aee982a ............*..* 02080070: aaa0aaac aaeeab9a aeae84c8 ac2e8aaa ................ => load mmc 1 ${kernel_addr_r} /dtb/rockchip/rk3328-rock64.dtb 53513 bytes read in 10 ms (5.1 MiB/s) => md ${kernel_addr_r} 20 02080000: edfe0d50 09d10000 38000000 70ba0000 P..........8...p 02080010: 28000000 11000000 10000000 00000000 ...(............ 02080020: 99120000 38ba0000 00000000 00000000 .......8........ 02080030: 00000000 00000000 01000000 00000000 ................ 02080040: 03000000 1e000000 00000000 656e6970 ............pine 02080050: 722c3436 366b636f 6f720034 68636b63 64,rock64.rockch 02080060: 722c7069 3233336b 00000038 03000000 ip,rk3328....... 02080070: 04000000 0b000000 01000000 03000000 ................ => fdt addr ${kernel_addr_r} libfdt fdt_check_header(): FDT_ERR_BADMAGIC =>Rockchip TPL:
Hit any key to stop autoboot: 0 => print kernel_addr_r kernel_addr_r=0x02080000 => md ${kernel_addr_r} 20 02080000: adaaaa9a ea23286a baa8aeae aa7aa3aa ....j(#.......z. 02080010: aaf200ab abaa080a a8b2aa88 aa88aafa ................ 02080020: e92a4ca3 aafcf8aa eaaaa8fa aaaaab88 .L*............. 02080030: aaa8a0aa 2aea888b e2a5ebfa abea6abe .......*.....j.. 02080040: abb188aa a29aac4a 8aaaaaaa 8eee8cae ....J........... 02080050: d2aaaace aaa4feea ea8a6ebe eaa38b8a .........n...... 02080060: aee4b2a9 aaacaaba aaea8ab8 3aeeb8aa ...............: 02080070: aaa1aaae aaeeab9e aeae8ce8 ae2e8aaa ................ => load mmc 1 ${kernel_addr_r} /dtb/rockchip/rk3328-rock64.dtb 53513 bytes read in 10 ms (5.1 MiB/s) => md ${kernel_addr_r} 20 02080000: edfe0dd0 09d10000 38000000 70ba0000 ...........8...p 02080010: 28000000 11000000 10000000 00000000 ...(............ 02080020: 99120000 38ba0000 00000000 00000000 .......8........ 02080030: 00000000 00000000 01000000 00000000 ................ 02080040: 03000000 1e000000 00000000 656e6970 ............pine 02080050: 722c3436 366b636f 6f720034 68636b63 64,rock64.rockch 02080060: 722c7069 3233336b 00000038 03000000 ip,rk3328....... 02080070: 04000000 0b000000 01000000 03000000 ................ => fdt addr ${kernel_addr_r} =>-- Simon South simon@simonsouth.net
Simon, maybe you'll describe a way to obtain SD card image which you use? For the case if anybody wants to reproduce the issue in situ?
сб, 21 сент. 2019 г. в 20:15, Matwey V. Kornilov matwey.kornilov@gmail.com:
сб, 21 сент. 2019 г. в 16:38, Simon South simon@simonsouth.net:
On 2019-09-21 8:29 a.m., Matwey V. Kornilov wrote:
Could you try to load dtb (and check that dtb is valid) into the other memory location?
Same result: With the U-Boot TPL this fails (the FDT header is corrupt); with the Rockchip TPL this works fine. See below.
I notice though it always seems to be only that one single bit (bit 7 of the first 32-bit word loaded) that differs between the two runs. Everything else in memory looks fine. And with the U-Boot TPL in use I can manually set a new value for that word, and it seems to stick:
I have no idea. Only suggestion is that something thinks that this memory position is MMIO and then it writes 7 bit for the some purpose.
=> md ${kernel_addr_r} 2 02080000: edfe0d50 09d10000 P....... => fdt addr ${kernel_addr_r} libfdt fdt_check_header(): FDT_ERR_BADMAGIC => nm ${kernel_addr_r} 02080000: edfe0d50 ? edfe0dd0 02080000: edfe0dd0 ? q => md ${kernel_addr_r} 2 02080000: edfe0dd0 09d10000 ........ => fdt addr ${kernel_addr_r} =>I wonder if this is a clue as to what's going on.
U-Boot TPL:
Hit any key to stop autoboot: 0 => print kernel_addr_r kernel_addr_r=0x02080000 => md ${kernel_addr_r} 20 02080000: adaaaa8a ea22286a baa0aeae aa7aa1aa ....j(".......z. 02080010: a8b200ab abaa000a a8a2aa08 aa88aaea ................ 02080020: e92a00a3 2aacf8aa eaaaa8fa aaaaab88 ..*....*........ 02080030: aaa8a0aa 2aa8888a 22a5e2ba 8aea4a3c .......*..."<J.. 02080040: 23b188aa a28a884a 82aaaaaa 8eee8cae ...#J........... 02080050: 92aaaa4a aaa4feaa ea8a2eae eaa38b8a J............... 02080060: aee4b2a8 aaacaaba aaea8ab8 2aee982a ............*..* 02080070: aaa0aaac aaeeab9a aeae84c8 ac2e8aaa ................ => load mmc 1 ${kernel_addr_r} /dtb/rockchip/rk3328-rock64.dtb 53513 bytes read in 10 ms (5.1 MiB/s) => md ${kernel_addr_r} 20 02080000: edfe0d50 09d10000 38000000 70ba0000 P..........8...p 02080010: 28000000 11000000 10000000 00000000 ...(............ 02080020: 99120000 38ba0000 00000000 00000000 .......8........ 02080030: 00000000 00000000 01000000 00000000 ................ 02080040: 03000000 1e000000 00000000 656e6970 ............pine 02080050: 722c3436 366b636f 6f720034 68636b63 64,rock64.rockch 02080060: 722c7069 3233336b 00000038 03000000 ip,rk3328....... 02080070: 04000000 0b000000 01000000 03000000 ................ => fdt addr ${kernel_addr_r} libfdt fdt_check_header(): FDT_ERR_BADMAGIC =>Rockchip TPL:
Hit any key to stop autoboot: 0 => print kernel_addr_r kernel_addr_r=0x02080000 => md ${kernel_addr_r} 20 02080000: adaaaa9a ea23286a baa8aeae aa7aa3aa ....j(#.......z. 02080010: aaf200ab abaa080a a8b2aa88 aa88aafa ................ 02080020: e92a4ca3 aafcf8aa eaaaa8fa aaaaab88 .L*............. 02080030: aaa8a0aa 2aea888b e2a5ebfa abea6abe .......*.....j.. 02080040: abb188aa a29aac4a 8aaaaaaa 8eee8cae ....J........... 02080050: d2aaaace aaa4feea ea8a6ebe eaa38b8a .........n...... 02080060: aee4b2a9 aaacaaba aaea8ab8 3aeeb8aa ...............: 02080070: aaa1aaae aaeeab9e aeae8ce8 ae2e8aaa ................ => load mmc 1 ${kernel_addr_r} /dtb/rockchip/rk3328-rock64.dtb 53513 bytes read in 10 ms (5.1 MiB/s) => md ${kernel_addr_r} 20 02080000: edfe0dd0 09d10000 38000000 70ba0000 ...........8...p 02080010: 28000000 11000000 10000000 00000000 ...(............ 02080020: 99120000 38ba0000 00000000 00000000 .......8........ 02080030: 00000000 00000000 01000000 00000000 ................ 02080040: 03000000 1e000000 00000000 656e6970 ............pine 02080050: 722c3436 366b636f 6f720034 68636b63 64,rock64.rockch 02080060: 722c7069 3233336b 00000038 03000000 ip,rk3328....... 02080070: 04000000 0b000000 01000000 03000000 ................ => fdt addr ${kernel_addr_r} =>-- Simon South simon@simonsouth.net
-- With best regards, Matwey V. Kornilov
On 2019-09-24 12:54 p.m., Matwey V. Kornilov wrote:
Simon, maybe you'll describe a way to obtain SD card image which you use? For the case if anybody wants to reproduce the issue in situ?
Sure---I'd love for other people to test this, and it's pretty easy since it doesn't require anything other than U-Boot itself.
Assuming the CROSS_COMPILE and BL31 environment variables are set up appropriately (I'm using ATF 2.1 from the mainline repository):
1. Switch to U-Boot's master branch and/or configure the source tree identically to my own with
git checkout a9fa70b7b7167487affc5d919e541872c99e814b
2. Apply the attached patch, which enables the "mtest" command, with
patch -p1 < rock64-enable-mtest.patch
3. Build U-Boot as usual:
make distclean; rm -f ./idbloader.img make rock64-rk3328_defconfig all u-boot.itb
4. Write U-Boot to a microSD card as usual:
sudo dd if=idbloader.img of=/dev/mmcblk0 seek=64 sudo dd if=u-boot.itb of=/dev/mmcblk0 seek=16384
5. Place the microSD card in the ROCK64 and start it. Once the SPL loads, hit the spacebar to enter the shell and start the memory test with
mtest
On my ROCK64, this immediately begins throwing off errors like
FAILURE (read/write) @ 0x020014c0: expected 0x00000299, actual 0x00000219)
and if I continue the boot process with "boot", the NetBSD kernel will load but panic almost immediately.
Note mine is a ROCK64 v2.0 with a 32Gb SpecTek LPDDR3 memory module---there is also a ROCK64 v3.0 and some variance in the memory chips used, so it could be this is limited to a specific model or production run.
Now here's what I've discovered lately:
Everything works normally if I have U-Boot TPL's set a slightly lower clock rate for the memory, by editing arch/arm/dts/rk3328-sdram-lpddr3-1600.dtsi and changing the "800" on line 27 to "766".
That suggests a hardware issue with my ROCK64, and in fact others have reported similar problems with the 2.0 version [1]. I'd be willing to leave it at that, except that if I also patch the (333MHz) Rockchip TPL to set a _higher_ clock rate for the memory, it continues to work just fine---all the way up to 933MHz!
So this is not just a hardware problem; there is some incompatibility between the RK3328 SDRAM driver as it exists in U-Boot and at least one group of ROCK64s in the world.
At the moment I'm still trying to identify what's different in the closed-source loader that causes it not to have this issue. Anyone have any thoughts?
[1] https://forum.pine64.org/showthread.php?tid=7387
24.09.2019 20:54, Simon South пишет:
On 2019-09-24 12:54 p.m., Matwey V. Kornilov wrote:
Simon, maybe you'll describe a way to obtain SD card image which you use? For the case if anybody wants to reproduce the issue in situ?
Sure---I'd love for other people to test this, and it's pretty easy since it doesn't require anything other than U-Boot itself.
Assuming the CROSS_COMPILE and BL31 environment variables are set up appropriately (I'm using ATF 2.1 from the mainline repository):
- Switch to U-Boot's master branch and/or configure the source tree
identically to my own with
git checkout a9fa70b7b7167487affc5d919e541872c99e814b
- Apply the attached patch, which enables the "mtest" command, with
patch -p1 < rock64-enable-mtest.patch
- Build U-Boot as usual:
make distclean; rm -f ./idbloader.img make rock64-rk3328_defconfig all u-boot.itb
- Write U-Boot to a microSD card as usual:
sudo dd if=idbloader.img of=/dev/mmcblk0 seek=64 sudo dd if=u-boot.itb of=/dev/mmcblk0 seek=16384
- Place the microSD card in the ROCK64 and start it. Once the SPL
loads, hit the spacebar to enter the shell and start the memory test with
mtest
On my ROCK64, this immediately begins throwing off errors like
FAILURE (read/write) @ 0x020014c0: expected 0x00000299, actual 0x00000219)
and if I continue the boot process with "boot", the NetBSD kernel will load but panic almost immediately.
How do you take NetBSD image?
Note mine is a ROCK64 v2.0 with a 32Gb SpecTek LPDDR3 memory module---there is also a ROCK64 v3.0 and some variance in the memory chips used, so it could be this is limited to a specific model or production run.
Now here's what I've discovered lately:
Everything works normally if I have U-Boot TPL's set a slightly lower clock rate for the memory, by editing arch/arm/dts/rk3328-sdram-lpddr3-1600.dtsi and changing the "800" on line 27 to "766".
That suggests a hardware issue with my ROCK64, and in fact others have reported similar problems with the 2.0 version [1]. I'd be willing to leave it at that, except that if I also patch the (333MHz) Rockchip TPL to set a _higher_ clock rate for the memory, it continues to work just fine---all the way up to 933MHz!
So this is not just a hardware problem; there is some incompatibility between the RK3328 SDRAM driver as it exists in U-Boot and at least one group of ROCK64s in the world.
At the moment I'm still trying to identify what's different in the closed-source loader that causes it not to have this issue. Anyone have any thoughts?
[1] https://forum.pine64.org/showthread.php?tid=7387
U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
On 2019-09-24 2:35 p.m., Matwey V. Kornilov wrote> How do you take NetBSD image? I've been building my own images from NetBSD-CURRENT, following the guides [1][2].
There was until recently a buffer-alignment issue in NetBSD's EFI bootloader that prevented it from working with the current version of U-Boot, but a fix has been committed so something like the following should now work, without patching:
cd /usr/src # or $HOME/netbsd/usr/src, etc. cvs update -A -dP -C -D '20190922 1140Z' ./build.sh -U -u -m evbarm64 release
Then write the resulting image to a microSD card with
zcat obj/releasedir/evbarm/binary/gzimg/arm64.img.gz | sudo dd bs=32768 of=/dev/mmcblk0 skip=512 seek=512
and build and install U-Boot as in my previous email.
[1] https://netbsd.org/docs/current/index.html [2] https://netbsd.org/docs/guide/en/chap-build.html
On 2019-09-20 1:27 p.m., Simon South wrote:
I'm going to apply Elaine's patches and see if that changes things.
Unfortunately no; other than this line now appearing in the output:
PMIC: RK8050 (on=0x10, off=0x04)
nothing seems to have changed. Either the machine hangs or NetBSD's kernel panics.
On Thu, Sep 19, 2019 at 4:02 PM Simon South simon@simonsouth.net wrote:
I've been unable so far to boot NetBSD on my PINE64 ROCK64 (v2.0, Rockchip RK3328) using U-Boot built with its own TPL, the default since commit ff3dd0a474.
The same NetBSD image boots fine using the binary TPL supplied by Rockchip.
The exact failure varies but it always seems memory-related. Typically the NetBSD kernel fails right after starting with a panic in its virtual-memory system, uvm.
Also significant may be these messages sometimes output by U-Boot at startup, which I haven't seen when the Rockchip TPL is used:
libfdt fdt_check_header(): FDT_ERR_BADMAGICI've pasted the output from a typical failed boot below and have submitted a problem report (with a bit more detail) to the NetBSD project:
https://gnats.netbsd.org/54557
Does anyone know why this might be happening? Is there perhaps some additional setup the U-Boot TPL expects from the OS to finish configuring the RK3328's memory controller that's currently missing from NetBSD?
How I can diagnose this further?
U-Boot TPL 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:11:29) LPDDR3 Trying to boot from BOOTROM Returning to boot ROM...
U-Boot SPL 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:11:29 -0400) Trying to boot from MMC2 Card did not respond to voltage select! spl: mmc init failed with error: -95 Trying to boot from MMC1 NOTICE: BL31: v2.1(release):v2.1-678-g2fc6ffc4-dirty NOTICE: BL31: Built : 12:10:07, Sep 12 2019 ERROR: over or zero region, nr=4187432, max=10 NOTICE: BL31:Rockchip release version: v1.2
U-Boot 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:12:31 -0400)
Model: Pine64 Rock64 DRAM: 4 GiB MMC: rksdmmc@ff500000: 1, rksdmmc@ff520000: 0 Loading Environment from MMC... *** Warning - bad CRC, using default environment
In: serial@ff130000 Out: serial@ff130000 Err: serial@ff130000 Model: Pine64 Rock64 Net: Warning: ethernet@ff540000 (eth0) using random MAC address - da:e0:78:b1:50:d2 eth0: ethernet@ff540000 Hit any key to stop autoboot: 0 Card did not respond to voltage select! switch to partitions #0, OK mmc1 is current device Scanning mmc 1:1... 53513 bytes read in 9 ms (5.7 MiB/s) Found EFI removable media binary efi/boot/bootaa64.efi libfdt fdt_check_header(): FDT_ERR_BADMAGIC
It seems to be a memory overlap or similar. replace TPL with rkbin would be worth checking since the DRAM is managed by rkbin instead of TPL here.
rkbin => SPL => U-Boot proper
On 2019-09-19 12:29 p.m., Jagan Teki wrote:
It seems to be a memory overlap or similar. replace TPL with rkbin would be worth checking since the DRAM is managed by rkbin instead of TPL here.
rkbin => SPL => U-Boot proper
That configuration works and allows NetBSD to boot just fine. It's only when using U-Boot's own TPL that the kernel panics shortly after starting.
Can you think of what the difference might be between the two TPLs?
19.09.2019 11:56, Simon South пишет:
I've been unable so far to boot NetBSD on my PINE64 ROCK64 (v2.0, Rockchip RK3328) using U-Boot built with its own TPL, the default since commit ff3dd0a474.
The same NetBSD image boots fine using the binary TPL supplied by Rockchip.
The exact failure varies but it always seems memory-related. Typically the NetBSD kernel fails right after starting with a panic in its virtual-memory system, uvm.
Also significant may be these messages sometimes output by U-Boot at startup, which I haven't seen when the Rockchip TPL is used:
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
I've pasted the output from a typical failed boot below and have submitted a problem report (with a bit more detail) to the NetBSD project:
https://gnats.netbsd.org/54557
Does anyone know why this might be happening? Is there perhaps some additional setup the U-Boot TPL expects from the OS to finish configuring the RK3328's memory controller that's currently missing from NetBSD?
How I can diagnose this further?
U-Boot TPL 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:11:29)
I've just checked this commit: EFI Grub + Linux 5.2 is still booting for me.
LPDDR3 Trying to boot from BOOTROM Returning to boot ROM...
U-Boot SPL 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:11:29 -0400) Trying to boot from MMC2 Card did not respond to voltage select! spl: mmc init failed with error: -95 Trying to boot from MMC1 NOTICE: BL31: v2.1(release):v2.1-678-g2fc6ffc4-dirty NOTICE: BL31: Built : 12:10:07, Sep 12 2019 ERROR: over or zero region, nr=4187432, max=10 NOTICE: BL31:Rockchip release version: v1.2
U-Boot 2019.10-rc3-00361-ga9fa70b7b7 (Sep 19 2019 - 03:12:31 -0400)
Model: Pine64 Rock64 DRAM: 4 GiB MMC: rksdmmc@ff500000: 1, rksdmmc@ff520000: 0 Loading Environment from MMC... *** Warning - bad CRC, using default environment
In: serial@ff130000 Out: serial@ff130000 Err: serial@ff130000 Model: Pine64 Rock64 Net: Warning: ethernet@ff540000 (eth0) using random MAC address - da:e0:78:b1:50:d2 eth0: ethernet@ff540000 Hit any key to stop autoboot: 0 Card did not respond to voltage select! switch to partitions #0, OK mmc1 is current device Scanning mmc 1:1... 53513 bytes read in 9 ms (5.7 MiB/s) Found EFI removable media binary efi/boot/bootaa64.efi libfdt fdt_check_header(): FDT_ERR_BADMAGIC Scanning disk rksdmmc@ff500000.blk... Card did not respond to voltage select! Scanning disk rksdmmc@ff520000.blk... Disk rksdmmc@ff520000.blk not ready Found 3 disks BootOrder not defined EFI boot manager: Cannot load any image 205136 bytes read in 16 ms (12.2 MiB/s) libfdt fdt_check_header(): FDT_ERR_BADMAGIC
NetBSD/evbarm EFI Boot (aarch64), Revision 1.11 (Wed Sep 18 15:50:45
UTC 2019) (from NetBSD 9.99.12) Press return to boot now, any other key for boot prompt booting netbsd - starting in 0 seconds. 6100056+2730512+1985652+1823764+490622=0xec5b00 [ 1.0000000] NetBSD/evbarm (fdt) booting ... [ 1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, [ 1.0000000] 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, [ 1.0000000] 2018, 2019 The NetBSD Foundation, Inc. All rights reserved. [ 1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993 [ 1.0000000] The Regents of the University of California. All rights reserved.
[ 1.0000000] NetBSD 9.99.12 (GENERIC64) #16: Wed Sep 18 12:30:03 EDT 2019 [ 1.0000000] ssouth@laptop:/usr/src/sys/arch/evbarm/compile/obj/GENERIC64 [ 1.0000000] total memory = 4064 MB [ 1.0000000] avail memory = 3918 MB [ 1.0000000] panic: kernel diagnostic assertion "ucpu == VM_FREE_PAGE_TO_CPU(pg)" failed: file "/usr/src/sys/uvm/uvm_page.c", line 852 [ 1.0000000] cpu0: Begin traceback... [ 1.0000000] trace fp ffffffc000bec8d0 [ 1.0000000] fp ffffffc000bec8f0 vpanic() at ffffffc0004924c0 netbsd:vpanic+0x198 [ 1.0000000] fp ffffffc000bec950 kern_assert() at ffffffc0005cdf24 netbsd:kern_assert+0x5c [ 1.0000000] fp ffffffc000bec9e0 uvm_pagealloc_pgfl() at ffffffc00040f388 netbsd:uvm_pagealloc_pgfl+0x5c8 [ 1.0000000] fp ffffffc000beca40 uvm_pagealloc_strat() at ffffffc000410034 netbsd:uvm_pagealloc_strat+0x18c [ 1.0000000] fp ffffffc000becad0 uvm_km_kmem_alloc() at ffffffc000403b30 netbsd:uvm_km_kmem_alloc+0x60 [ 1.0000000] fp ffffffc000becb40 pool_page_alloc() at ffffffc00048b0c0 netbsd:pool_page_alloc+0x28 [ 1.0000000] fp ffffffc000becb60 pool_grow() at ffffffc00048d294 netbsd:pool_grow+0x35c [ 1.0000000] fp ffffffc000becbd0 pool_get() at ffffffc00048c5e8 netbsd:pool_get+0xa0 [ 1.0000000] fp ffffffc000becc30 pool_cache_get_slow() at ffffffc00048e8a4 netbsd:pool_cache_get_slow+0x1dc [ 1.0000000] fp ffffffc000becc80 pool_cache_get_paddr() at ffffffc0004901e8 netbsd:pool_cache_get_paddr+0x258 [ 1.0000000] fp ffffffc000becce0 kmem_intr_alloc() at ffffffc0004859f0 netbsd:kmem_intr_alloc+0x60 [ 1.0000000] fp ffffffc000becd10 kmem_intr_zalloc() at ffffffc000485b20 netbsd:kmem_intr_zalloc+0x10 [ 1.0000000] fp ffffffc000becd30 kmem_zalloc() at ffffffc000485dc4 netbsd:kmem_zalloc+0x84 [ 1.0000000] fp ffffffc000becd50 module_builtin_add() at ffffffc0004432a0 netbsd:module_builtin_add+0x98 [ 1.0000000] fp ffffffc000becdf0 module_init() at ffffffc0004437f8 netbsd:module_init+0xf0 [ 1.0000000] fp ffffffc000bece50 main() at ffffffc0005d0b08 netbsd:main+0x108 [ 1.0000000] fp 0000000000000000 aarch64_start() at ffffffc00000183c netbsd:aarch64_start+0x103c [ 1.0000000] cpu0: End traceback... Stopped in pid 0.1 (system) at netbsd:cpu_Debugger+0x4: ret db{0}>
From: Simon South simon@simonsouth.net Date: Thu, 19 Sep 2019 04:56:57 -0400
I've been unable so far to boot NetBSD on my PINE64 ROCK64 (v2.0, Rockchip RK3328) using U-Boot built with its own TPL, the default since commit ff3dd0a474.
The same NetBSD image boots fine using the binary TPL supplied by Rockchip.
The exact failure varies but it always seems memory-related. Typically the NetBSD kernel fails right after starting with a panic in its virtual-memory system, uvm.
Also significant may be these messages sometimes output by U-Boot at startup, which I haven't seen when the Rockchip TPL is used:
libfdt fdt_check_header(): FDT_ERR_BADMAGICI've pasted the output from a typical failed boot below and have submitted a problem report (with a bit more detail) to the NetBSD project:
https://gnats.netbsd.org/54557
Does anyone know why this might be happening? Is there perhaps some additional setup the U-Boot TPL expects from the OS to finish configuring the RK3328's memory controller that's currently missing from NetBSD?
Hi Simon,
One thing to look at is whether (voltage) regulators and/or clocks are set up right. I've seen the OpenBSD kernel crash because one of the voltages supplied to the SoC was too low. I beieve that was on the rockpro64. The regulator in question was a "boot-on" regulator so the OpenBSD kernel assumed the regulator was already properly configured. But I believe the Linux kernel doesn't trust the bootloader and will configure such a regulator anyway.
In my rockpro64 case the regulator was skipped because of some DM-conversion fall-out. Using the U=boot "dm" commands you can check whether drivers for various regulators are enabled. If a "boot-on" regulator doesn't have a "*" next to it, that is a clear indication of a problem. Worth looking at "always-on" regulators as well.
Cheers,
Mark
On 2019-09-20 6:03 a.m., Mark Kettenis wrote:
One thing to look at is whether (voltage) regulators and/or clocks are set up right...
Thanks, Mark. This is helpful.
But I believe the Linux kernel doesn't trust the bootloader and will configure such a regulator anyway.
This is my assumption right now as well, that Linux is itself configuring the RK3328 "correctly" and either the U-Boot TPL or the NetBSD kernel needs to be updated to do the same thing.
I'm going to grab the Linux source and see if I can spot anything it does at startup that's missing from the TPL.
Using the U=boot "dm" commands you can check whether drivers for various regulators are enabled.
I tried capturing the output of these commands when each TPL is used and comparing them. Unfortunately nothing jumps out: After booting with U-Boot's TPL "dm tree" shows additional entries for the S/PDIF and MMC controllers, and when booting with Rockchip's TPL a (possibly spurious) "Initializing UCLASS_EFI" message is output, but otherwise they're the same.
I've attached the two capture logs in case anyone is curious.
hi I have added the IDE support and I am able to operate with pATA harddrives, but microdrives are not recognized ...
any help to what and where I have to investigate?
On 2019-09-19 4:56 a.m., Simon South wrote:
I've been unable so far to boot NetBSD on my PINE64 ROCK64 (v2.0, Rockchip RK3328) using U-Boot built with its own TPL...
This definitely seems memory-related: After enabling the "mtest" command (patch attached) and running the memory test at the U-Boot shell I see all kinds of errors:
Testing 02000000 ... 03ffffff: Iteration: 1 FAILURE (read/write) @ 0x020014c0: expected 0x00000299, actual 0x00000219)
FAILURE (read/write) @ 0x02001580: expected 0x000002b1, actual 0x00000231)
FAILURE (read/write) @ 0x02001680: expected 0x000002d1, actual 0x00000251)
FAILURE (read/write) @ 0x02001e40: expected 0x000003c9, actual 0x00000349) ...
Using the Rockchip TPL instead the memory test runs fine.
Is anyone else able to reproduce this?
My guess is the supplied configuration parameters (in arch/arm/dts/rk3328-sdram-lpddr3-1600.dtsi) for the RK3328's SDRAM controller aren't a match for the SpecTek LPDDR3 module on the 4GB ROCK64. (The Rockchip TPL seems to apply a different configuration, but also claims to drive the memory at only 333MHz.)
Assuming I'm on the right track here, what's the likelihood I'd be able to determine from the SpecTek datasheet how the SDRAM controller's configuration should be tweaked to get this working?
participants (6)
-
Carlo Pisani -
Jagan Teki -
Mark Kettenis -
Martin Husemann -
Matwey V. Kornilov -
Simon South