Hello - this is my first message to this list, so I apologize in advance if I've got some of the customs wrong in doing so. I have searched prior messages and am running a very recent build (commit 08cca188c120b8337600091c94dc0e211cd03e10) which incorporates all of the patches relating to similar issues from what I can tell. My environment is running yocto dunfell on kernel 5.4.85.
When I take a stock 2GB CM4 module (empty EMMC, latest EEPROM) and boot via pcie/xhci (USB VL805-based PCIE card on CM4IO board) U-Boot initializes the USB device, finds and runs the U-Boot script, and starts the kernel. However when I take a stock 4GB CM4 module (also empty EMMC, latest EEPROM) I get errors - see console output for both scenarios below.
CM4-2GB Successful Boot: DRAM: 1.9 GiB RPI Compute Module 4 (0xb03140) MMC: mmcnr@7e300000: 1, emmc2@7e340000: 0 In: serial Out: serial Err: serial Net: eth0: ethernet@7d580000 PCIe BRCM: link up, 5.0 Gbps x1 (SSC) starting USB... Bus usb@7e980000: USB DWC2 Bus xhci_pci: Register 5000420 NbrPorts 5 Starting the controller USB XHCI 1.00 scanning bus usb@7e980000 for devces... 1 USB Device(s) found scanning bus xhci_pci for devices... 3 USB Device(s) found scanning usb for storage devices... 1 Storage Device(s) found
Device 0: Vendor: USB Rev: 1.00 Prod: SanDisk 3.2Gen1 Tye: Removable Hard Disk Capacity: 29340.0 MB = 28.6 GB (60088320 x 512) ... is now current device Scanning usb 0:1... Found U-Boot script /boot.scr 616 bytes read in 2 ms (300.8 KiB/s) ## Exeuting script at 02400000 Scanning mmc usb devices 0 1 2 Failed to load 'resin-image-flasher' 24 bytes read in 13 ms (1000 Bytes/s) Card did not respond to voltage select! : -110 Can't set block device Can't set block devic 24 bytes read in 4 ms (5.9 KiB/s) Found resin flasher on usb 0 Loading resinOS_uEnv.txt from usb device 0 partition 1 0 bytes read in 3 ms (0 Bytes/s) Import resinOS_uEnv.txt in environmen Loading extra_uEnv.txt from usb device 0 partition 1 0 bytes read in 2 ms (0 Bytes/s) Import extra_uEnv.txt in environment Loading bootcount.env from us device 0 partition 1 Failed to load 'bootcount.env' No bootcount.env file. Setting bootcount=0 in environment 24 bytes read in 4 ms (5.9 KiB/s) 12468223 bytes read in 112 ms (106.2 MiB/s) Uncompressed size: 28365312 = 0x1B0D200 ## Flattened Device ree blob at 2eff4100 Booting using the fdt blob at 0x2eff4100 Using Device Tree in place at 000000002eff4100, end 00000002f002f80
Starting kernel ...
CM4-4GB Unsuccessful Boot: U-Boot 2021.04-rc2 (Feb 26 2021 - 20:11:08 +0000)
DRAM: 3.8 GiB RPI Compute Module 4 (0xc03140) MMC: mmcnr@7e300000: 1, emmc2@7e340000: 0 In: serial Out: serial Err: serial Net: eth0: ethernet@7d580000 PCIe BRCM: link up, 5.0 Gbps x1 (SSC) starting USB... Bus usb@7e980000: USB DWC2 Bus xhci_pci: Register 5000420 NbrPorts 5 Starting the controller Port not available. scanning bus usb@7e980000 for devices... 1 USB Device(s) found scanning usb for storag devices... 0 Storage Device(s) found "Synchronous Abort" handler, esr 0x96000004 elr: 000000000009c320 lr : 0000000000092090 (reloc) elr: 000000037c76320 lr : 0000000037c6c090 x0 : d519b040aa010000 x1 : 000000000000005c x2 : 0000000000800000 x3 : 0000000037cd340 x4 : b900080152b00000 x5 : 000000000000005c x6 : 0000000037cd3460 x7 : b900080152afff90 x8 : 0000000000000000 x9 : 000000000000008 x10: 000000000000ad84 x11: 0000000037849d8c x12: 000000000000ad88 x13: 0000000037849dd8 x14: 000000003789ef0 x15: 0000000000000002 x16: 0000000037c78438 x17: 0000000000000000 x18: 0000000037855d90 x19: 0000000000000070 x20 0000000037cd2ea0 x21: 0000000000000000 x22: 000000000000000b x23: 00000000ffffffff x24: 0000000000000000 x25: 000000007ce4e64 x26: 0000000037cbe3fe x27: 0000000037cc777b x28: 0000000000000064 x29: 0000000037849460
Code: eb03005f 54ffff3 f9400ca4 17ffffe0 (f9400404) Resetting CPU ...
resetting ...
The closest issue I found in searching the mailing list was this one: https://lists.denx.de/pipermail/u-boot/2020-December/435143.html But I believe these patches are already applied in the version that I am running. Does anyone have any idea what could be causing this? When I limit the 4GB module to use only 3GB of ram (via total_mem=3072 in config.txt) it works, but obviously this is just a workaround and not an actual solution.
I posted this same issue to the RPI forums, and an RPI engineer noted the following which may help to pinpoint the issue:
It's more likely to be related to how u-boot configures the inbound address window (BAR2) depending on the memory configuration. In the latest releases, the firmware patches device-tree to set this @ 16GB. I don't know if u-boot reads this, it's free to ignore it.
IIRC this has to be multiple of the SDRAM size because the bus address matching just masks off bits and 16GB is the maximum size that the bus could support. Previously, the firmware used to do something like pick the smallest multiple of SDRAM size.
Picking older/newer versions of the firmware may help to verify but u-boot should override the number if it wants something else.
This recent FW change allows the Pi4 R6 VLI FW loading (not relevant on CM4) to pickup the BAR2 configuration from the chain-loader / host OS by reading from the PCIe registers. That might have been a blocker for u-boot in the past.