u-boot on RPi4 32bit - issue after upgrading the firmware
Hi!
Recently I've upgraded a video core firmware on my RPi4 and I've noticed that u-boot in ARM 32bit mode stopped working properly or even randomly crashes. I've bisected the firmware releases and the issue appears for the first time in the commit 63b1922311 ("firmware: arm_loader: Update armstubs with those from PR 117 See: https://github.com/raspberrypi/tools/pull/117").
This issue can be easily observed as a failure of the xhci. See the attached log:
Read start4x.elf bytes 2984808 hnd 0x000006e2 sha256 d21bf9d3954fea04 Read fixup4x.dat bytes 8432 hnd 0x000006dd sha256 10d3b073ab232613 0x00a03111 0x00000000 0x0000001f MEM GPU: 76 ARM: 948 TOTAL: 1024 Starting start4x.elf @ 0xfec00200 partition 0 PCI reset +
MESS:00:00:04.800023:0: arasan: arasan_emmc_open MESS:00:00:04.975752:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:04.979162:0: brfs: File read: 2153 bytes MESS:00:00:05.046081:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.121174:0: gpioman: gpioman_get_pin_num: pin DISPLAY_DSI_PORT not defined MESS:00:00:06.128433:0: *** Restart logging MESS:00:00:06.129897:0: brfs: File read: 2153 bytes MESS:00:00:06.148533:0: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead MESS:00:00:06.154474:0: HDMI0: hdmi_pixel_encoding: 300000000 MESS:00:00:06.159948:0: HDMI1: hdmi_pixel_encoding: 300000000 MESS:00:00:06.166901:0: dtb_file 'bcm2711-rpi-4-b.dtb' MESS:00:00:06.178359:0: brfs: File read: /mfs/sd/bcm2711-rpi-4-b.dtb MESS:00:00:06.181601:0: Loading 'bcm2711-rpi-4-b.dtb' to 0x100 size 0xc901 MESS:00:00:06.201386:0: brfs: File read: 51457 bytes MESS:00:00:06.267975:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.270915:0: dtparam: audio=on MESS:00:00:06.283974:0: brfs: File read: 2153 bytes MESS:00:00:06.286148:0: Failed to load overlay 'vc4-fkms-v3d' MESS:00:00:06.291378:0: brfs: File read: /mfs/sd/overlays/vc4-fkms-v3d.dtbo MESS:00:00:06.304096:0: Failed to open command line file 'cmdline.txt' MESS:00:00:07.484256:0: brfs: File read: /mfs/sd/u-boot.bin MESS:00:00:07.486726:0: Loading 'u-boot.bin' to 0x8000 size 0x84cf4 MESS:00:00:07.492726:0: Device tree loaded to 0x2eff3100 (size 0xce24) MESS:00:00:07.500863:0: uart: Set PL011 baud rate to 103448.300000 Hz MESS:00:00:07.508031:0: uart: Baud rate change done... MESS:00:00:07.510053:0:
U-Boot 2021.04-rc2 (Feb 25 2021 - 11:21:44 +0100)
DRAM: 948 MiB RPI 4 Model B (0xa03111) MMC: mmcnr@7e300000: 1, emmc2@7e340000: 0 Loading Environment from FAT... OK In: serial Out: serial Err: serial Net: eth0: ethernet@7d580000 Hit any key to stop autoboot: 0 U-Boot> pci enum PCIe BRCM: link up, 5.0 Gbps x1 (SSC) U-Boot> usb start starting USB... Bus xhci_pci: Host not halted after 16000 microseconds. probe failed, error -16 No working controllers found U-Boot>
Is it a known issue? In ARM64 mode everything works fine, but this not very surprising, because the firmware change in the mentioned commit is related to the ARM 32bit mode. Does anyone have an idea how to fix this issue? I've checked also the latest version of the RPi firmware, but the issue is still there.
Best regards
On Thu, 2021-02-25 at 11:43 +0100, Marek Szyprowski wrote:
Hi!
Recently I've upgraded a video core firmware on my RPi4 and I've noticed that u-boot in ARM 32bit mode stopped working properly or even randomly crashes. I've bisected the firmware releases and the issue appears for the first time in the commit 63b1922311 ("firmware: arm_loader: Update armstubs with those from PR 117 See: https://github.com/raspberrypi/tools/pull/117").
Just for confirmation,
This issue can be easily observed as a failure of the xhci. See the attached log:
Read start4x.elf bytes 2984808 hnd 0x000006e2 sha256 d21bf9d3954fea04 Read fixup4x.dat bytes 8432 hnd 0x000006dd sha256 10d3b073ab232613 0x00a03111 0x00000000 0x0000001f MEM GPU: 76 ARM: 948 TOTAL: 1024 Starting start4x.elf @ 0xfec00200 partition 0 PCI reset
MESS:00:00:04.800023:0: arasan: arasan_emmc_open MESS:00:00:04.975752:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:04.979162:0: brfs: File read: 2153 bytes MESS:00:00:05.046081:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.121174:0: gpioman: gpioman_get_pin_num: pin DISPLAY_DSI_PORT not defined MESS:00:00:06.128433:0: *** Restart logging MESS:00:00:06.129897:0: brfs: File read: 2153 bytes MESS:00:00:06.148533:0: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead MESS:00:00:06.154474:0: HDMI0: hdmi_pixel_encoding: 300000000 MESS:00:00:06.159948:0: HDMI1: hdmi_pixel_encoding: 300000000 MESS:00:00:06.166901:0: dtb_file 'bcm2711-rpi-4-b.dtb' MESS:00:00:06.178359:0: brfs: File read: /mfs/sd/bcm2711-rpi-4-b.dtb MESS:00:00:06.181601:0: Loading 'bcm2711-rpi-4-b.dtb' to 0x100 size 0xc901 MESS:00:00:06.201386:0: brfs: File read: 51457 bytes MESS:00:00:06.267975:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.270915:0: dtparam: audio=on MESS:00:00:06.283974:0: brfs: File read: 2153 bytes MESS:00:00:06.286148:0: Failed to load overlay 'vc4-fkms-v3d' MESS:00:00:06.291378:0: brfs: File read: /mfs/sd/overlays/vc4-fkms-v3d.dtbo MESS:00:00:06.304096:0: Failed to open command line file 'cmdline.txt' MESS:00:00:07.484256:0: brfs: File read: /mfs/sd/u-boot.bin MESS:00:00:07.486726:0: Loading 'u-boot.bin' to 0x8000 size 0x84cf4 MESS:00:00:07.492726:0: Device tree loaded to 0x2eff3100 (size 0xce24) MESS:00:00:07.500863:0: uart: Set PL011 baud rate to 103448.300000 Hz MESS:00:00:07.508031:0: uart: Baud rate change done... MESS:00:00:07.510053:0:
U-Boot 2021.04-rc2 (Feb 25 2021 - 11:21:44 +0100)
Can you try with today's master branch, I'm specially interested in seeing if 56f1bcc4b7fb helps.
DRAM: 948 MiB RPI 4 Model B (0xa03111) MMC: mmcnr@7e300000: 1, emmc2@7e340000: 0 Loading Environment from FAT... OK In: serial Out: serial Err: serial Net: eth0: ethernet@7d580000 Hit any key to stop autoboot: 0 U-Boot> pci enum PCIe BRCM: link up, 5.0 Gbps x1 (SSC) U-Boot> usb start starting USB... Bus xhci_pci: Host not halted after 16000 microseconds. probe failed, error -16 No working controllers found U-Boot>
Is it a known issue? In ARM64 mode everything works fine, but this not very surprising, because the firmware change in the mentioned commit is related to the ARM 32bit mode. Does anyone have an idea how to fix this issue? I've checked also the latest version of the RPi firmware, but the issue is still there.
Best regards
Hi Nicolas,
On 25.02.2021 11:55, Nicolas Saenz Julienne wrote:
On Thu, 2021-02-25 at 11:43 +0100, Marek Szyprowski wrote:
Recently I've upgraded a video core firmware on my RPi4 and I've noticed that u-boot in ARM 32bit mode stopped working properly or even randomly crashes. I've bisected the firmware releases and the issue appears for the first time in the commit 63b1922311 ("firmware: arm_loader: Update armstubs with those from PR 117 See: https://protect2.fireeye.com/v1/url?k=a201e8b0-fd9ad069-a20063ff-0cc47a3356b...").
Just for confirmation,
This issue can be easily observed as a failure of the xhci. See the attached log:
Read start4x.elf bytes 2984808 hnd 0x000006e2 sha256 d21bf9d3954fea04 Read fixup4x.dat bytes 8432 hnd 0x000006dd sha256 10d3b073ab232613 0x00a03111 0x00000000 0x0000001f MEM GPU: 76 ARM: 948 TOTAL: 1024 Starting start4x.elf @ 0xfec00200 partition 0 PCI reset
MESS:00:00:04.800023:0: arasan: arasan_emmc_open MESS:00:00:04.975752:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:04.979162:0: brfs: File read: 2153 bytes MESS:00:00:05.046081:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.121174:0: gpioman: gpioman_get_pin_num: pin DISPLAY_DSI_PORT not defined MESS:00:00:06.128433:0: *** Restart logging MESS:00:00:06.129897:0: brfs: File read: 2153 bytes MESS:00:00:06.148533:0: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead MESS:00:00:06.154474:0: HDMI0: hdmi_pixel_encoding: 300000000 MESS:00:00:06.159948:0: HDMI1: hdmi_pixel_encoding: 300000000 MESS:00:00:06.166901:0: dtb_file 'bcm2711-rpi-4-b.dtb' MESS:00:00:06.178359:0: brfs: File read: /mfs/sd/bcm2711-rpi-4-b.dtb MESS:00:00:06.181601:0: Loading 'bcm2711-rpi-4-b.dtb' to 0x100 size 0xc901 MESS:00:00:06.201386:0: brfs: File read: 51457 bytes MESS:00:00:06.267975:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.270915:0: dtparam: audio=on MESS:00:00:06.283974:0: brfs: File read: 2153 bytes MESS:00:00:06.286148:0: Failed to load overlay 'vc4-fkms-v3d' MESS:00:00:06.291378:0: brfs: File read: /mfs/sd/overlays/vc4-fkms-v3d.dtbo MESS:00:00:06.304096:0: Failed to open command line file 'cmdline.txt' MESS:00:00:07.484256:0: brfs: File read: /mfs/sd/u-boot.bin MESS:00:00:07.486726:0: Loading 'u-boot.bin' to 0x8000 size 0x84cf4 MESS:00:00:07.492726:0: Device tree loaded to 0x2eff3100 (size 0xce24) MESS:00:00:07.500863:0: uart: Set PL011 baud rate to 103448.300000 Hz MESS:00:00:07.508031:0: uart: Baud rate change done... MESS:00:00:07.510053:0:
U-Boot 2021.04-rc2 (Feb 25 2021 - 11:21:44 +0100)
Can you try with today's master branch, I'm specially interested in seeing if 56f1bcc4b7fb helps.
I've checked the mentioned commit, sadly nothing changes:
U-Boot 2021.04-rc2-00041-g56f1bcc4b7 (Feb 25 2021 - 12:05:07 +0100)
DRAM: 948 MiB RPI 4 Model B (0xa03111) MMC: mmcnr@7e300000: 1, emmc2@7e340000: 0 Loading Environment from FAT... OK In: serial Out: serial Err: serial Net: eth0: ethernet@7d580000 Hit any key to stop autoboot: 0 U-Boot> pci enum PCIe BRCM: link up, 5.0 Gbps x1 (SSC) U-Boot> usb start starting USB... Bus xhci_pci: Host not halted after 16000 microseconds. probe failed, error -16 No working controllers found U-Boot>
I've checked that on two RPi4 boards (one with 1GB RAM and one with 8GB).
Best regards
Hi,
On 2/25/21 8:11 PM, Marek Szyprowski wrote:
Hi Nicolas,
On 25.02.2021 11:55, Nicolas Saenz Julienne wrote:
On Thu, 2021-02-25 at 11:43 +0100, Marek Szyprowski wrote:
Recently I've upgraded a video core firmware on my RPi4 and I've noticed that u-boot in ARM 32bit mode stopped working properly or even randomly crashes. I've bisected the firmware releases and the issue appears for the first time in the commit 63b1922311 ("firmware: arm_loader: Update armstubs with those from PR 117 See: https://protect2.fireeye.com/v1/url?k=a201e8b0-fd9ad069-a20063ff-0cc47a3356b...").
Just for confirmation,
This issue can be easily observed as a failure of the xhci. See the attached log:
Read start4x.elf bytes 2984808 hnd 0x000006e2 sha256 d21bf9d3954fea04 Read fixup4x.dat bytes 8432 hnd 0x000006dd sha256 10d3b073ab232613 0x00a03111 0x00000000 0x0000001f MEM GPU: 76 ARM: 948 TOTAL: 1024 Starting start4x.elf @ 0xfec00200 partition 0 PCI reset
MESS:00:00:04.800023:0: arasan: arasan_emmc_open MESS:00:00:04.975752:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:04.979162:0: brfs: File read: 2153 bytes MESS:00:00:05.046081:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.121174:0: gpioman: gpioman_get_pin_num: pin DISPLAY_DSI_PORT not defined MESS:00:00:06.128433:0: *** Restart logging MESS:00:00:06.129897:0: brfs: File read: 2153 bytes MESS:00:00:06.148533:0: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead MESS:00:00:06.154474:0: HDMI0: hdmi_pixel_encoding: 300000000 MESS:00:00:06.159948:0: HDMI1: hdmi_pixel_encoding: 300000000 MESS:00:00:06.166901:0: dtb_file 'bcm2711-rpi-4-b.dtb' MESS:00:00:06.178359:0: brfs: File read: /mfs/sd/bcm2711-rpi-4-b.dtb MESS:00:00:06.181601:0: Loading 'bcm2711-rpi-4-b.dtb' to 0x100 size 0xc901 MESS:00:00:06.201386:0: brfs: File read: 51457 bytes MESS:00:00:06.267975:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.270915:0: dtparam: audio=on MESS:00:00:06.283974:0: brfs: File read: 2153 bytes MESS:00:00:06.286148:0: Failed to load overlay 'vc4-fkms-v3d' MESS:00:00:06.291378:0: brfs: File read: /mfs/sd/overlays/vc4-fkms-v3d.dtbo MESS:00:00:06.304096:0: Failed to open command line file 'cmdline.txt' MESS:00:00:07.484256:0: brfs: File read: /mfs/sd/u-boot.bin MESS:00:00:07.486726:0: Loading 'u-boot.bin' to 0x8000 size 0x84cf4 MESS:00:00:07.492726:0: Device tree loaded to 0x2eff3100 (size 0xce24) MESS:00:00:07.500863:0: uart: Set PL011 baud rate to 103448.300000 Hz MESS:00:00:07.508031:0: uart: Baud rate change done... MESS:00:00:07.510053:0:
U-Boot 2021.04-rc2 (Feb 25 2021 - 11:21:44 +0100)
Can you try with today's master branch, I'm specially interested in seeing if 56f1bcc4b7fb helps.
I've checked the mentioned commit, sadly nothing changes:
U-Boot 2021.04-rc2-00041-g56f1bcc4b7 (Feb 25 2021 - 12:05:07 +0100)
DRAM: 948 MiB RPI 4 Model B (0xa03111) MMC: mmcnr@7e300000: 1, emmc2@7e340000: 0 Loading Environment from FAT... OK In: serial Out: serial Err: serial Net: eth0: ethernet@7d580000 Hit any key to stop autoboot: 0 U-Boot> pci enum PCIe BRCM: link up, 5.0 Gbps x1 (SSC) U-Boot> usb start starting USB... Bus xhci_pci: Host not halted after 16000 microseconds. probe failed, error -16 No working controllers found U-Boot>
I've checked that on two RPi4 boards (one with 1GB RAM and one with 8GB).
It seems that couldn't get hc_length from capbase.
us xhci_pci: XHCI-PCI init hccr ff800000 and hcor ff800000 hc_length 0 xhci_register: dev='xhci_pci', ctrl=3a3679c0, hccr=ff800000, hcor=ff800000
When i have compared with working and not, hc_length is different. hc_length should be to set 0x20 when it's working fine.
I don't have the knowledge of USB..So it's taking too more time for debugging.
Could you share what i do to check more?
Best Regards, Jaehoon Chung
Best regards
Hi
On 04.03.2021 02:33, Jaehoon Chung wrote:
On 2/25/21 8:11 PM, Marek Szyprowski wrote:
On 25.02.2021 11:55, Nicolas Saenz Julienne wrote:
On Thu, 2021-02-25 at 11:43 +0100, Marek Szyprowski wrote:
Recently I've upgraded a video core firmware on my RPi4 and I've noticed that u-boot in ARM 32bit mode stopped working properly or even randomly crashes. I've bisected the firmware releases and the issue appears for the first time in the commit 63b1922311 ("firmware: arm_loader: Update armstubs with those from PR 117 See: https://protect2.fireeye.com/v1/url?k=a201e8b0-fd9ad069-a20063ff-0cc47a3356b...").
Just for confirmation,
This issue can be easily observed as a failure of the xhci. See the attached log:
Read start4x.elf bytes 2984808 hnd 0x000006e2 sha256 d21bf9d3954fea04 Read fixup4x.dat bytes 8432 hnd 0x000006dd sha256 10d3b073ab232613 0x00a03111 0x00000000 0x0000001f MEM GPU: 76 ARM: 948 TOTAL: 1024 Starting start4x.elf @ 0xfec00200 partition 0 PCI reset
MESS:00:00:04.800023:0: arasan: arasan_emmc_open MESS:00:00:04.975752:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:04.979162:0: brfs: File read: 2153 bytes MESS:00:00:05.046081:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.121174:0: gpioman: gpioman_get_pin_num: pin DISPLAY_DSI_PORT not defined MESS:00:00:06.128433:0: *** Restart logging MESS:00:00:06.129897:0: brfs: File read: 2153 bytes MESS:00:00:06.148533:0: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead MESS:00:00:06.154474:0: HDMI0: hdmi_pixel_encoding: 300000000 MESS:00:00:06.159948:0: HDMI1: hdmi_pixel_encoding: 300000000 MESS:00:00:06.166901:0: dtb_file 'bcm2711-rpi-4-b.dtb' MESS:00:00:06.178359:0: brfs: File read: /mfs/sd/bcm2711-rpi-4-b.dtb MESS:00:00:06.181601:0: Loading 'bcm2711-rpi-4-b.dtb' to 0x100 size 0xc901 MESS:00:00:06.201386:0: brfs: File read: 51457 bytes MESS:00:00:06.267975:0: brfs: File read: /mfs/sd/config.txt MESS:00:00:06.270915:0: dtparam: audio=on MESS:00:00:06.283974:0: brfs: File read: 2153 bytes MESS:00:00:06.286148:0: Failed to load overlay 'vc4-fkms-v3d' MESS:00:00:06.291378:0: brfs: File read: /mfs/sd/overlays/vc4-fkms-v3d.dtbo MESS:00:00:06.304096:0: Failed to open command line file 'cmdline.txt' MESS:00:00:07.484256:0: brfs: File read: /mfs/sd/u-boot.bin MESS:00:00:07.486726:0: Loading 'u-boot.bin' to 0x8000 size 0x84cf4 MESS:00:00:07.492726:0: Device tree loaded to 0x2eff3100 (size 0xce24) MESS:00:00:07.500863:0: uart: Set PL011 baud rate to 103448.300000 Hz MESS:00:00:07.508031:0: uart: Baud rate change done... MESS:00:00:07.510053:0:
U-Boot 2021.04-rc2 (Feb 25 2021 - 11:21:44 +0100)
Can you try with today's master branch, I'm specially interested in seeing if 56f1bcc4b7fb helps.
I've checked the mentioned commit, sadly nothing changes:
U-Boot 2021.04-rc2-00041-g56f1bcc4b7 (Feb 25 2021 - 12:05:07 +0100)
DRAM: 948 MiB RPI 4 Model B (0xa03111) MMC: mmcnr@7e300000: 1, emmc2@7e340000: 0 Loading Environment from FAT... OK In: serial Out: serial Err: serial Net: eth0: ethernet@7d580000 Hit any key to stop autoboot: 0 U-Boot> pci enum PCIe BRCM: link up, 5.0 Gbps x1 (SSC) U-Boot> usb start starting USB... Bus xhci_pci: Host not halted after 16000 microseconds. probe failed, error -16 No working controllers found U-Boot>
I've checked that on two RPi4 boards (one with 1GB RAM and one with 8GB).
It seems that couldn't get hc_length from capbase.
us xhci_pci: XHCI-PCI init hccr ff800000 and hcor ff800000 hc_length 0 xhci_register: dev='xhci_pci', ctrl=3a3679c0, hccr=ff800000, hcor=ff800000
When i have compared with working and not, hc_length is different. hc_length should be to set 0x20 when it's working fine.
I don't have the knowledge of USB..So it's taking too more time for debugging.
Could you share what i do to check more?
I've finally had some time to analyze it more. It looks that 0xff800000 base address in ARM 32bit virtual address space chosen for XHCI PCI device conflicts with the updated ARM/VideoCore firmware. I have no idea why, but switching to 0xffc00000 fixed the issue.
Best regards
participants (3)
-
Jaehoon Chung -
Marek Szyprowski -
Nicolas Saenz Julienne