XHCI bringup on the Raspberry Pi 4
Hi All, I'm working on enabling the VIA805 XCHI controller found on the new Raspberry Pi 4. The controller sits behind a PCIe bus, which I've already implemented[1] and will submit once the XCHI issues are resolved, as it's worthless otherwise.
The XHCI initialization gets stuck after issuing the fist 'enable slot' command. I've been reviewing the whole init process and comparing it to Linux's for days without finding anything fishy. DMA constraints, which are important on the RPi are mantained, and on top of that, I can confirm DMA trough PCIe works fine as I see two 'port status change' events available in the ring before completly stalling. Also note that, unsurprisingly, the CRR bit in the CRCR register (which marks the running state of the Command Ring state machine) is never set after ringing the relevant doorbell.
I'm clueless at this point, I figure the VIA805 is sensitive to the ordering of some of the operations we perform in u-boot, or worse, the timing. For example, I tried replicanting Linux's behaviour with regard to 'port status change' events, processing them before calling the 'enable slot' command. I also tried to mimic Linux by enabling port-0's power (the USB3 port) before starting the HC. Again, no luck.
I attached the usb/xhci debug output, any ideas on where to look will be apreciated.
Regards, Nicolas
On 4/1/20 7:30 PM, Nicolas Saenz Julienne wrote:
Hi All,
Hi,
I'm working on enabling the VIA805 XCHI controller found on the new Raspberry Pi 4. The controller sits behind a PCIe bus, which I've already implemented[1] and will submit once the XCHI issues are resolved, as it's worthless otherwise.
The XHCI initialization gets stuck after issuing the fist 'enable slot' command. I've been reviewing the whole init process and comparing it to Linux's for days without finding anything fishy. DMA constraints, which are important on the RPi are mantained, and on top of that, I can confirm DMA trough PCIe works fine as I see two 'port status change' events available in the ring before completly stalling. Also note that, unsurprisingly, the CRR bit in the CRCR register (which marks the running state of the Command Ring state machine) is never set after ringing the relevant doorbell.
I'm clueless at this point, I figure the VIA805 is sensitive to the ordering of some of the operations we perform in u-boot, or worse, the timing. For example, I tried replicanting Linux's behaviour with regard to 'port status change' events, processing them before calling the 'enable slot' command. I also tried to mimic Linux by enabling port-0's power (the USB3 port) before starting the HC. Again, no luck.
I attached the usb/xhci debug output, any ideas on where to look will be apreciated.
Try disabling caches (dcache off ; icache off) and see if it magically starts working. That comes to mind.
On Wed, 2020-04-01 at 20:50 +0200, Marek Vasut wrote:
On 4/1/20 7:30 PM, Nicolas Saenz Julienne wrote:
Hi All,
Hi,
I'm working on enabling the VIA805 XCHI controller found on the new Raspberry Pi 4. The controller sits behind a PCIe bus, which I've already implemented[1] and will submit once the XCHI issues are resolved, as it's worthless otherwise.
The XHCI initialization gets stuck after issuing the fist 'enable slot' command. I've been reviewing the whole init process and comparing it to Linux's for days without finding anything fishy. DMA constraints, which are important on the RPi are mantained, and on top of that, I can confirm DMA trough PCIe works fine as I see two 'port status change' events available in the ring before completly stalling. Also note that, unsurprisingly, the CRR bit in the CRCR register (which marks the running state of the Command Ring state machine) is never set after ringing the relevant doorbell.
I'm clueless at this point, I figure the VIA805 is sensitive to the ordering of some of the operations we perform in u-boot, or worse, the timing. For example, I tried replicanting Linux's behaviour with regard to 'port status change' events, processing them before calling the 'enable slot' command. I also tried to mimic Linux by enabling port-0's power (the USB3 port) before starting the HC. Again, no luck.
I attached the usb/xhci debug output, any ideas on where to look will be apreciated.
Try disabling caches (dcache off ; icache off) and see if it magically starts working. That comes to mind.
Same outcome :(
On 4/1/20 10:13 PM, Nicolas Saenz Julienne wrote:
On Wed, 2020-04-01 at 20:50 +0200, Marek Vasut wrote:
On 4/1/20 7:30 PM, Nicolas Saenz Julienne wrote:
Hi All,
Hi,
I'm working on enabling the VIA805 XCHI controller found on the new Raspberry Pi 4. The controller sits behind a PCIe bus, which I've already implemented[1] and will submit once the XCHI issues are resolved, as it's worthless otherwise.
The XHCI initialization gets stuck after issuing the fist 'enable slot' command. I've been reviewing the whole init process and comparing it to Linux's for days without finding anything fishy. DMA constraints, which are important on the RPi are mantained, and on top of that, I can confirm DMA trough PCIe works fine as I see two 'port status change' events available in the ring before completly stalling. Also note that, unsurprisingly, the CRR bit in the CRCR register (which marks the running state of the Command Ring state machine) is never set after ringing the relevant doorbell.
I'm clueless at this point, I figure the VIA805 is sensitive to the ordering of some of the operations we perform in u-boot, or worse, the timing. For example, I tried replicanting Linux's behaviour with regard to 'port status change' events, processing them before calling the 'enable slot' command. I also tried to mimic Linux by enabling port-0's power (the USB3 port) before starting the HC. Again, no luck.
I attached the usb/xhci debug output, any ideas on where to look will be apreciated.
Try disabling caches (dcache off ; icache off) and see if it magically starts working. That comes to mind.
Same outcome :(
Hmmm, no idea then, maybe others have some.
I presume PCIe works correctly , right?
On Wed, 2020-04-01 at 22:14 +0200, Marek Vasut wrote:
On 4/1/20 10:13 PM, Nicolas Saenz Julienne wrote:
On Wed, 2020-04-01 at 20:50 +0200, Marek Vasut wrote:
On 4/1/20 7:30 PM, Nicolas Saenz Julienne wrote:
Hi All,
Hi,
I'm working on enabling the VIA805 XCHI controller found on the new Raspberry Pi 4. The controller sits behind a PCIe bus, which I've already implemented[1] and will submit once the XCHI issues are resolved, as it's worthless otherwise.
The XHCI initialization gets stuck after issuing the fist 'enable slot' command. I've been reviewing the whole init process and comparing it to Linux's for days without finding anything fishy. DMA constraints, which are important on the RPi are mantained, and on top of that, I can confirm DMA trough PCIe works fine as I see two 'port status change' events available in the ring before completly stalling. Also note that, unsurprisingly, the CRR bit in the CRCR register (which marks the running state of the Command Ring state machine) is never set after ringing the relevant doorbell.
I'm clueless at this point, I figure the VIA805 is sensitive to the ordering of some of the operations we perform in u-boot, or worse, the timing. For example, I tried replicanting Linux's behaviour with regard to 'port status change' events, processing them before calling the 'enable slot' command. I also tried to mimic Linux by enabling port-0's power (the USB3 port) before starting the HC. Again, no luck.
I attached the usb/xhci debug output, any ideas on where to look will be apreciated.
Try disabling caches (dcache off ; icache off) and see if it magically starts working. That comes to mind.
Same outcome :(
Hmmm, no idea then, maybe others have some.
I presume PCIe works correctly , right?
I was also responsible for the Linux version of the PCIe driver so I know it fairly well. I also double-checked all pcie-brcmstb's registers have the expected values.
From the USB perspective, I'm both able to map XHCI's registers into memory and get some output out of the event ring, which implies XHCI accessed memory trough the bus. On top of that if something went wrong with PCI there is a register that should point it out, Host System Error (HSE), which so far has been silent.
Overall I'm confident PCIe is fine.
Regards, Nicolas
On 4/1/20 10:47 PM, Nicolas Saenz Julienne wrote:
On Wed, 2020-04-01 at 22:14 +0200, Marek Vasut wrote:
On 4/1/20 10:13 PM, Nicolas Saenz Julienne wrote:
On Wed, 2020-04-01 at 20:50 +0200, Marek Vasut wrote:
On 4/1/20 7:30 PM, Nicolas Saenz Julienne wrote:
Hi All,
Hi,
I'm working on enabling the VIA805 XCHI controller found on the new Raspberry Pi 4. The controller sits behind a PCIe bus, which I've already implemented[1] and will submit once the XCHI issues are resolved, as it's worthless otherwise.
The XHCI initialization gets stuck after issuing the fist 'enable slot' command. I've been reviewing the whole init process and comparing it to Linux's for days without finding anything fishy. DMA constraints, which are important on the RPi are mantained, and on top of that, I can confirm DMA trough PCIe works fine as I see two 'port status change' events available in the ring before completly stalling. Also note that, unsurprisingly, the CRR bit in the CRCR register (which marks the running state of the Command Ring state machine) is never set after ringing the relevant doorbell.
I'm clueless at this point, I figure the VIA805 is sensitive to the ordering of some of the operations we perform in u-boot, or worse, the timing. For example, I tried replicanting Linux's behaviour with regard to 'port status change' events, processing them before calling the 'enable slot' command. I also tried to mimic Linux by enabling port-0's power (the USB3 port) before starting the HC. Again, no luck.
I attached the usb/xhci debug output, any ideas on where to look will be apreciated.
Try disabling caches (dcache off ; icache off) and see if it magically starts working. That comes to mind.
Same outcome :(
Hmmm, no idea then, maybe others have some.
I presume PCIe works correctly , right?
I was also responsible for the Linux version of the PCIe driver so I know it fairly well. I also double-checked all pcie-brcmstb's registers have the expected values.
From the USB perspective, I'm both able to map XHCI's registers into memory and get some output out of the event ring, which implies XHCI accessed memory trough the bus. On top of that if something went wrong with PCI there is a register that should point it out, Host System Error (HSE), which so far has been silent.
Overall I'm confident PCIe is fine.
OK, then I hope Bin can help here, he's the xHCI expert.
Hi Nicolas,
On Thu, Apr 2, 2020 at 1:30 AM Nicolas Saenz Julienne nsaenzjulienne@suse.de wrote:
Hi All, I'm working on enabling the VIA805 XCHI controller found on the new Raspberry Pi 4. The controller sits behind a PCIe bus, which I've already implemented[1] and will submit once the XCHI issues are resolved, as it's worthless otherwise.
The XHCI initialization gets stuck after issuing the fist 'enable slot' command. I've been reviewing the whole init process and comparing it to Linux's for days without finding anything fishy. DMA constraints, which are important on the RPi are mantained, and on top of that, I can confirm DMA trough PCIe works fine as I see two 'port status change' events available in the ring before completly stalling. Also note that, unsurprisingly, the CRR bit in the CRCR register (which marks the running state of the Command Ring state machine) is never set after ringing the relevant doorbell.
This is probably caused by the required structure setup by U-Boot is viewed as buggy by the xHCI controller, hence there is no response to the first "enable slot" command.
Could you please compare all the data structures, with the one set up by the Linux kernel?
BTW: does Linux kernel xHCI driver work on this RPI 4 board?
I'm clueless at this point, I figure the VIA805 is sensitive to the ordering of some of the operations we perform in u-boot, or worse, the timing. For example, I tried replicanting Linux's behaviour with regard to 'port status change' events, processing them before calling the 'enable slot' command. I also tried to mimic Linux by enabling port-0's power (the USB3 port) before starting the HC. Again, no luck.
I attached the usb/xhci debug output, any ideas on where to look will be apreciated.
Regards, Bin
I'm working on enabling the VIA805 XCHI controller found on the new Raspberry Pi 4. The controller sits behind a PCIe bus, which I've already implemented[1] and will submit once the XCHI issues are resolved, as it's worthless otherwise.
The XHCI initialization gets stuck after issuing the fist 'enable slot' command. I've been reviewing the whole init process and comparing it to Linux's for days without finding anything fishy. DMA constraints, which are important on the RPi are mantained, and on top of that, I can confirm DMA trough PCIe works fine as I see two 'port status change' events available in the ring before completly stalling. Also note that, unsurprisingly, the CRR bit in the CRCR register (which marks the running state of the Command Ring state machine) is never set after ringing the relevant doorbell.
This is probably caused by the required structure setup by U-Boot is viewed as buggy by the xHCI controller, hence there is no response to the first "enable slot" command.
Could you please compare all the data structures, with the one set up by the Linux kernel?
BTW: does Linux kernel xHCI driver work on this RPI 4 board?
Yes, it uses the standard xhci-hcd controller driver, it's a fairly vanilla VIA controller.
Hi Bin, sorry for the late reply but I was off for easter.
On Fri, 2020-04-03 at 13:54 +0800, Bin Meng wrote:
Hi Nicolas, This is probably caused by the required structure setup by U-Boot is viewed as buggy by the xHCI controller, hence there is no response to the first "enable slot" command.
Yes, that's my impression too. That said I can't seem to find the faulty config.
Since I wasn't all that clear on my previous mail, this is a VIA805 chip, directly soldered on the board and interfaced with trough RPi4's PCIe bus (the PCIe bus isn't exposed, so it's the only user). The Linux version of this works fine, and is already supported upstream.
Could you please compare all the data structures, with the one set up by the Linux kernel?
I've attached some logs comparing u-boot's and Linux's view of the registers. I dumped them before and after calling the 'enable slot' command. I left all the debuging information just in case it's helpful and also attached some patches to show my changes.
Note that I thoughfully compared them myself. The only odd thing I found was "USBSTS: 0x10" (Port Detect Change is set), which isn't the case in Linux. I tried to clear it at different places (before and after changing the individual port status), without effect.
That's all. If the data I sent is a little bit too dense, please tell me and I'll remove all the less important stuff :).
Regards, Nicolas
Hi Nicolas,
On Mon, Apr 13, 2020 at 11:45 PM Nicolas Saenz Julienne nsaenzjulienne@suse.de wrote:
Hi Bin, sorry for the late reply but I was off for easter.
On Fri, 2020-04-03 at 13:54 +0800, Bin Meng wrote:
Hi Nicolas, This is probably caused by the required structure setup by U-Boot is viewed as buggy by the xHCI controller, hence there is no response to the first "enable slot" command.
Yes, that's my impression too. That said I can't seem to find the faulty config.
Since I wasn't all that clear on my previous mail, this is a VIA805 chip, directly soldered on the board and interfaced with trough RPi4's PCIe bus (the PCIe bus isn't exposed, so it's the only user). The Linux version of this works fine, and is already supported upstream.
Could you please compare all the data structures, with the one set up by the Linux kernel?
I've attached some logs comparing u-boot's and Linux's view of the registers. I dumped them before and after calling the 'enable slot' command. I left all the debuging information just in case it's helpful and also attached some patches to show my changes.
Note that I thoughfully compared them myself. The only odd thing I found was "USBSTS: 0x10" (Port Detect Change is set), which isn't the case in Linux. I tried to clear it at different places (before and after changing the individual port status), without effect.
That's all. If the data I sent is a little bit too dense, please tell me and I'll remove all the less important stuff :).
The following patch should fix this issue you were seeing. Please try. http://patchwork.ozlabs.org/project/uboot/patch/20200421165059.19394-2-s.naw...
Regards, Bin
participants (4)
-
Bin Meng -
Marek Vasut -
Nicolas Saenz Julienne -
Peter Robinson