[U-Boot] [u-boot][xhci] Dealing with XHCI controllers with spurious success
Hi All,
I'm running into a problem with the XHCI driver on our SOC where I am getting spurious success TRBs after a short packet response. In this particular case, the driver for a USB network controller has a receive buffer that crosses a 64K boundary. This results in two TRBs being created to handle this with the first TRB having the chain and ISP bits set and the second having the IOC and ISP bits set as expected in this case.
The network controller has a short response (as is expected) followed by a success response event (which is also valid) for the second TRB. The U-Boot XHCI driver, however, does not handle this properly. The xhci_bulk_tx function only looks at the first response and does not dequeue the second success response. The problem is when the next transaction occurs it dequeues the success response from the previous transaction so the responses events no longer match the TRBs. At some point the network driver goes from receive to transmit and at this point the driver panics because the event is still a response from a previous receive event and not the transmit event causing the endpoints to mismatch. This only happens in the two TRB case due to the receive buffer crossing a 64K boundary. The short packet response points to the first TRB and the success points to the second TRB.
In the Linux driver I see the XHCI_SPURIOUS_SUCCESS flag is available. I'm not certain if this applies to our hardware or not. In any case, this could be solved in one of two ways. One way to solve this is to make sure that any driver where a short response is possible to not have the buffer cross a 64K boundary. The other (and correct) solution is to have the XHCI driver handle this case. If a short packet response is received and there are multiple TRBs involved, check if there is an additional success response that points to the last TRB.
To me it seems valid that the two response events are received. The specification only says that there will be one response if the TRB has both the ISP and IOC bits set, but this is not the case. I came across this commit in the Linux kernel which seems to address this case: https://kernel.opensuse.org/cgit/kernel/commit/?id=ad808333d8201d53075a11bc8...
-Aaron
Hi Aaron,
On Thu, Jul 11, 2019 at 12:50 PM Aaron Williams awilliams@marvell.com wrote:
Hi All,
I'm running into a problem with the XHCI driver on our SOC where I am getting spurious success TRBs after a short packet response. In this particular case, the driver for a USB network controller has a receive buffer that crosses a 64K boundary. This results in two TRBs being created to handle this with the first TRB having the chain and ISP bits set and the second having the IOC and ISP bits set as expected in this case.
The network controller has a short response (as is expected) followed by a success response event (which is also valid) for the second TRB. The U-Boot XHCI driver, however, does not handle this properly. The xhci_bulk_tx function only looks at the first response and does not dequeue the second success response. The problem is when the next transaction occurs it dequeues the success response from the previous transaction so the responses events no longer match the TRBs. At some point the network driver goes from receive to transmit and at this point the driver panics because the event is still a response from a previous receive event and not the transmit event causing the endpoints to mismatch. This only happens in the two TRB case due to the receive buffer crossing a 64K boundary. The short packet response points to the first TRB and the success points to the second TRB.
In the Linux driver I see the XHCI_SPURIOUS_SUCCESS flag is available. I'm not certain if this applies to our hardware or not. In any case, this could be solved in one of two ways. One way to solve this is to make sure that any driver where a short response is possible to not have the buffer cross a 64K boundary. The other (and correct) solution is to have the XHCI driver handle this case. If a short packet response is received and there are multiple TRBs involved, check if there is an additional success response that points to the last TRB.
To me it seems valid that the two response events are received. The specification only says that there will be one response if the TRB has both the ISP and IOC bits set, but this is not the case. I came across this commit in the Linux kernel which seems to address this case: https://kernel.opensuse.org/cgit/kernel/commit/?id=ad808333d8201d53075a11bc8...
Are you using an Intel SoC? From the commit you gave, it looks it only applies to a specific Intel PCH chipset.
Regards, Bin
No, it's our Marvell OcteonTX2 SoC. I put in what I thought was a fix, but I'm still working on it. This issue only shows up when using a USB Ethernet device where the rx buffer crosses a 64K boundary and the packet is received in the first buffer. I don't know how other hardware behaves in this situation. The case I'm seeing this is where the receive buffer starts at 128 bytes away from a 64K boundary so two receive TRBs are used. The first is for 128 bytes and the other is for the rest of the buffer. This is with the asix88179.
I am seeing some other strange behavior which is leading me to believe there may be some underlying hardware issue which I am looking into. In my case I'm seeing the transfer event response length with a short packet (13) saying it's 48 bytes (the size of the ARP packet) instead of 5K - 48 which is what it looks like the code expects.
I am in contact now with one of our hardware designers who can hopefully answer what's going on.
-Aaron
From: Bin Meng bmeng.cn@gmail.com
Sent: Wednesday, July 10, 2019 9:57 PM
To: Aaron Williams
Cc: u-boot@lists.denx.de
Subject: [EXT] Re: [U-Boot] [u-boot][xhci] Dealing with XHCI controllers with spurious success
External Email
----------------------------------------------------------------------
Hi Aaron,
On Thu, Jul 11, 2019 at 12:50 PM Aaron Williams awilliams@marvell.com wrote:
Hi All,
I'm running into a problem with the XHCI driver on our SOC where I am getting spurious success TRBs after a short packet response. In this particular case, the driver for a USB network controller has a receive buffer that crosses a 64K boundary. This results
in two TRBs being created to handle this with the first TRB having the chain and ISP bits set and the second having the IOC and ISP bits set as expected in this case.
The network controller has a short response (as is expected) followed by a success response event (which is also valid) for the second TRB. The U-Boot XHCI driver, however, does not handle this properly. The xhci_bulk_tx function only looks at the first
response and does not dequeue the second success response. The problem is when the next transaction occurs it dequeues the success response from the previous transaction so the responses events no longer match the TRBs. At some point the network driver goes from receive to transmit and at this point the driver panics because the event is still a response from a previous receive event and not the transmit event causing the endpoints to mismatch. This only happens in the two TRB case due to the receive buffer crossing a 64K boundary. The short packet response points to the first TRB and the success points to the second TRB.
In the Linux driver I see the XHCI_SPURIOUS_SUCCESS flag is available. I'm not certain if this applies to our hardware or not. In any case, this could be solved in one of two ways. One way to solve this is to make sure that any driver where a short response
is possible to not have the buffer cross a 64K boundary. The other (and correct) solution is to have the XHCI driver handle this case. If a short packet response is received and there are multiple TRBs involved, check if there is an additional success response that points to the last TRB.
To me it seems valid that the two response events are received. The specification only says that there will be one response if the TRB has both the ISP and IOC bits set, but this is not the case. I came across this commit in the Linux kernel which seems to
address this case: https://kernel.opensuse.org/cgit/kernel/commit/?id=ad808333d8201d53075a11bc8...
Are you using an Intel SoC? From the commit you gave, it looks it only
applies to a specific Intel PCH chipset.
Regards,
Bin
Hi Bin,
I have some more data. It looks like U-Boot is not properly handling the short transfer case according to section 4.10.1.1.2 in the XHCI 3.1 specification. In this case, when the data fits in the first TRB, two transfer event TRBs will be generated. The first one will indicate that there is a short packet and points to the first TRB and the length is calculated based on the TRB pointed to, not the total TRB. A second success TRB is also received.
This seems consistent with what I am seeing. U-Boot does not handle this at all. It expects there to always be a single transfer event TRB with the length calculated based on the total length, not the TRB length. I can re-work this so that short packets can be better handled.
The easiest way to test this is with a USB Ethernet dongle then modifying drivers/usb/eth/usb_ether.c to force it to allocate a buffer that will span 64K where packets fit in the first half. I have attached this patch to force the XHCI problem to manifest itself.
-Aaron
On Friday, July 12, 2019 8:51:41 PM PDT Aaron Williams wrote:
No, it's our Marvell OcteonTX2 SoC. I put in what I thought was a fix, but I'm still working on it. This issue only shows up when using a USB Ethernet device where the rx buffer crosses a 64K boundary and the packet is received in the first buffer. I don't know how other hardware behaves in this situation. The case I'm seeing this is where the receive buffer starts at 128 bytes away from a 64K boundary so two receive TRBs are used. The first is for 128 bytes and the other is for the rest of the buffer. This is with the asix88179.
I am seeing some other strange behavior which is leading me to believe there may be some underlying hardware issue which I am looking into. In my case I'm seeing the transfer event response length with a short packet (13) saying it's 48 bytes (the size of the ARP packet) instead of 5K - 48 which is what it looks like the code expects.
I am in contact now with one of our hardware designers who can hopefully answer what's going on.
-Aaron
From: Bin Meng bmeng.cn@gmail.com
Sent: Wednesday, July 10, 2019 9:57 PM
To: Aaron Williams
Cc: u-boot@lists.denx.de
Subject: [EXT] Re: [U-Boot] [u-boot][xhci] Dealing with XHCI controllers with spurious success
External Email
Hi Aaron,
On Thu, Jul 11, 2019 at 12:50 PM Aaron Williams awilliams@marvell.com
wrote:
Hi All,
I'm running into a problem with the XHCI driver on our SOC where I am getting spurious success TRBs after a short packet response. In this particular case, the driver for a USB network controller has a receive buffer that crosses a 64K boundary. This results
in two TRBs being created to handle this with the first TRB having the chain and ISP bits set and the second having the IOC and ISP bits set as expected in this case.
The network controller has a short response (as is expected) followed by a success response event (which is also valid) for the second TRB. The U-Boot XHCI driver, however, does not handle this properly. The xhci_bulk_tx function only looks at the first
response and does not dequeue the second success response. The problem is when the next transaction occurs it dequeues the success response from the previous transaction so the responses events no longer match the TRBs. At some point the network driver goes from receive to transmit and at this point the driver panics because the event is still a response from a previous receive event and not the transmit event causing the endpoints to mismatch. This only happens in the two TRB case due to the receive buffer crossing a 64K boundary. The short packet response points to the first TRB and the success points to the second TRB.
In the Linux driver I see the XHCI_SPURIOUS_SUCCESS flag is available. I'm not certain if this applies to our hardware or not. In any case, this could be solved in one of two ways. One way to solve this is to make sure that any driver where a short response
is possible to not have the buffer cross a 64K boundary. The other (and correct) solution is to have the XHCI driver handle this case. If a short packet response is received and there are multiple TRBs involved, check if there is an additional success response that points to the last TRB.
To me it seems valid that the two response events are received. The specification only says that there will be one response if the TRB has both the ISP and IOC bits set, but this is not the case. I came across this commit in the Linux kernel which seems to
address this case: https://kernel.opensuse.org/cgit/kernel/commit/?id=ad808333d8201d53075a11bc8 dd83b81f3d68f0b
Are you using an Intel SoC? From the commit you gave, it looks it only
applies to a specific Intel PCH chipset.
Regards,
Bin
U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Hi Aaron,
On Tue, Sep 10, 2019 at 7:53 AM Aaron Williams awilliams@marvell.com wrote:
Hi Bin,
I have some more data. It looks like U-Boot is not properly handling the short transfer case according to section 4.10.1.1.2 in the XHCI 3.1 specification. In this case, when the data fits in the first TRB, two transfer event TRBs will be generated. The first one will indicate that there is a short packet and points to the first TRB and the length is calculated based on the TRB pointed to, not the total TRB. A second success TRB is also received.
This seems consistent with what I am seeing. U-Boot does not handle this at all. It expects there to always be a single transfer event TRB with the length calculated based on the total length, not the TRB length. I can re-work this so that short packets can be better handled.
Thank you very much for your efforts on analyzing this. It looks short packet indeed is not correctly handled in the bulk transfer. But in the control transfer, looks it is being handled. Could you please adding the same piece of codes to the xhci_bulk_tx() to see if it fixes the issue?
The easiest way to test this is with a USB Ethernet dongle then modifying drivers/usb/eth/usb_ether.c to force it to allocate a buffer that will span 64K where packets fit in the first half. I have attached this patch to force the XHCI problem to manifest itself.
Regards, Bin
participants (2)
-
Aaron Williams -
Bin Meng