[PATCH] tqma6: Fix DDR configuration
Initially investigating a Linux network issue causing a lot of drop and poor network performances on a custom system based on a TQMA6A module (based on an iMX6Q), [1st link below].
I eventually correlated my observations with a contention at the NIC level when in concurrency with the graphics pipeline. Troubleshooting this in the kernel lead to disabling DMA bursts accesses made by the IPU in order to avoid triggering the QoS at the interconnect level, reducing from 50 to 10% the drop rate on eth0, [2nd link below]. The solution worked on my setup but not on others, which still suffered from abnormally high drop rates even with this "fix".
After looking a while into TQ Systems BSP I figured out a number of differences in recent U-Boot out-of-tree patches they had in their repository [3rd link]. Parsing the differences one after the other lead me to this final solution.
The reset pad of the DDR controller was apparently misconfigured, Bit 18-19 picturing the "DDR select field". The current value b11 is reserved. The only defined value as of version 6 of the iMX6Q manual was b00 "DDR3 and LPDDR2 mode". In practice no register difference has been spotted after changing this configuration but all issues tracked thus far just vanished. All previous fixes have been proven irrelevant. Just clearing this field solved all our network issues and the drop rate as measured by iperf3 felt back to 0%.
Link: https://lore.kernel.org/netdev/20231012193410.3d1812cf@xps-13/ Link: https://lists.freedesktop.org/archives/dri-devel/2023-October/428251.html Link: https://github.com/tq-systems/u-boot-tqmaxx/commit/15eb6abbefbf6916c28467b85... Signed-off-by: Miquel Raynal miquel.raynal@bootlin.com --- board/tq/tqma6/tqma6q.cfg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/board/tq/tqma6/tqma6q.cfg b/board/tq/tqma6/tqma6q.cfg index a49489aed3f..a345c4de93d 100644 --- a/board/tq/tqma6/tqma6q.cfg +++ b/board/tq/tqma6/tqma6q.cfg @@ -36,7 +36,7 @@ DATA 4, MX6_IOM_DRAM_SDCLK_1, 0x00008030 DATA 4, MX6_IOM_DRAM_CAS, 0x00008030 DATA 4, MX6_IOM_DRAM_RAS, 0x00008030 DATA 4, MX6_IOM_GRP_ADDDS, 0x00000030 -DATA 4, MX6_IOM_DRAM_RESET, 0x000C3030 +DATA 4, MX6_IOM_DRAM_RESET, 0x00003030 DATA 4, MX6_IOM_DRAM_SDCKE0, 0x00003000 DATA 4, MX6_IOM_DRAM_SDCKE1, 0x00000000 DATA 4, MX6_IOM_DRAM_SDBA2, 0x00000000
On Fri, Nov 17, 2023 at 1:50 PM Miquel Raynal miquel.raynal@bootlin.com wrote:
Initially investigating a Linux network issue causing a lot of drop and poor network performances on a custom system based on a TQMA6A module (based on an iMX6Q), [1st link below].
I eventually correlated my observations with a contention at the NIC level when in concurrency with the graphics pipeline. Troubleshooting this in the kernel lead to disabling DMA bursts accesses made by the IPU in order to avoid triggering the QoS at the interconnect level, reducing from 50 to 10% the drop rate on eth0, [2nd link below]. The solution worked on my setup but not on others, which still suffered from abnormally high drop rates even with this "fix".
After looking a while into TQ Systems BSP I figured out a number of differences in recent U-Boot out-of-tree patches they had in their repository [3rd link]. Parsing the differences one after the other lead me to this final solution.
The reset pad of the DDR controller was apparently misconfigured, Bit 18-19 picturing the "DDR select field". The current value b11 is reserved. The only defined value as of version 6 of the iMX6Q manual was b00 "DDR3 and LPDDR2 mode". In practice no register difference has been spotted after changing this configuration but all issues tracked thus far just vanished. All previous fixes have been proven irrelevant. Just clearing this field solved all our network issues and the drop rate as measured by iperf3 felt back to 0%.
Link: https://lore.kernel.org/netdev/20231012193410.3d1812cf@xps-13/ Link: https://lists.freedesktop.org/archives/dri-devel/2023-October/428251.html Link: https://github.com/tq-systems/u-boot-tqmaxx/commit/15eb6abbefbf6916c28467b85... Signed-off-by: Miquel Raynal miquel.raynal@bootlin.com
Yes, better not to write to reserved fields:
Reviewed-by: Fabio Estevam festevam@gmail.com
On Fri, Nov 17, 2023 at 1:50 PM Miquel Raynal miquel.raynal@bootlin.com wrote:
Initially investigating a Linux network issue causing a lot of drop and poor network performances on a custom system based on a TQMA6A module (based on an iMX6Q), [1st link below].
I eventually correlated my observations with a contention at the NIC level when in concurrency with the graphics pipeline. Troubleshooting this in the kernel lead to disabling DMA bursts accesses made by the IPU in order to avoid triggering the QoS at the interconnect level, reducing from 50 to 10% the drop rate on eth0, [2nd link below]. The solution worked on my setup but not on others, which still suffered from abnormally high drop rates even with this "fix".
After looking a while into TQ Systems BSP I figured out a number of differences in recent U-Boot out-of-tree patches they had in their repository [3rd link]. Parsing the differences one after the other lead me to this final solution.
The reset pad of the DDR controller was apparently misconfigured, Bit 18-19 picturing the "DDR select field". The current value b11 is reserved. The only defined value as of version 6 of the iMX6Q manual was b00 "DDR3 and LPDDR2 mode". In practice no register difference has been spotted after changing this configuration but all issues tracked thus far just vanished. All previous fixes have been proven irrelevant. Just clearing this field solved all our network issues and the drop rate as measured by iperf3 felt back to 0%.
Link: https://lore.kernel.org/netdev/20231012193410.3d1812cf@xps-13/ Link: https://lists.freedesktop.org/archives/dri-devel/2023-October/428251.html Link: https://github.com/tq-systems/u-boot-tqmaxx/commit/15eb6abbefbf6916c28467b85... Signed-off-by: Miquel Raynal miquel.raynal@bootlin.com
Applied, thanks.
Hi Fabio,
festevam@gmail.com wrote on Tue, 12 Dec 2023 13:11:53 -0300:
On Fri, Nov 17, 2023 at 1:50 PM Miquel Raynal miquel.raynal@bootlin.com wrote:
Initially investigating a Linux network issue causing a lot of drop and poor network performances on a custom system based on a TQMA6A module (based on an iMX6Q), [1st link below].
I eventually correlated my observations with a contention at the NIC level when in concurrency with the graphics pipeline. Troubleshooting this in the kernel lead to disabling DMA bursts accesses made by the IPU in order to avoid triggering the QoS at the interconnect level, reducing from 50 to 10% the drop rate on eth0, [2nd link below]. The solution worked on my setup but not on others, which still suffered from abnormally high drop rates even with this "fix".
After looking a while into TQ Systems BSP I figured out a number of differences in recent U-Boot out-of-tree patches they had in their repository [3rd link]. Parsing the differences one after the other lead me to this final solution.
The reset pad of the DDR controller was apparently misconfigured, Bit 18-19 picturing the "DDR select field". The current value b11 is reserved. The only defined value as of version 6 of the iMX6Q manual was b00 "DDR3 and LPDDR2 mode". In practice no register difference has been spotted after changing this configuration but all issues tracked thus far just vanished. All previous fixes have been proven irrelevant. Just clearing this field solved all our network issues and the drop rate as measured by iperf3 felt back to 0%.
Link: https://lore.kernel.org/netdev/20231012193410.3d1812cf@xps-13/ Link: https://lists.freedesktop.org/archives/dri-devel/2023-October/428251.html Link: https://github.com/tq-systems/u-boot-tqmaxx/commit/15eb6abbefbf6916c28467b85... Signed-off-by: Miquel Raynal miquel.raynal@bootlin.com
Applied, thanks.
Thanks a lot. Any chances you would have access to additional information to explain how the network vs video pipeline interact without this patch? I am eager to understand.
Cheers, Miquèl
Hi Miquel,
On Tue, Dec 12, 2023 at 1:21 PM Miquel Raynal miquel.raynal@bootlin.com wrote:
Thanks a lot. Any chances you would have access to additional information to explain how the network vs video pipeline interact without this patch? I am eager to understand.
Unfortunately, I don't have additional information on this topic.
I think someone at NXP who is close to the i.MX6 DDR controller design would be able to explain it.
I am adding Peng Fan on Cc just in case. Or maybe you need to enter a ticket to the NXP folks.
Regards,
Fabio Estevam
participants (2)
-
Fabio Estevam -
Miquel Raynal