rk3399 issue: no DMA in Linux with mainline TF-A and U-Boot SPL
Hello Kever,
on a rk3399, booting current U-Boot SPL with mainline TF-A leads to missing DMA (and no sound) on Linux.
However, when using rockchip its so called mini-loader (rk3399_miniloader_v1.26.bin) and their BL31 (rk3399_bl31_v1.35.elf) to boot, DMA works perfectly fine.
Tested on a custom rk3399 board and on ROCK Pi4.
Attached to this mail are two boot logs with some debug prints:
- good_amba_log.txt (DMA works) - no_amba_log.txt (no DMA device)
The main difference I can spot between the two logs is that on Linux (drivers/amba/bus.c) AMBA_CID (0xb105f00d) cannot be found. Instead, only some CORESIGHT_CIDs (presumably for debugging) and four 0x00000000 CIDs are detected.
As a result, the "PL330 DMAC-241330" driver does not load. My theory is that DMA needs to be allowed somewhere in the undocumented syscon- registers, similar to what U-Boot is already doing for eMMC in arch_cpu_init() (arch/arm/mach-rockchip/rk3399/rk3399.c).
Any ideas?
Or maybe I'm just missing some configuration?
Since multiple software projects are involved (TF-A, OP-TEE, U-Boot, Linux), I Cc'ed a bit.
Thanks -- Christoph
Hi Christoph,
The ARM PL330 DMA driver in kernel only relate to:
- DTS kernel used, can be check in /proc/device-tree/
- kernel driver which should mach the compatible name.
This driver should has nothing to do with U-Boot SPL or TF-A, because we don't have any special setting for PL330 in loader stage.
Thanks,
- Kever
On 2023/4/3 03:01, Christoph Fritz wrote:
Hello Kever,
on a rk3399, booting current U-Boot SPL with mainline TF-A leads to missing DMA (and no sound) on Linux.
However, when using rockchip its so called mini-loader (rk3399_miniloader_v1.26.bin) and their BL31 (rk3399_bl31_v1.35.elf) to boot, DMA works perfectly fine.
Tested on a custom rk3399 board and on ROCK Pi4.
Attached to this mail are two boot logs with some debug prints:
- good_amba_log.txt (DMA works)
- no_amba_log.txt (no DMA device)
The main difference I can spot between the two logs is that on Linux (drivers/amba/bus.c) AMBA_CID (0xb105f00d) cannot be found. Instead, only some CORESIGHT_CIDs (presumably for debugging) and four 0x00000000 CIDs are detected.
As a result, the "PL330 DMAC-241330" driver does not load. My theory is that DMA needs to be allowed somewhere in the undocumented syscon- registers, similar to what U-Boot is already doing for eMMC in arch_cpu_init() (arch/arm/mach-rockchip/rk3399/rk3399.c).
Any ideas?
Or maybe I'm just missing some configuration?
Since multiple software projects are involved (TF-A, OP-TEE, U-Boot, Linux), I Cc'ed a bit.
Thanks -- Christoph
The ARM PL330 DMA driver in kernel only relate to:
DTS kernel used, can be check in /proc/device-tree/
kernel driver which should mach the compatible name.
drivers/dma/pl330.c needs also a successfully matched amba, but this fails when using mainline TF-A and U-Boot SPL.
I'm using the same kernel and devicetree on both tests, the only thing changed is TF-A and U-Boot SPL vs mini-loader and rk3399_bl31.
This driver should has nothing to do with U-Boot SPL or TF-A, because we don't have any special setting for PL330 in loader stage.
It is drivers/amba/bus.c, which is unable to find an AMBA_CID on the ARM bus. The DMA driver not loading is just a symptom of this issue.
Any ideas?
Thanks -- Christoph
On Mon, Apr 03, 2023 at 12:11:40PM +0200, Christoph Fritz wrote:
The ARM PL330 DMA driver in kernel only relate to:
DTS kernel used, can be check in /proc/device-tree/
kernel driver which should mach the compatible name.
drivers/dma/pl330.c needs also a successfully matched amba, but this fails when using mainline TF-A and U-Boot SPL.
I'm using the same kernel and devicetree on both tests, the only thing changed is TF-A and U-Boot SPL vs mini-loader and rk3399_bl31.
This driver should has nothing to do with U-Boot SPL or TF-A, because we don't have any special setting for PL330 in loader stage.
It is drivers/amba/bus.c, which is unable to find an AMBA_CID on the ARM bus. The DMA driver not loading is just a symptom of this issue.
That brings up the obvious question: why is it unable to find the AMBA CID? Is that because some resource needed to read it hasn't been enabled yet? If the AMBA CID is not accessible, presumably the rest of the PL330 also isn't accessible, so even if we could bind the driver, it still wouldn't work?
On Mon, 2023-04-03 at 14:17 +0100, Russell King (Oracle) wrote:
On Mon, Apr 03, 2023 at 12:11:40PM +0200, Christoph Fritz wrote:
The ARM PL330 DMA driver in kernel only relate to:
DTS kernel used, can be check in /proc/device-tree/
kernel driver which should mach the compatible name.
drivers/dma/pl330.c needs also a successfully matched amba, but this fails when using mainline TF-A and U-Boot SPL.
I'm using the same kernel and devicetree on both tests, the only thing changed is TF-A and U-Boot SPL vs mini-loader and rk3399_bl31.
This driver should has nothing to do with U-Boot SPL or TF-A, because we don't have any special setting for PL330 in loader stage.
It is drivers/amba/bus.c, which is unable to find an AMBA_CID on the ARM bus. The DMA driver not loading is just a symptom of this issue.
That brings up the obvious question: why is it unable to find the AMBA CID? Is that because some resource needed to read it hasn't been enabled yet?
That's my guess too, and that's why I'm asking Kever from Rockchip. To me, it seems that their closed-source BL31 and/or mini-loader is enabling something that is missing in upstream TF-A and/or U-Boot SPL. On top of that, the registers responsible for that are undocumented.
But I could be wrong, and it's possible that I'm missing something else. Maybe dumping the syscon-registers and comparing them might help, but as far as I can see, it's a bit of an effort because they require special treatment to be accessed.
If the AMBA CID is not accessible, presumably the rest of the PL330 also isn't accessible, so even if we could bind the driver, it still wouldn't work?
The PL330 driver attempted to probe, but was deferred due to a missing AMBA_CID. It's in the bootlog that I attached in my first email in this thread.
Thanks -- Christoph
Hi Kever,
I also see this issue when switching between Rockchip ATF and Upstream ATF.
Versions: Rockchip DDR Blob - rk3399_ddr_800MHz_v1.30.bin Rockchip Miniloader - rk3399_miniloader_v1.30.bin Rockchip ATF - rk3399_bl31_v1.36.elf Upstream ATF - git://git.trustedfirmware.org/TF-A/trusted-firmware-a.git, git tag v2.8.0, with RK3399_BAUDRATE changed from 115200 to 1500000 in plat/rockchip/rk3399/rk3399_def.h U-Boot - git://git.denx.de/u-boot.git, git tag v2022.01
Results: Rockchip DDR Blob + Rockchip Miniloader + Rockchip ATF + U-Boot = DMA working dma-pl330 ff6d0000.dma-controller: Loaded driver for PL330 DMAC-241330 dma-pl330 ff6d0000.dma-controller: DBUFF-32x8bytes Num_Chans-6 Num_Peri-12 Num_Events-12 dma-pl330 ff6e0000.dma-controller: Loaded driver for PL330 DMAC-241330 dma-pl330 ff6e0000.dma-controller: DBUFF-128x8bytes Num_Chans-8 Num_Peri-20 Num_Events-16 Rockchip DDR Blob + Rockchip Miniloader + Upstream ATF + U-Boot = DMA not working OF: amba_device_add() failed (-19) for /bus/dma-controller@ff6d0000 OF: amba_device_add() failed (-19) for /bus/dma-controller@ff6e0000
I can't check the Rockchip ATF source code as it isn't available. Any idea what is different between Rockchip ATF and Upstream ATF for DMA to work properly?
Regards, Jonathan
On Mon, 3 Apr 2023 at 18:38, Kever Yang kever.yang@rock-chips.com wrote:
Hi Christoph,
The ARM PL330 DMA driver in kernel only relate to:
DTS kernel used, can be check in /proc/device-tree/
kernel driver which should mach the compatible name.
This driver should has nothing to do with U-Boot SPL or TF-A, because we don't have any special setting for PL330 in loader stage.
Thanks,
- Kever
On 2023/4/3 03:01, Christoph Fritz wrote:
Hello Kever,
on a rk3399, booting current U-Boot SPL with mainline TF-A leads to missing DMA (and no sound) on Linux.
However, when using rockchip its so called mini-loader (rk3399_miniloader_v1.26.bin) and their BL31 (rk3399_bl31_v1.35.elf) to boot, DMA works perfectly fine.
Tested on a custom rk3399 board and on ROCK Pi4.
Attached to this mail are two boot logs with some debug prints:
- good_amba_log.txt (DMA works)
- no_amba_log.txt (no DMA device)
The main difference I can spot between the two logs is that on Linux (drivers/amba/bus.c) AMBA_CID (0xb105f00d) cannot be found. Instead, only some CORESIGHT_CIDs (presumably for debugging) and four 0x00000000 CIDs are detected.
As a result, the "PL330 DMAC-241330" driver does not load. My theory is that DMA needs to be allowed somewhere in the undocumented syscon- registers, similar to what U-Boot is already doing for eMMC in arch_cpu_init() (arch/arm/mach-rockchip/rk3399/rk3399.c).
Any ideas?
Or maybe I'm just missing some configuration?
Since multiple software projects are involved (TF-A, OP-TEE, U-Boot, Linux), I Cc'ed a bit.
Thanks -- Christoph
Hello Jonathan and Kevar,
I also see this issue when switching between Rockchip ATF and Upstream ATF.
Versions: Rockchip DDR Blob - rk3399_ddr_800MHz_v1.30.bin Rockchip Miniloader - rk3399_miniloader_v1.30.bin Rockchip ATF - rk3399_bl31_v1.36.elf Upstream ATF - git://git.trustedfirmware.org/TF-A/trusted-firmware-a.git, git tag v2.8.0, with RK3399_BAUDRATE changed from 115200 to 1500000 in plat/rockchip/rk3399/rk3399_def.h U-Boot - git://git.denx.de/u-boot.git, git tag v2022.01
Results: Rockchip DDR Blob + Rockchip Miniloader + Rockchip ATF + U-Boot = DMA working dma-pl330 ff6d0000.dma-controller: Loaded driver for PL330 DMAC-241330 dma-pl330 ff6d0000.dma-controller: DBUFF-32x8bytes Num_Chans-6 Num_Peri-12 Num_Events-12 dma-pl330 ff6e0000.dma-controller: Loaded driver for PL330 DMAC-241330 dma-pl330 ff6e0000.dma-controller: DBUFF-128x8bytes Num_Chans-8 Num_Peri-20 Num_Events-16 Rockchip DDR Blob + Rockchip Miniloader + Upstream ATF + U-Boot = DMA not working OF: amba_device_add() failed (-19) for /bus/dma-controller@ff6d0000 OF: amba_device_add() failed (-19) for /bus/dma-controller@ff6e0000
I can't check the Rockchip ATF source code as it isn't available. Any idea what is different between Rockchip ATF and Upstream ATF for DMA to work properly?
@Kevar: It would be really great if you could have a look into it.
I am still having this issue.
Thanks -- Christoph
participants (4)
-
Christoph Fritz -
Jonathan Liu -
Kever Yang -
Russell King (Oracle)