qemu_arm64_defconfig: PCI autoconfig fails for qemu-system-aarch64 -m 4G
Dear all,
I want to run qemu_arm64_defconfig with 4G.
qemu-system-aarch64 -machine virt -m 4G -smp cores=2 \ -bios u-boot.bin -cpu cortex-a53 -nographic -gdb tcp::1234 \ -netdev user,id=eth0,tftp=tftp -device e1000,netdev=eth0 \ -device virtio-rng-pci
I see an error:
No working controllers found e1000: 52:54:00:12:34:56
Warning: e1000#0 using MAC address from ROM BOOTP broadcast 1 pci_hose_phys_to_bus: invalid physical address BOOTP broadcast 2 pci_hose_phys_to_bus: invalid physical address
The error does not occur for a smaller RAM size.
I added some debug output:
_dm_pci_phys_to_bus: phys_addr 0x000000013ffecb40, res->phys_start 0x0000000010000000, res->bus_start 0x10000000
As qemu_arm64_defconfig does not define CONFIG_SYS_PCI_64BIT the calculated bus address is truncated.
If CONFIG_SYS_PCI_64BIT is defined for qemu_arm64_defconfig and the memory is less then 4 GiB an error PCI: Failed autoconfig bar 10 occurs.
For 3GiB memory I observed the following values:
_dm_pci_phys_to_bus: phys_addr 0x00000000fffecbc0, res->phys_start 0x0000000010000000, res->bus_start 0x0000000010000000
When I define type pci_addr_t as 64bit everything works correctly.
Why does the number of bits in pci_addr_t depend on CONFIG_SYS_PCI_64BIT and not on the bitness of the system?
It is especially worrisome that CONFIG_SYS_PCI_64BIT is not documented at all. I wonder if Kumar remembers why he introduced it in 2008.
I tried to change the definition of the following types in pci.h
typedef phys_addr_t pci_addr_t; typedef phys_addr_t pci_size_t;
This lets qemu-x86_defconfig fail with Error binding driver 'cpu_qemu': -12
Using
typedef unsigned long pci_addr_t; typedef unsigned long pci_size_t;
does not produce the error but I am not sure if this is the right approach.
Best regards
Heinrich
Hi Heinrich,
On Wed, 1 Jan 2020 at 19:58, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Dear all,
I want to run qemu_arm64_defconfig with 4G.
qemu-system-aarch64 -machine virt -m 4G -smp cores=2 \ -bios u-boot.bin -cpu cortex-a53 -nographic -gdb tcp::1234 \ -netdev user,id=eth0,tftp=tftp -device e1000,netdev=eth0 \ -device virtio-rng-pci
Out of curiosity, why E1000 over virtio-net?
I see an error:
...
pci_hose_phys_to_bus: invalid physical address
The error does not occur for a smaller RAM size.
I added some debug output:
_dm_pci_phys_to_bus: phys_addr 0x000000013ffecb40, res->phys_start 0x0000000010000000, res->bus_start 0x10000000
As qemu_arm64_defconfig does not define CONFIG_SYS_PCI_64BIT the calculated bus address is truncated.
Makes sense.
If CONFIG_SYS_PCI_64BIT is defined for qemu_arm64_defconfig and the memory is less then 4 GiB an error PCI: Failed autoconfig bar 10 occurs.
I believe this happens because the handling of the ranges DT property handling (or it could be in the PCI core itself, not sure) is a bit simplistic in U-Boot.
The ranges property in QEMU-generated dtb contains two PCI_REGION_MEM ranges, one below 32-bit boundary and one above it. But decode_regions() in U-Boot only supports one range of given type. When CONFIG_SYS_PCI_64BIT is unset, only the region below 32-bit boundary is used, otherwise the one above 32-bits is used. See commit 52ba907328ec069ff1cec6cbe659f1714c68ed33.
Thus with CONFIG_SYS_PCI_64BIT set any PCI devices which do not support 64-bit mem BARs stop working. This should happen regardless of the RAM size, right?
For 3GiB memory I observed the following values:
_dm_pci_phys_to_bus: phys_addr 0x00000000fffecbc0, res->phys_start 0x0000000010000000, res->bus_start 0x0000000010000000
When I define type pci_addr_t as 64bit everything works correctly.
Why does the number of bits in pci_addr_t depend on CONFIG_SYS_PCI_64BIT and not on the bitness of the system?
As I understand it, some PCI host controllers may require using 64-bit PCI bus addresses but still be connected to a 32-bit CPU.
It is especially worrisome that CONFIG_SYS_PCI_64BIT is not documented at all. I wonder if Kumar remembers why he introduced it in 2008.
I tried to change the definition of the following types in pci.h
typedef phys_addr_t pci_addr_t; typedef phys_addr_t pci_size_t;
This lets qemu-x86_defconfig fail with Error binding driver 'cpu_qemu': -12
Using
typedef unsigned long pci_addr_t; typedef unsigned long pci_size_t;
does not produce the error but I am not sure if this is the right approach.
One solution would be to fix decode_regions() to work with multiple regions of the same type and then enable CONFIG_SYS_PCI_64BIT.
I think another solution would be to simply limit U-Boot from being relocated above the 32-bit boundary. IIRC some other 64-bit boards do this as well. In fact, this might be the better solution because some bus mastering PCI devices cannot access memory above 32-bit. As far as I can tell, E1000 is not affected but for example EHCI is.
On 1/2/20 12:28 PM, Tuomas Tynkkynen wrote:
Hi Heinrich,
On Wed, 1 Jan 2020 at 19:58, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Dear all,
I want to run qemu_arm64_defconfig with 4G.
qemu-system-aarch64 -machine virt -m 4G -smp cores=2 \ -bios u-boot.bin -cpu cortex-a53 -nographic -gdb tcp::1234 \ -netdev user,id=eth0,tftp=tftp -device e1000,netdev=eth0 \ -device virtio-rng-pciOut of curiosity, why E1000 over virtio-net?
I see an error:
...
pci_hose_phys_to_bus: invalid physical address
The error does not occur for a smaller RAM size.
I added some debug output:
_dm_pci_phys_to_bus: phys_addr 0x000000013ffecb40, res->phys_start 0x0000000010000000, res->bus_start 0x10000000
As qemu_arm64_defconfig does not define CONFIG_SYS_PCI_64BIT the calculated bus address is truncated.
Makes sense.
If CONFIG_SYS_PCI_64BIT is defined for qemu_arm64_defconfig and the memory is less then 4 GiB an error PCI: Failed autoconfig bar 10 occurs.
I believe this happens because the handling of the ranges DT property handling (or it could be in the PCI core itself, not sure) is a bit simplistic in U-Boot.
The ranges property in QEMU-generated dtb contains two PCI_REGION_MEM ranges, one below 32-bit boundary and one above it. But decode_regions() in U-Boot only supports one range of given type. When CONFIG_SYS_PCI_64BIT is unset, only the region below 32-bit boundary is used, otherwise the one above 32-bits is used. See commit 52ba907328ec069ff1cec6cbe659f1714c68ed33.
Thus with CONFIG_SYS_PCI_64BIT set any PCI devices which do not support 64-bit mem BARs stop working. This should happen regardless of the RAM size, right?
For 3GiB memory I observed the following values:
_dm_pci_phys_to_bus: phys_addr 0x00000000fffecbc0, res->phys_start 0x0000000010000000, res->bus_start 0x0000000010000000
When I define type pci_addr_t as 64bit everything works correctly.
Why does the number of bits in pci_addr_t depend on CONFIG_SYS_PCI_64BIT and not on the bitness of the system?
As I understand it, some PCI host controllers may require using 64-bit PCI bus addresses but still be connected to a 32-bit CPU.
It is especially worrisome that CONFIG_SYS_PCI_64BIT is not documented at all. I wonder if Kumar remembers why he introduced it in 2008.
I tried to change the definition of the following types in pci.h
typedef phys_addr_t pci_addr_t; typedef phys_addr_t pci_size_t;
This lets qemu-x86_defconfig fail with Error binding driver 'cpu_qemu': -12
Using
typedef unsigned long pci_addr_t; typedef unsigned long pci_size_t;
does not produce the error but I am not sure if this is the right approach.
One solution would be to fix decode_regions() to work with multiple regions of the same type and then enable CONFIG_SYS_PCI_64BIT.
I think another solution would be to simply limit U-Boot from being relocated above the 32-bit boundary. IIRC some other 64-bit boards do this as well. In fact, this might be the better solution because some bus mastering PCI devices cannot access memory above 32-bit. As far as I can tell, E1000 is not affected but for example EHCI is.
Hello Tuomas,
thank you for your analysis.
Changing the relocation address has amongst others the following side effects:
* reducing the maximum memory for function tracing * reducing the available memory for UEFI
Actually I was increasing RAM for QEMU due to working with function tracing.
Where in decode_regions() do you see the limitation to work with multiple regions of the same type?
Best regards
Heinrich
participants (2)
-
Heinrich Schuchardt -
Tuomas Tynkkynen