NVMe support on RPi CM4 board
I compiled u-boot version 2023-07 and boot it on a RPi CM4 using a PCIe M.2 NVMe drive, but it doesn't appear to be working.
Once u-boot has started and boot successfully it recognizes the PCIe device giving me the next output
U-Boot> pci BusDevFun VendorId DeviceId Device Class Sub-Class _____________________________________________________________ 00.00.00 0x14e4 0x2711 Bridge device 0x04 01.00.00 0x1c5c 0x174a Mass storage controller 0x08
but when trying to use the nvme scan and nvme info there is not output.
NVMe drive has two partitions 1st is a FAT partition, and 2nd is a EXT4 partition, the RPi firmware recognizes the FAT partition and u-boot boot correctly.
I don't know if this worked before, but I saw in the mailing list that there could be already support for other ARM64 boards.
Hi Luis,
On Tue, 8 Aug 2023 at 17:09, Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
I compiled u-boot version 2023-07 and boot it on a RPi CM4 using a PCIe M.2 NVMe drive, but it doesn't appear to be working.
Once u-boot has started and boot successfully it recognizes the PCIe device giving me the next output
U-Boot> pci BusDevFun VendorId DeviceId Device Class Sub-Class _____________________________________________________________ 00.00.00 0x14e4 0x2711 Bridge device 0x04 01.00.00 0x1c5c 0x174a Mass storage controller 0x08
but when trying to use the nvme scan and nvme info there is not output.
NVMe drive has two partitions 1st is a FAT partition, and 2nd is a EXT4 partition, the RPi firmware recognizes the FAT partition and u-boot boot correctly.
I don't know if this worked before, but I saw in the mailing list that there could be already support for other ARM64 boards.
Is NVME_PCI enabled?
Regards, SImon
Hi Simon, I enabled the next four variables
CONFIG_NVME_PCI=y CONFIG_NVME=y CONFIG_CMD_NVME=y CONFIG_PCI=y
how different is CONFIG_NVME_PCI from NVME_PCI, I took the latter value from several configs files in u-boot repo and from [1]
[1] https://github.com/u-boot/u-boot/blob/ef8336e2705fce2502383f25d68188c6b1f94d...
El mar, 8 ago 2023 a las 22:05, Simon Glass (sjg@google.com) escribió:
Hi Luis,
On Tue, 8 Aug 2023 at 17:09, Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
I compiled u-boot version 2023-07 and boot it on a RPi CM4 using a PCIe
M.2
NVMe drive, but it doesn't appear to be working.
Once u-boot has started and boot successfully it recognizes the PCIe
device
giving me the next output
U-Boot> pci BusDevFun VendorId DeviceId Device Class Sub-Class _____________________________________________________________ 00.00.00 0x14e4 0x2711 Bridge device 0x04 01.00.00 0x1c5c 0x174a Mass storage controller 0x08
but when trying to use the nvme scan and nvme info there is not output.
NVMe drive has two partitions 1st is a FAT partition, and 2nd is a EXT4 partition, the RPi firmware recognizes the FAT partition and u-boot boot correctly.
I don't know if this worked before, but I saw in the mailing list that there could be already support for other ARM64 boards.
Is NVME_PCI enabled?
Regards, SImon
Hi Luis,
On Wed, 9 Aug 2023 at 10:43, Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
Hi Simon, I enabled the next four variables
CONFIG_NVME_PCI=y CONFIG_NVME=y CONFIG_CMD_NVME=y CONFIG_PCI=y
how different is CONFIG_NVME_PCI from NVME_PCI, I took the latter value from several configs files in u-boot repo and from [1]
I just left off the CONFIG as they all have that. This looks right to me. Assuming that the NVMe controller appears in 'dm tree' perhaps you can enable debugging in the driver and see why it is not working?
Regards, Simon
[1] https://github.com/u-boot/u-boot/blob/ef8336e2705fce2502383f25d68188c6b1f94d...
El mar, 8 ago 2023 a las 22:05, Simon Glass (sjg@google.com) escribió:
Hi Luis,
On Tue, 8 Aug 2023 at 17:09, Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
I compiled u-boot version 2023-07 and boot it on a RPi CM4 using a PCIe M.2 NVMe drive, but it doesn't appear to be working.
Once u-boot has started and boot successfully it recognizes the PCIe device giving me the next output
U-Boot> pci BusDevFun VendorId DeviceId Device Class Sub-Class _____________________________________________________________ 00.00.00 0x14e4 0x2711 Bridge device 0x04 01.00.00 0x1c5c 0x174a Mass storage controller 0x08
but when trying to use the nvme scan and nvme info there is not output.
NVMe drive has two partitions 1st is a FAT partition, and 2nd is a EXT4 partition, the RPi firmware recognizes the FAT partition and u-boot boot correctly.
I don't know if this worked before, but I saw in the mailing list that there could be already support for other ARM64 boards.
Is NVME_PCI enabled?
Regards, SImon
Hi Simon, it took me some time to come back to this issue, but I was able to enable the debug messages on drivers/nvme/nvme.c.
This is what I got when running the nvme scan command
U-Boot> pci BusDevFun VendorId DeviceId Device Class Sub-Class _____________________________________________________________ 00.00.00 0x14e4 0x2711 Bridge device 0x04 01.00.00 0x1c5c 0x174a Mass storage controller 0x08 U-Boot> nvme info U-Boot> nvme scan Cannot set queue count (err=-110) Unable to setup I/O queues(err=-110) Failed to probe 'nvme#0': err=-110 U-Boot>
I'm reporting as soon as I got the messages, but I'll continue investigating.
Regards, Luis
El mié, 9 ago 2023 a las 20:15, Simon Glass (sjg@google.com) escribió:
Hi Luis,
On Wed, 9 Aug 2023 at 10:43, Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
Hi Simon, I enabled the next four variables
CONFIG_NVME_PCI=y CONFIG_NVME=y CONFIG_CMD_NVME=y CONFIG_PCI=y
how different is CONFIG_NVME_PCI from NVME_PCI, I took the latter value
from several configs files in u-boot repo and from [1]
I just left off the CONFIG as they all have that. This looks right to me. Assuming that the NVMe controller appears in 'dm tree' perhaps you can enable debugging in the driver and see why it is not working?
Regards, Simon
[1]
https://github.com/u-boot/u-boot/blob/ef8336e2705fce2502383f25d68188c6b1f94d...
El mar, 8 ago 2023 a las 22:05, Simon Glass (sjg@google.com) escribió:
Hi Luis,
On Tue, 8 Aug 2023 at 17:09, Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
I compiled u-boot version 2023-07 and boot it on a RPi CM4 using a
PCIe M.2
NVMe drive, but it doesn't appear to be working.
Once u-boot has started and boot successfully it recognizes the PCIe
device
giving me the next output
U-Boot> pci BusDevFun VendorId DeviceId Device Class Sub-Class _____________________________________________________________ 00.00.00 0x14e4 0x2711 Bridge device 0x04 01.00.00 0x1c5c 0x174a Mass storage controller 0x08
but when trying to use the nvme scan and nvme info there is not
output.
NVMe drive has two partitions 1st is a FAT partition, and 2nd is a
EXT4
partition, the RPi firmware recognizes the FAT partition and u-boot
boot
correctly.
I don't know if this worked before, but I saw in the mailing list that there could be already support for other ARM64 boards.
Is NVME_PCI enabled?
Regards, SImon
Hi Luis,
L On Tue, Sep 5, 2023, 12:00 Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
Hi Simon, it took me some time to come back to this issue, but I was able to enable the debug messages on drivers/nvme/nvme.c.
This is what I got when running the nvme scan command
U-Boot> pci BusDevFun VendorId DeviceId Device Class Sub-Class _____________________________________________________________ 00.00.00 0x14e4 0x2711 Bridge device 0x04 01.00.00 0x1c5c 0x174a Mass storage controller 0x08 U-Boot> nvme info U-Boot> nvme scan Cannot set queue count (err=-110) Unable to setup I/O queues(err=-110) Failed to probe 'nvme#0': err=-110 U-Boot>
I'm reporting as soon as I got the messages, but I'll continue investigating.
Can you check if the device has PCI bus master enabled?
Regards, Simon
Regards, Luis
El mié, 9 ago 2023 a las 20:15, Simon Glass (sjg@google.com) escribió:
Hi Luis,
On Wed, 9 Aug 2023 at 10:43, Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
Hi Simon, I enabled the next four variables
CONFIG_NVME_PCI=y CONFIG_NVME=y CONFIG_CMD_NVME=y CONFIG_PCI=y
how different is CONFIG_NVME_PCI from NVME_PCI, I took the latter value
from several configs files in u-boot repo and from [1]
I just left off the CONFIG as they all have that. This looks right to me. Assuming that the NVMe controller appears in 'dm tree' perhaps you can enable debugging in the driver and see why it is not working?
Regards, Simon
[1]
https://github.com/u-boot/u-boot/blob/ef8336e2705fce2502383f25d68188c6b1f94d...
El mar, 8 ago 2023 a las 22:05, Simon Glass (sjg@google.com)
escribió:
Hi Luis,
On Tue, 8 Aug 2023 at 17:09, Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
I compiled u-boot version 2023-07 and boot it on a RPi CM4 using a
PCIe M.2
NVMe drive, but it doesn't appear to be working.
Once u-boot has started and boot successfully it recognizes the PCIe
device
giving me the next output
U-Boot> pci BusDevFun VendorId DeviceId Device Class Sub-Class _____________________________________________________________ 00.00.00 0x14e4 0x2711 Bridge device 0x04 01.00.00 0x1c5c 0x174a Mass storage controller 0x08
but when trying to use the nvme scan and nvme info there is not
output.
NVMe drive has two partitions 1st is a FAT partition, and 2nd is a
EXT4
partition, the RPi firmware recognizes the FAT partition and u-boot
boot
correctly.
I don't know if this worked before, but I saw in the mailing list
that
there could be already support for other ARM64 boards.
Is NVME_PCI enabled?
Regards, SImon
Thanks for your answer Simon,
Can you check if the device has PCI bus master enabled?
I think this one will take me a bit more time, should this be in raspberry pi’s eeprom firmware?
Will the disabled PCI bus master also be a problem for the Linux kernel? As this is working as expected
Regards Luis
Hi Luis,
On Tue, 5 Sept 2023 at 17:17, Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
Thanks for your answer Simon,
Can you check if the device has PCI bus master enabled?
I think this one will take me a bit more time, should this be in raspberry pi’s eeprom firmware?
Will the disabled PCI bus master also be a problem for the Linux kernel? As this is working as expected
(BTW the quoting seems broken in your email; also the text came out white so at first I thought you had sent an empty reply)
Actually I can see that nvme_probe() turns on bus mastering.
So no, I don't know what is wrong, sorry. Perhaps there are some quirks in the Linux driver which need to be added to U-Boot? Perhaps there is a power GPIO to set?
Regards, Simon
Hi Simon, no problem, apologies for the quoting issue, and thank you for all your prompt answers and some guidance to try to understand what is happening.
Although I'm still having the same issue, I'll share more insights just in case they might be useful for anyone having the same problem.
1. After I run nvme scan I'm noticing that the m.2 led starts flashing (it was not flashing, before running the command it was always On). 2. I'm having the same issue on the Official IO Board and Waveshare cm4-io-base-b, using latest stable rpi-eeprom firmware and boot files (start4x.elf, etc), latest u-boot version 2023.10-rc4 (I've also used previous versions, but using the latest one because it has debug messages)
It does not appear that there is any GPIO related to NVMe. Unfortunately I'll move forward and try other boards.
Thank you again for your help Simon.
El dom, 10 sept 2023 a las 17:37, Simon Glass (sjg@google.com) escribió:
Hi Luis,
On Tue, 5 Sept 2023 at 17:17, Luis Alfredo da Silva luis.adasilvaf@gmail.com wrote:
Thanks for your answer Simon,
Can you check if the device has PCI bus master enabled?
I think this one will take me a bit more time, should this be in
raspberry pi’s eeprom firmware?
Will the disabled PCI bus master also be a problem for the Linux kernel?
As this is working as expected
(BTW the quoting seems broken in your email; also the text came out white so at first I thought you had sent an empty reply)
Actually I can see that nvme_probe() turns on bus mastering.
So no, I don't know what is wrong, sorry. Perhaps there are some quirks in the Linux driver which need to be added to U-Boot? Perhaps there is a power GPIO to set?
Regards, Simon
participants (2)
-
Luis Alfredo da Silva -
Simon Glass