Difference between revisions of "NVMe SSD boot with the Raspberry Pi 5"

Latest revision as of 19:05, 16 July 2024

This tutorial^[1] describes how to configure the Raspberry Pi OS on PI 5 to get pcie pip working

Important Notes

NVMe SSD Incompatibility List

We recommend avoiding the following NVMe SSD drives which is equipped with a Phison controller due to their proven incompatibility:

WD Blue SN550 series (Solved! Refer to New rpi-eeprom-update 2024-01-24 WD Blue SN550 nvme works now.)
WD Blue SN580 series (Solved! Refer to NVMe_SSD_boot_with_the_Raspberry_Pi_5#comment-4708)
WD Green SN350 series (Solved! Refer to NVMe_SSD_boot_with_the_Raspberry_Pi_5#comment-4602)
WD Black SN850 series
WD Black SN770
WD SN740
Inland tn446 nvme drive
Corsair MP600 SSD
Micron 2450 SSD (Can be recognised but not support boot from NVME)
Other NVMe SSD drivers equipped with the same Phison controller

These specific models have demonstrated compatibility issues, and it is advisable to avoid them when considering NVMe SSD options for the X10xx series NVMe shield. You can run "lspci" command to check the controller brand of the SSD.

We confirm that the following SSDs are incompatible:

Micron 2200 256GB M.2 NVMe Gen3 x4, MODEL: MTFDHBA256TCH, The SSD is recognized but I cannot boot from it.

Also note:

Compatible with M.2 NVMe SSDs only, Not compatible with M.2 SATA SSDs, M.2 PCIe AHCI SSDs, or other M.2 non-NVMe devices
Older NVMe drives with less efficient flash media may not perform as well as newer drives
New NVMe SSDs are not partitioned and will need to be both partitioned and formatted when first connected to the Raspberry Pi before they will be accessed in the Explorer.
We get feedback from customers that Polaris Controller will also have compatibility problems. Please replace the other SSD test if it not work, whether it is compatible with the Raspberry Pi 5 does not depend on the X100X series boards
NVMEs using the MAP1202 controller may not support PCIe Gen 2, and must be forced to enable PCIe Gen 3 in order to be recognised. This is due to the fact that the controller is not backward compatible with PCIe Gen 2, and NVMEs using this controller will have compatibility issues, and are not recommended for use. Can refer to https://zhuanlan.zhihu.com/p/644984347

PS: There is also feedback from buyers that even NVME SSDs with Phison controller are supported after updating the latest firmware. Please refer to go to: X1001#comment-4638

Enable PCIe

By default the PCIe connector is not enabled. To enable it you should add the following option into /boot/firmware/config.txt and reboot:

sudo nano /boot/firmware/config.txt

Then add the following comment;

# Enable the PCIe External connector.
dtparam=pciex1

# This line is an alias for above (you can use either/or to enable the port).
dtparam=nvme

Press Ctrl-O, then enter, to write the change to the file.

Press Ctrl-X to exit nano (the editor).

And the connection is certified for Gen 2.0 speed (5 GT/sec), but you can force it to Gen 3.0 (10 GT/sec) if you add the following line after:

dtparam=pciex1_gen=3

WARNING:The Raspberry Pi 5 is not certified for Gen 3.0 speeds, and connections to PCIe devices at these speeds may be unstable.

Then DON'T forgot to reboot

sudo reboot

You also can refer to official documentatio:Enabling PCIe

After reboot,use the lspci command to display your PCIe devices

 sudo lspci

The output is as follows, note that the content of the third line depends on the NVME control you are using.

pi@raspberrypi: ~ $ sudo lspci
0000:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
0000:01:00.0 Non-Volatile memory controller: Silicon Motion, Inc. SM2263EN/SM2263XT SSD Controller (rev 03)
0001:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
0001:01:00.0 Ethernet controller: Device 1de4:0001

Make sure your NVME SSD is plugged into the PCIe PIP, not just connecting PCIe PIP to the Pi 5, otherwise the lspci will not be able to display any PCIe device.
If your NVMe SSD is not recognized, updating the bootloader firmware is essential!Refer to How to update eeprom firmware to update firmware or #FAQ Q1 to know more details.

Flash OS onto NVME SSD

To get the NVMe SSD to boot your Pi, it needs to have an OS, so the Raspberry Pi OS needs to be flashed onto NVME SSDs, this is very important!

Only support Raspberry Pi OS (Bookworm) version, Raspberry Pi OS bullseye or Ubuntu or Home Assistant OS is NOT supported, refer to https://www.raspberrypi.com/software/operating-systems/

Here are a few ways to flash the OS to an NVME SSD:

1. Use SD Card Copier tool to flash OS onto the NVME SSD On Raspberry Pi OS(Recommended)

This method works if you have an SD card and have booted the device successfully

Cick Applications =>Accessories =>SD Card Copier on the main screen, run the SD Card Copier program, and copy the OS to the NVME ssd as shown in the figure below.

Click Start to run. Then shut down, unplug the SD card, and restart the device.

This is a visual operation and we highly recommend it!

2. Flash the SSD with Raspberry Pi Imager

You can also directly use the Raspberry Pi Imager tool on a MAC computer or windows computer to flash a fresh Pi OS to NVME SSD, but you will need an additional USB to nvme adapter.

Install Pi Imager and open it
Plug your NVMe SSD into your computer using a USB to NVMe adapter
Choose an OS to install
Choose the drive (connected through your adapter) to flash
Click write (and set any options you'd like)
Once you have finished flashing the OS, DON'T remove the nvme SSD. You must #Enable PCIe.

Then pull the NVMe drive, attach it to your Pi 5, and it should boot off it (with or without a microSD card inserted)—assuming you have the bootloader up to date and set the BOOT_ORDER appropriately!

NOTE:

If you are flashing a fresh Pi OS to NVME ssd, you must #Enable PCIe; but if you are COPY or CLONE an old Pi OS from SD card to NVME ssd, and you have already enabled pcie in the old Pi OS in advance, then you don't need to do enable pcie again!

Set NVMe early in the boot order

The PCIe connection should work after a reboot, but your Pi won't try booting off an NVMe SSD yet. For that, you need to change the BOOT_ORDER in the Raspberry Pi's bootloader configuration:

Use tool raspi-config to set boot order

sudo raspi-config

Then select 6 Advanced Opitions => A4 Boot Order => B2 NVMe/USB Boot answer Yes, then sudo reboot

Run the following command to set boot order.

sudo rpi-eeprom-config --edit

Then change the BOOT_ORDER line to the following:

BOOT_ORDER=0xf416

- Press Ctrl-O, then enter, to write the change to the file.

- Press Ctrl-X to exit nano (the editor).

Read Raspberry Pi's documentation on BOOT_ORDER for all the details. For now, the pertinent bit is the 6 at the end: that is what tells the Pi to attempt NVMe boot first!

Reboot your Raspberry Pi 5 to make the change take effect.

Decrease wattage when turned off

By default, the Raspberry Pi 5 consumes around 1W to 1.4W of power when turned off. This can be decreased by manually editing the EEPROM configuration with sudo rpi-eeprom-config -e. Change the settings to the following:

BOOT_UART=1
POWER_OFF_ON_HALT=1
BOOT_ORDER=0xf416

Then reboot the device with sudo reboot, this should drop the power consumption when powered down to around 0.01W.

References

↑ Refer to NVM NVMe SSD boot with the Raspberry Pi 5

FAQ

In fact, some PIPs such as X1001 / X1002 / X1003 / X1000 etc are just an extension of the Raspberry Pi 5 PCIe, and their own hardware functions are limited. If the OS configuration and PCle FFC cable faults can be ruled out, the normal operation of PCIe (recognizing and booting) depends more on the firmware and SSD compatibility. Since October 2023, the Raspberry Pi Foundation has been updating the bootloader version to solve similar problems. You can learn more from the link below.

Raspberry Pi5 bootloader EEPROM release notes

After several major version updates, the bootloader has basically solved the firmware issue, so the remaining is the OS configuration and SSD compatibility issues.

If you are using a very old SSD, if you are experiencing issues with it being recognised and not being able to boot, the first thing you should consider is replacing the NVME SSD.

Q1: Can't boot/recognize from NVMe SSD?

click it to zoom out

A: Here are a few steps to help you troubleshoot some issues：

Only Raspberry Pi OS (Bookworm) supports PCIe, Raspberry Pi OS bullseye is NOT supported, refer to https://www.raspberrypi.com/software/operating-systems/
Make sure that your OS configuration (/boot/firmware/config.txt) is completely correct: refer to the tutorial:NVMe SSD boot with the Raspberry Pi 5 (will Continuously updating)
Update the bootloader firmware to the last verson, refer to How to update eeprom firmware, this is very important!!! Make sure your upgrade is successful by running command vcgencmd bootloader_version
Add PCIE_PROBE=1 to the eeprom, (Run sudo rpi-eeprom-config --edit command, then add this line, then press CTRL+O to write change to file, press CTRL+X to exit, then reboot)
Make sure the SSD type you use is correct, only support M.2 NVME (Sata) Key-M SSD NOT support M.2 NGFF Key-B SSD. Refer to the right diagram:
Make sure the FPC cable is firmly and fully inserted into the PI 5, this is very important. For X1003 shield, make sure the FPC cable direction is correct also.
For some small brand SSD, consider forcing open PCIe Gen3 to test (add dtparam=pciex1_gen=3 line in config.txt then reboot), because some small brand SSDs are NOT backward compatible with PCIe Gen 2 for cost reasons.
For X1003 shield, make sure the FPC cable direction is correct also. If an error still occurs, replace the FFC cable to test or purchase other PFC from here^[1]to troubleshoot the issue.
Read this post to get some help;
Although you can also boot from a SD card, Pi OS is required for the NVMe SSD to boot, DON'T forget Flash OS into NVMe SSD.
If you're flashing a fresh Pi OS into NVME SSD , DON'T forget to ENABLE PCIe to NVME SSD.
Run the lspci and lsblk command to check the NVMe SSD is recognized. (NOTE: Make sure your NVMe SSD is plugged into the PCIe PIP, not just connecting PCIe PIP to the Pi5, otherwise the lspci will not be able to display any PCIe device). For X1004, X1005, Q200 and X1011, , you can only see several ASMedia Technology PCIe devices when running lspci, but not the NVME SSD information. You need to use the lsblk command to check whether NVME can be recognized.
Start thinking about SSD compatibility issues and try to switch to other brands of NVME SSDs. Some customers have reported that they have successfully solved their problems by replacing their SSDs.
Contact us at email: support@geekworm.com, and attached your product model, OS version (run uname -a to get), bootloader version (run vcgencmd bootloader_version to get) and order number

How to get FPC Wires:

↑
Amazon US: https://www.amazon.com/dp/B0CXCTPQ2N
Amazon DE: https://www.amazon.de/dp/B0CX4T993J
Amazon UK: https://www.amazon.co.uk/dp/B0CX4T993J
Amazon JP: https://www.amazon.co.jp/dp/B0CYGKN7G1

PS: The shortest length of the FPC cable here is 30mm, but the length of the FPC cable used by X1003 is only 22mm. For X1003, this may not be perfect, but you can use these FPC cables to troubleshoot the issue. DON'T buy it if you mind.
If the issue is confirmed to be a FFC cable, we will reimburse you the cost of purchasing the FFC cable.

Q2: Don't recognize / boot from NVME SSD?

A: If you have enabled pcie and find that you still cannot boot from NVME SSD, please perform the following actions

1. Follow the picture below to check the version of X1001 or X1000

2. If it is an Old Version, refer to How to update eeprom firmware to update the firmware with pieeprom-2023-10-30.bin file.

3. If it is an New Version, refer to How to update eeprom firmware to update the firmware to lasted version.

4. If you still have not solved the problem, please refer to the next FAQ:Q1

Q3: How to fix the 2230/2242/2260 SSD on X1001 or other PIP?

A: The X1001 and other pips does not reserve the nut column to fix the 2230/2242/2260 SSD. Based on our test, you can secure the 2230/2242/2260 SSD using an M2*6mm screw and 2pcs M2 nuts. Place one M2 nut under the SSD, then use the M2*6mm screw and the other M2 nut to fasten the SSD.

PS: As of April 2024, we have added customized copper pillar in only X1001 packing list to solve this issue, please refer to the packing list of X1001.

Q4. X1002 can't boot when install into the P580 case?

A: Please check if the FPC cable touch the metal case make shorting when install X1002 into the metal case, you can stick the electrical tape on the case to prevent the ribbon FPC cable from contacting the case to test.

Q5: Which NVMe drives work with a Raspberry Pi 5?

A: You can refer to the video: https://youtu.be/3mUgVTBmKio.

Q6: Can they support gen3 PCIe speeds or only gen2?

A:

All PIPs (such X1001, X1002, X1003 etc) are just an extension of the Raspberry Pi 5 PCIe interface. Apart from whether the chip on the PIP board supports PCIe 3.0 or not, the more main focus is on the Raspberry Pi 5.
X1004, X1011, X1005 PIP boards do not support PCIe Gen 3 speed.

Q7: Why can the SSD be recognized but not bootable?

A:Please consider the following advice:

The method of flashing OS to NVME SSD may be problematic.

It is recommended that you boot from the SD card first and then use the SD Card Copier tool to flash the OS to the NVME SSD. Refer to NVMe_SSD_boot_with_the_Raspberry_Pi_5#Flash_OS_onto_NVME_SSD

The SSD used may be incompatible. Please replace it with other brand SSD.

Q8: Why does the LED of X1001/X1004 still blinking after issuing the shutdown command poweroff?

A:Please run the folloing command:

sudo rpi-eeprom-config -e

Then Change the setting of POWER_OFF_ON_HALT from 0 to 1

Then save & reboot your rpi 5.

PS: Currently setting is 1 by default

Return to X1000 / X1001 / X1002 / X1003

Enable comment auto-refresher

Anonymous user #19

26 days ago

Score 0

Hey Guy's,

i bought the X1001 Adapter for my Raspi 5 and was able to shoot WD Blue SN580 nvme 2TB at a discount. Unfortunately, i am still not able to run the nvme using Umbrel OS. Even an update to the latest boot loader version was not helping. Using a USB-C to NVME Adapter the nvme SSD is running smoothly.

This is the Error message i get: [ 63.713349] EXT4-fs error (device nume0n1p7): ext4_check_bdev_write_error:224: comm systemd-journal: Error while async write back metadata [ 63.713380] EXT4-fs error (device nume0n1p7) in ext4_reserve_inode_write:5787: Journal has aborted [ 63.726758] EXT4-fs (nume0n1p6): shut down requested (2) [ 63.727649] EXT4-fs (nume0n1p7): shut down requested (2)

[ 63.737091] EXT4-fs error (device nume0n1p7): __ext4_dirty:202: inode #1080313: comm kworker/u11:0: mark_inode_dirty error