Finally, Raspberry Pi users can get rid of flaky microSD cards and boot off of NVMe SSDs
using a HAT like this.
This is the HatDrive! Top Edition from Pineberry Pi, and I'd like to thank them for sending
this as well as the HatDrive! BM1—or the HatDrive! Bottom—that goes underneath the Raspberry
Pi.
These two boards allow you to put an NVMe SSD, which can be much more reliable and much
faster, on a Raspberry Pi 5 using its PCI Express connection.
That's this little connector on the Raspberry Pi 5.
Now a lot of people are asking me, "Does this fit in a Raspberry Pi 5 case?"
And the answer to that is, "Yes, it will."
Now this is not the actual final production hardware.
This is a prototype that they sent me to test.
And if I pop that on there, it fits just inside of this little case if you remove the fan
from this case.
There's normally a fan there.
The problem is that when you put all this together, there's not much ventilation.
So I don't really recommend that.
It will fit and it will run, but it will probably throttle somewhat.
So instead I have over here my other Raspberry Pi 5 and it has an active cooler on top.
This is not the actual hardware that will be shipping.
This is just my little janky setup.
But if you put this on top, it will hold it off of the Pi enough so that there's airflow
into the active cooler.
So the way that this works is you put an NVMe SSD on top.
This is a Makerdisk from Cytron.
They sent me this like a year or two ago.
I've never actually tested it.
But I did format it so it's empty right now.
You put this in and you put this on top of the Pi and then you connect one of these little
cables.
Now, this is called an FPC or FFC—Flat Flexible Circuit board cable.
It's literally a circuit board, but it's very flat and it's flexible.
That's why they call it that. [captain obvious here...]
This is an impedance-controlled one.
You can see that there's different trace widths and it has some shielding.
The one that I was using in my early testing of the Raspberry Pi is not impedance-controlled.
So I had some issues with signaling and so this cable is very handy.
They also make a 100 millimeter version of this cable.
But one thing to keep in mind if you're buying any of these kind of things is orientation
matters very much here.
So there's a little arrow on the cable and I don't believe there's an arrow on the Raspberry
Pi 5, but there is an arrow on this board so when you're installing the cable you
want to line up those little itty-bitty triangles. But the first thing I want to
do is put Raspberry Pi OS on here so I can boot from it. Raspberry Pis don't
come with bootable NVMe by default, you have to kind of enable that, so I'll show
you how to do that. So this Raspberry Pi 5 already has a microSD card with
Raspberry Pi OS on it so I'm going to boot that up too, but first let's flash
this in Raspberry Pi Imager. On my computer I have this little USB, I call it
the puck. It's like a little puck that I can put this NVMe drive in and it's
nice to have but you can pop this in and I'm going to plug it into my Mac.
[puck drop] Whoa this is the kind of thing that I edit out if I'm scripting a video but
I'll leave it in for your enjoyment. When you're trying to work one-handed, it's not
always that fun. Okay, there we go. It's plugged in and up here if I open up
Imager, I should be able to choose an OS and I'm going to choose my pre-downloaded
copy of Raspberry Pi OS and then choose a storage device. So I'll choose that and
write and I'll just save the settings that I had before. So I'm going to write
this... No... Yes... And touch ID. There we go. So it's gonna write that image to the NVMe
and while it's doing that we'll set up the Raspberry Pi. So I'm gonna go ahead
and turn this guy on which I need a power cable– if you don't have power
you're not gonna get anything so let's plug it in like so and I'll show you how
to set up the Raspberry Pi 5 for PCI Express devices—specifically NVMe SSDs
Quick editor note about power: It's important that you use the Raspberry Pi
power adapter or something equivalent that's going to supply enough power
because this HAT is going to use the power from the Pi and it should use
around 5 watts. There's actually a power measurement circuit on the HAT. There's
not one on the one on the bottom but there's one on the HAT. So you can
monitor the power usage of the NVMe SSD that you plug in. The drives that I've
tested have all worked fine, but make sure you follow the issue that's linked in the
description where I'm testing these things because there might be some drives that don't
work as well if they require too much power for the Pi to supply to it.
So we just finished writing that file, writing the Raspberry Pi OS to the NVMe SSD, so I'll
pop that out from my computer and get ready to put it on the Pi.
And then I'm going to log into the Raspberry Pi that I just booted up, and I'm going to
run the command 'sudo rpi-eeprom-config --edit'.
If you want a guide for this, I do have a guide on my website.
It's the "NVMe SSD boot with the Raspberry Pi 5" guide.
I will link it in the description so you don't have to sit there and try to type all the
commands as you see them on the screen.
That's this command.
It's going to edit the EEPROM so that we can tell it, "Hey, boot from NVMe if that exists."
The key is that we want to change this boot order command to have 6 at the end.
That's going to tell the Raspberry Pi first try putting off the NVMe drive, then try microSD,
then try USB.
We're going to edit this, go down here and put ^ right there, and then hit Ctrl-O to
write the file [confirm with 'Return'], and then Ctrl-X.
Then now it's going to say, "Please reboot to apply this update."
I'm going to say, 'sudo reboot', and it's applied that.
The next step is we need to get the HAT and the NVMe installed on the Raspberry Pi 5.
So I'm going to wait for it to boot up and I'm going to shut it down.
So we're booted back up and if I go back up to the EEPROM and look at it, it saved it
as '416' so we know we're good to go.
I'm going to exit out of here and shut it down.
'sudo shutdown now'
Now to get this HAT installed, I'm going to disconnect power because I don't want to accidentally
short anything out here.
So I'll unplug that.
And then the final hardware version, the production version they ship will have the right standoffs
and everything.
So I'm literally just setting it here. I don't recommend doing that. I need to put this Flexible
Printed Circuit board right here and these gold these gold pins need to face towards the Pi and
this arrow needs to be towards the power side of the Pi or you can line it up on the HAT right here.
The key with these cables is to get them in all the way. If you don't you're going to have signal
issues. So I pull up on this little latch and I wiggle this a little bit to make sure it's in
in there nicely and then I gently push down until it kind of pops in. Now this
cable is in and now I can put the HAT up here and work on this part. So I'm going
to do the same thing here making sure that this arrow lines up and this little
this little tab you lift it up to let it go. Put in the wire. There we go it's in
and again I kind of wiggle things a little bit to make sure I'm in nicely
and you might need to go in and out a couple times to just to make sure that it's all straight and
then push down on the latch that latches that in and
That's all you need for this board. And like I said, the production version
will be a little bit more secure than this. Follow the instructions that Pineberry gives you. I'm going to put the
SSD that I just flashed on the computer in
here and I'm going to go ahead and take out the microSD card.
You don't actually need to do this because it should boot off of here anyway
but I'm gonna do that just to swing for the fences.
Now let's plug in power,
and we'll see what happens.
So that was the first boot,
and then it's gonna reboot and resize the disc partition,
but now it's doing its boot,
and then it's gonna shut down again,
and boot up again, and then it should be up and running.
And this time it should boot up
and get to the Pi desktop, and there it is.
We're booting off that NVMe SSD,
and I'm going to do a little benchmark
just to see how fast this particular SSD is.
Now this is a PCI Express Gen 3 x2 SSD,
so we should be able to get a lot of speed out of it
if we can kind of 'overclock' PCI Express,
go to the Gen 3 speeds instead of Gen 2.
Okay, so I'm gonna log into this thing.
I just flashed it.
Yes, I'll accept that host key.
And if I do 'lsblk',
it should show that there's just an NVMe drive now.
If I had the microSD card in there, you'd still see that too.
But I'm going to run a benchmark on the NVMe
that I'm booting off of to see how Gen 2 and Gen 3 speeds
compare.
So I'm going to go to pibenchmarks.com.
And this is a nice benchmark to run on any storage device
on Raspberry Pi just to get a point of comparison.
OK, description is "Cytron Makerdisk".
Okay, so we got a score of 31,702.
I don't know how that score is calculated,
but basically we have 350 to 400 megabytes per second
maximum write speeds.
Now if I say 'sudo nano /boot/config.txt',
this is where you can change the PCI Express settings.
So I'm gonna go back to my blog post,
and you can see up here is where
the PCI Express generation settings.
Now this is an interesting thing.
If you boot off an NVMe drive,
You don't actually have to have this first line, but if you boot off of a microSD card,
you have to put this line in to enable that external port.
If you just want to use an NVMe drive for storage, you can put this or this, and that
enables the external PCI Express.
If you boot off an NVMe, it just automatically knows that.
That's an interesting thing.
I don't know why it's not enabled by default.
Maybe it will be someday–
this whole discussion that I just had with you will be moot! But I'm going to put that
in there anyway just because I like belt-and-suspenders.
And then we're going to change the generation to 3.
So I'm going to write this file and exit, and I'll show you if I say 'sudo lspci -vvvv'.
This is how you find out all the information about PCI Express devices.
So up here, this is our NVMe drive, "NVM Express" right here.
And if we look at link status, link status is this: it says speed, 5 gigatransfers
per second downgraded– x1 downgraded.
The drive is actually a PCI Express Gen 3 x2 drive, so it could do a lot more bandwidth.
We're going to try to get it to PCI Express Gen 3 x1, which the Pi supports—well, hopefully
it supports—which should be 8 gigatransfers per second.
Let's try that out.
I will say 'sudo reboot'.
The Pi looks like it rebooted, so I'm going to log back in.
Then we'll do 'sudo lspci -vvvv' and go up to the NVMe drive, which is right here.
Link status, 8 gigatransfers per second.
It's still downgraded because now it's x1 and the drive could do it x2, but
this should be faster.
So let us run that benchmark again.
We'll do this and we'll call it Cytron MakerDisk.
Now we can see the score is a bit higher.
PCI Express Gen 2 versus Gen 3 isn't going to magically make everything so much faster,
but throughput can be much faster.
We're getting 776 megabytes per second on this drive.
I've gotten up to around 900 megabytes per second on faster NVMe SSDs.
You get a lot more performance when you're doing large file copies and things.
It's pretty awesome that you can do this.
One other thing to watch for if you're experimenting with different PCI Express devices is you
can look into 'dmesg' and you can follow the log ['dmesg --follow'].
If there are any errors, they'll start popping up in dmesg.
Luckily, I'm not seeing any errors at all even though I just did a bunch of file copies.
It looks like this cable and this board are doing a good job with that.
Finally! We are freed from the restriction of microSD cards.
We can run any NVMe SSD. The HatDrive! Top allows you to do 2230 and 2242 like this one, and
the HatDrive! Bottom, which I haven't gotten around to a lot of testing yet—I just plugged
it in and made sure it worked—it will allow you to do 2280, which are much more common
and you can get a lot more high performance NVMe SSDs. Even if they're PCIe Gen 3, it's
going to be great for Raspberry Pi. So another thing that you can do with these is actually
adapt them to other PCI Express devices. So I'm going to be doing a lot of fun
exploration with PCI Express on the Raspberry Pi 5, including this—for those
who know what this is. So stay tuned and make sure you follow my Pi PCI Express
database on the on GitHub and on the website to see all the things that I'm
testing and to follow the community as they're testing things like graphics
cards and TPUs for AI and machine learning and Frigate and stuff. Lots of
cool stuff, storage controllers... and until next time, I'm Jeff Geerling.