I've been running Raspberry Pi clusters for seven years. My first few iterations were
a bit of a jungle in terms of wires, so my current cluster uses Power over Ethernet HATs
to cut down on the clutter. And just last month, I installed my Pi cluster
in my network rack.
But let me show you one of the big problems with my current Pi rack. Even though it only
takes 1U of rackspace, it only holds a few Pis, an look at what's behind it... a LOTTA
wasted space!
The Raspberry Pi 4 model B is a small hobby board. It's not meant to be racked up like
a server. But what if there was a Raspberry Pi that
used all the depth available in a rack for built-in NVMe storage, a TPM module, built-in
PoE, and more... And what if we could fit 16 of these Pi servers in the same space as
5 Pi 4s?
Well, Ivan from Uptime Lab had that idea, and built this, the 0.6 beta version of his
Raspberry Pi Compute Module 4 Blade. He actually hand-made two of these blades
and sent them to me after one of his earliest prototypes was damaged in shipping. He hand-soldered
the boards in his workshop, and this time they both arrived intact.
The blade uses a Raspberry Pi Compute Module 4, so you get all the performance of a normal
Pi 4 model B, but it's specialized so it can use native NVMe storage, be rack mounted,
and increase ARM compute density.
For months, I've been waiting to get my hands on one of these boards, because for what I
do, it's one of the most exciting CM4 boards I've seen.
Think of this: using blades, you could cram 64 ARM CPU cores, 128 gigs of RAM, and 16
TB of high-speed NVMe storage in 1U of rack space. The same size rack only holds up to
six normal Raspberry Pis! The extra depth of the blade allows for some
really cool features.
Take, for instance, this built-in PoE daughtercard. When you're racking Pis, having to run separate
cables for power and network is really annoying. So the blade has a built-in Silvertel Ag5300
PoE+ board.
Using an off-the-shelf solution for PoE means the power circuit is already validated and
can supply extra power to the Pi and NVMe drive—more than 20 Watts in this case—if
the board needs it!
It also has a full-size M.2 M-key slot, so you could put any size NVMe SSD into it. Or
even stick in an adapter and put in a WiFi 6 module or most any PCI Express device.
The board even has a TPM 2.0 module built-in, so I guess it'll run Windows 11... but more
seriously, you can use the TPM module to store cryptography keys, verify the boot process,
and help with encryption.
It also has a partial GPIO header that lets you plug in a Real-Time Clock or RTC module
so you can get a persistent clock, or even a ZYMKEY 4i for full-disk encryption and secure
connections to services like Amazon's IoT Gateway.
Towards the rear, there's a PWM fan header, so a rack enclosure can have speed-controlled
fans to keep the Pi cool.
The blade also has more standard ports, like a single HDMI port for plugging in a display,
a single USB 2.0 port for data access, a USB-C port for flashing eMMC Compute Modules, and
a microSD card slot for Lite models without onboard flash.
Rounding out the visible features, there's a UART debug header on the top, making it
easy to get console access, even without an HDMI display. And there are four LEDs: first,
the standard activity and status LEDs on every pretty much every Pi, but then there's also
an ID LED controlled by a GPIO pin, and an NVMe activity LED.
I have a Samsung 970 EVO SSD, and a Compute Module 4 Lite with 4 gigs of RAM and built-in
WiFi and Bluetooth. To set things up, I'll pop the CM4 on top of the board, into the
100-pin board-to-board connectors, then insert and screw down the SSD into the M.2 slot.
I flashed Raspberry Pi OS 64-bit to this microSD card, and I'll slide that into the microSD
card slot. I can actually boot from NVMe storage, too, and I covered how to do that on a similar
board earlier this year. There's a link to that video in the description.
In this case, I'm okay just booting off the microSD card—at least for now. All that's
left is plugging the board into my PoE+ switch and letting it boot up.
If you're gonna use NVMe storage with the Pi, it's best to use PoE+ since it can supply
more than 12 Watts to the Pi. But if you're not using NVMe, you might be able to get by
with a cheaper PoE switch like this small Netgear switch I'm currently using for Starlink.
Once it's plugged in, I can find it on my network and SSH into it.
Logged into the Pi, I can see all the basic stats: lspci shows the NVMe drive on the PCI
Express bus. lsusb shows nothing because you have to actually enable the USB port by adding
this line to your boot config file on the Compute Module to activate it.
I formatted and mounted the SSD, and doing a few quick performance tests, I was able
to read and write data to it at 415 MiB/sec, which is as fast as the Pi's internal bus
allows it to go. If you put the same NVMe drive in a faster computer, it would be able
to perform a little better, but it's already a HUGE improvement over microSD or eMMC storage,
and it's even an improvement over NVMe drives in even the best USB enclosures.
To be clear, this is native NVMe storage on the Pi. It's not using an adapter like the
Argon M.2, or other storage adapters built to work with the Pi 4 model B. This M.2 slot
gives you the fastest storage possible on any Raspberry Pi.
To use some of the more advanced features, like an RTC or the built-in TPM module, you
have to do a little extra setup work. Most end-users don't know or care much about
TPM, but it's basically a secure element that can do things like help secure the boot process
or store crypto keys. But since the Raspberry Pi's bootloader is closed source and doesn't
support the TPM module out of the box, it's a little tricky to use it to its full potential
on a Pi.
Anyways, I was able to enable the module by adding this line to the boot config file and
rebooting.
To verify it was working correctly, I cloned Infineon's TPM Toolbox to the Pi, compiled
it, and ran it, and it gave me this output.
So it seems like it's working, and it's identified as an SLM9670... but I'm not much of an expert
at secure computing on Linux, so I'll leave it at that for now.
Next up, you can control the ID LED in software, because it's attached to the Pi's GPIO pin
23.
Running these three commands lets me turn it on.
And then I can echo a "0" to the value to turn it back off. Using a Python script, you
could manage the LED in software to indicate custom statuses, or to help identify a particular
blade.
ID LEDs are very common on servers, since it's often important to identify a server
in a rack where there might be twenty servers that look identical.
Now, one of these boards on its own is pretty neat. But these blades are built to live clustered
in 1U rackmount enclosures. I spoke with Ivan about long-term plans, and he already has
a mini-rack enclosure for it, complete with integrated Noctua fans, but he hasn't settled
on a full-size design.
But he did send me a file for 3D printing a desk stand, capable of holding up to four
blades. I printed this enclosure and put the two version 0.6 blades Ivan sent, plus that
early beta board that broke during shipping, and even this, the _original_ prototype board
design. It's a lot different than the version that's getting close to production!
It's always fascinating to see the progression from idea to prototype to production. And
I love the personality Ivan injects into the designs, seeing the vulkan salute, the DeLorean
from Back to the Future, and even Grogu! I'm guessing the final design won't have a copyright-infringing
silkscreen though.
So this board is awesome. But here's the bad news. These are two of only a few blades in
existence. You can't get them—yet. Right now Ivan is working out a few more bugs
so he can build the production version, and his plan is to start a Kickstarter campaign
soon. Since that campaign isn't up, I'm going to add a link in the description to his email
list, so you can be notified if you're interested.
Once the campaign is live, I'll be sure to update the link.
I can't wait to get my hands on a few more boards, and rebuild my Pi cluster for a new
2022 edition that crams tons of ARM compute power into a 1U rack space, with tons of high
speed SSD storage!
Follow Uptime.Lab on Instagram to find out more, and until next time, I'm Jeff Geerling.