(dramatic music)
- They're the same picture.
Except they're not.
This is a boring old U.2 drive.
You can find these puppies
with up to a four lane PCI Express Gen 4 interface
and up to about eight gigabytes per second speed
which is okay, I guess.
But it's not nearly as cool as this.
U.3 doesn't increase speeds at all.
And in fact, it could even be much slower,
but it could also be the same speed,
which doesn't really sound that great does it?
Until I tell you that it also allows cross compatibility
with SAS and even SATA drives in exactly the same slot.
What?
That's right my friends, this connector here,
is basically like putting a USB drive in a FireWire port.
And we've got both a drive from KIOXIA
who sponsored this video,
and this server over here from Supermicro,
and we are going to be testing it out.
(bright upbeat music)
U.3 is super new,
so while the drives are readily available,
hardware to actually plug them into,
is still pretty hard to find.
Our server for example, only supports U.2.
Fortunately KIOXIA provided us with everything
we're going to need starting with, aha!
This.
Now in the old days controller cards like this one,
usually had built in specialty processors on board,
to perform the parity calculations,
that protected the attached RAID Array from data loss.
Over time though hardware RAID card have given way to HBAs
or host bus adapters that simply convert,
your PCI Express slot into dumb storage ports,
leaning on the general purpose CPU to do the heavy lifting.
This kind of software RAID is more flexible,
more cost-effective and it's easier for administrators,
to troubleshoot and repair.
And these are actually some of the big driving forces,
behind the U.3 standard.
So this Broadcom 9500-16i is what's called,
a Tri-Mode Storage Adapter
It uses a PCI Express Gen 4.0 x8 interface
and can address up to 1024 SATA or SAS drives
using expanders or up to 32 NVMe drives.
Now let's have a look at the Broadcom Ellord.
Yes, that's right my friends,
it's a custom external U.3 testing enclosure.
That's designed for research and development of new drives
and adapter cards to make sure that they interface
with servers correctly.
In other words, this isn't something
that you can just go out and buy.
Normally the way it would work is a U.3 backplane,
like the one that's built into this enclosure
would just be directly in the server
behind the bays at the front
and you would plug right into them.
But as of right now,
we haven't found any servers with native U.3 support,
at least that we can get our hands on.
HP Enterprise is one of the only brands
that shipping U.3 technology currently,
though Lenovo has also expressed their support for it.
So here we go,
we just install our Gen 4 Tri-Mode card into our Caddy here
and plug that into our server.
Oh,
contact.
(fans revving)
Believe it or not, we haven't even turned it on yet.
(fans revving)
There it is.
Here we go.
You're ready.
(fans revving)
Now one little trick is normally you would run these cables
internally to the backplane
at the front of the device obviously.
Well, that's not a Tri-Mode U.3 backplane
so we're just gonna kinda...
(fans revving)
Yeah. Good.
It's good cooling.
Gotta have air holes on the side.
Okay there we go, very funny.
What am I Putin?
(feet stumping) (cameraman laughing)
I'm gonna be next level impressed
if this manages like hot-swap functionality.
Now you might have noticed
that even though we've only got an 8X card
we've got four, 4X capable cables coming out of here.
That's because that bandwidth can be shared.
It's in much the same way
that even if you did plug in a thousand SAS drives
obviously if you were actually hitting them all at once
they would have to share some of that bandwidth.
What do you think?
Is this just gonna like magically hot-swap pick up?
(fans revving)
Wait, probably not, cause we don't have a drive in it yet.
One moment please.
This just feels so wrong.
SATA port U.3 connector.
All right here we go.
(fans revving)
Just like that.
Let's put it online, initialize, tpt blah, blah blah.
Oh, oh, oh, oh, oh ouch.
60 megabytes a second reads.
Not exactly impressive,
but of course the performance of U.3 was never the point.
There's nothing you can do with an interface
to make a two and a half inch hard drive
suddenly magically fast.
The point is that we can do this.
This is a SAS SSD, and theoretically
I'm gonna pop that right in there, hot-swap,
and it's gonna just like work.
Wait for it.
(dramatic music)
There it is, Ooh. Ooh, Ooh.
Ooh. Ooh, Ooh
We can run a quick performance test here, there it is.
You can see it's a lot faster of course it's an SSD,
it's not a mechanical hard drive,
but the point here again is not the performance
it's to demonstrate
how just because you've got one physical connector
doesn't mean that you can't use different protocols over it.
And this is a much more appropriate analogy
for what we're doing today.
Then the whole USB FireWire thing
because those are completely different physical connectors.
SATA and SAS already used the same connector
just a different protocol.
So that's what we've looked at so far.
But now...
This one, now it's time to put in a Gen 4 PCI Express SSD
using again the same U.3 connector.
This is a CM6 from KIOXIA
which was the first available U.3 drive on the market
and it's rated at up to 6,900 megabytes per second.
Nice.
Just like our merchants is nice,
like the CPU Pillow lttstore.com.
Now you probably notice, I just shoved that in there
is hot-swap gonna work?
I actually don't know,
cause I've had some pretty bad experiences
hot-swapping PCI Express devices.
- Oh ooh ouch. Ooh.
- But that's supposed to be one of the benefits
as the technology has matured
is we're supposed to be able to take
any kind of drive we want, chuck it in there,
hot-swap, cold-swap.
Let's see if it's in here.
There it is, online and boom.
We've got our E drive.
I'm expecting this to be a lot faster and it is.
Now...
Wait a minute, you might say
That's not 6,900 megabytes per second.
That's not nice
but we're not actually reaching the drives full speed
because of two bottlenecks in our setup.
One, this system has two EPYC 7702 processors
pretty cool.
But although each of them has 64 cores and 128 threads.
Their single core performance isn't particularly impressive
because of their low clock speed.
And with this setup on Windows Server 2019
we're not taking advantage of the drive's full potential
but we can still try out U.3 hot-swap.
Oh, interesting.
So, okay, we're taking it one step further
with the hotting and the swapping apparently.
I'm just gonna pull out a SATA drive.
I'm gonna pull out an NVMe drive.
And theoretically they're just gonna come back.
My script says react to the hot-swap just merging.
We've got OO, we've got E, no the SATA drive is missing.
What if I re...
seriously?
Boom all three of them are there.
So we've got SATA OO and E.
OO E.
Okay then what does all of this mean
for me the average consumer.
For the time being, probably not much.
I don't expect that U.3 is gonna be sweeping in
to replace the popular M.2 gumstick form factor
in your home gaming PC anytime soon.
But in the data center
it offers a couple of clear benefits.
One of which is visible right on the face of it.
See these cooling holes.
I call them speed holes
because adequate cooling allows these drives
to perform their best,
even when they're under heavy load 24/7
like they would be in an enterprise or data center setting.
The other thing that the larger form factor does
is it allows the drives to have more capacity
because they can physically solder
more non dice to the thing
than can fit on a simple little gunstick.
So for the data center, these are gonna be huge
not just because they have a single connector
that natively supports multiple protocols.
That's not anything new.
We already showed that with SATA and SAS.
The big deal is that it allows a single server
with a single type of bay in the front
to serve all kinds of different roles.
So manufacturers, instead of having to have different skews
for like a slower bulk storage box
and then a faster all in NVMe one
are gonna be able to have just a single skew
simplifying their product development
and simplifying their product stack
that data center administrators
can just deploy however they want.
So I want this one to be all NVMe.
I want this one to be all SAS.
I want this one to be NVMe accelerant over here
and then SAS mass storage over there.
It's totally up to the admin,
and that is super, super cool.
Another fringe benefit actually that we heard
from one of the techs that's working on this
is that it makes
trying out different hardware configurations
and troubleshooting way simpler
because you can just use one machine
for all of your test benches
instead of having to like move giant servers around
as you're testing out different drives and config,
so that's another thing.
Thanks to you guys for watching.
Thanks to KIOXIA for sponsoring this one.
You guys can get more details about their CM6 drives
and the rest of their technology at the link down below.
If you guys are interested in the backstory
behind my nightmare experience
hot-swapping PCI Express devices,
you can check out the video
that we have linked in the description as well.
This was much smoother.
(bright upbeat music)