- Is this just like a-
- Oh, yeah, yeah, yeah, stop, stop, stop, stop, stop, stop.
- But where are they?
(hardware clattering)
Really?
By an order of magnitude, this
(boxes thudding)
will be the single, most expensive unboxing we've ever done.
Inside these boxes, this crate, this server rack,
this additional box, and this final box over here,
this monster server from Supermicro
is everything that we need
to deploy a server with a petabyte of SSDs.
Kioxia provided the SSDs, they sponsored this video,
it has been a year in the making,
and we are about to see performance to likes of which
we have never seen and may well never see again,
realistically like-
- [Jake] Yeah.
- In our lifetime.
- [Jake] They're not even just SSDs, they're NVMe SSDs.
- Let's bring back (beep),
that's a series we haven't done in a while, (beep).
- (beep).
- [Linus] (beep).
(bright music)
- I think that server's worth like $100,000 so-
- $100,000 server you say.
- Something like that.
- Well, let's move it gently then.
There's gonna be a whole follow up video detailing
the exact architect, yeah. - Follow up videos.
- There's gonna be multiple videos detailing the setup
and architecture of a petabyte of flash
because while Kioxia does make
such high capacity NVDMe SSDs now,
that you could pretty easily build a server
with a petabyte of flash accessible to it,
but in order to get the full performance
out of it. (Jake laughing)
(both laughing)
Oh my goodness. - Oh my God.
- You gotta do something a little special.
- I don't think I'm gonna let you pick this one up.
- You know, you gotta give Nvidia credit.
(Jake chuckling)
- It looks pretty special.
- Does their hardware have sex appeal
or does it have sex appeal?
- I think this is just the module,
this actually slots into the main server.
- So what are we looking at here?
- So each of these is an Nvidia A100, 80 gig,
so we've got eight of them, Nvidia linked together, that's-
- 640 gigabytes of VRAM.
- Yep.
- Video memory.
(Jake chuckling)
These cards alone are probably worth more
than any other individual server
we own. - I think this thing,
this unit is almost $100,000, just this part of it.
- And what are these?
Are these just like some kind of-
- [Jake] I think that's just
power delivery. - I expect-
- They asked us not to take them off but-
- Oh really?
- We already have have it so like,
we can just do it anyways.
- I mean, Nvidia can't hate us anymore can they?
- They will.
- No, they probably can, you know what?
Let's not piss off the enterprise guys because-
- This looks like power delivery.
- Can I just say that it's all sponsored by Kioxia
who provided the flash
but Nvidia has been absolute bros and broettes
about this whole thing
'cause we wouldn't have been able to do it without them.
So, I guess yeah, let's pick it up.
- Why don't we move this off of the server
so we at least have somewhere to put this?
- Well, I'm gonna know what's under it,
oh wait, that's it. - The server, ah, yes.
- Oh, that's why the box was so big.
- Yeah, oh, look at those rails.
- I know right, these are chunk-tastic.
- Let's just, yeah, let's just lift this out of the...
Put it on top of that?
Sure. - Sure.
- That's safe.
- So how did this even come to be?
Now, I've been babysitting this for-
- A year.
- A year but ideally wasn't there
for the first conversation.
- What happened was,
Kioxia has worked with us a number of times
and they were looking for
some kind of cool SSD-related video to do.
- Cool, you say.
- I took an absolute long shot,
'cause this was after we'd already seen the appetite
for a petabyte project on the channel
but they're a flash vendor and I was like,
"Well, it would have to be SSDs and it would have to be,"
and I thought they were gonna come back
with something way smaller, don't tell Kioxia
but they would've gotten away with like a quarter of this.
(Jake laughing)
But I was like, "Well it's gonna,
it can't be smaller than what we've done before,"
and they were like,
"Oh, this is actually a very interesting
engineering challenge from our standpoint,
let's see who we can partner with
to try and make this happen."
- And the answer was Supermicro, AMD, Weka,
fricking everyone you could possibly name
in the server space.
- So, how does this fricking work?
Okay, let's lift this boy up,
we're gonna start at the back which is a little unusual.
This is our head unit.
(Jake laughing)
(both laughing)
This is PCIe over fiber,
because the SSDs don't actually go in the control unit,
this is where all the compute lives.
In order to do this kind of throughput,
you need a very, very exotic, very powerful solution.
The SSDs themselves-
(Jake laughing)
- This is the size of an ATX power supply.
- Are gonna live in additional units
that we're gonna unbox in a few minutes here.
- Holy,
just- - How many watts are these?
- (beep).
This is a 3000 watt power supply.
- Can we even power this?
- I've never actually seen one of these plugs properly used.
(Jake laughing)
- I'm just curious what is in here?
Oh. - Oh God.
- Oh, wait, what?
is this just like a-
- Oh, yeah, yeah, yeah, this'll be like a...
Stop, stop, stop, stop, stop, stop.
- So you gotta unscrew it then.
- No, sorry, I meant this part.
- Oh, yeah, okay, now you're right.
- Stop though, you gotta take this side out.
- Well, I thought you were gonna.
Oh. - I don't have a screwdriver.
- What are you doing?
What are you doing over here?
You're just-
- Waiting for you to hand me the screwdriver.
- Well, hooey.
- Hooey.
- Okay. (Jake laughing)
- Oh my God, what even is that?
- So here's what we're looking at,
we've got our PCIe over fiber cards here.
- These are just Nvidia ConnectX-6, 200 gig cards.
- Wow.
- There's eight of them.
- And then we've got more PCIe, so you can see we've got,
(Linus groaning)
there you go right there, okay?
We got a little bit more PCIe
to these NVMe base at the back here,
so you got four more of those
and then you've got a little bit more PCIe, okay?
Up here, and that's going to this
which is probably like a dual 10 gig card
or something like that
that's just for, - It might just be gigabit,
like for management.
You know what?
I know you like that much PCIe, I gotta tell you,
this one, there's more PCIe.
(Jake laughing) - Oh, are you serious?
So this unit can take a total of 10 NVMe drives
and then it has another eight PCIe 16x slots,
then there were those NVMe connectors I didn't recognize
to the sleds at the back.
- And then there's all the A100s.
- Oh my goodness.
So do we slide it in?
- [Jake] I think we should put it in the rack first.
- They have to be EPYCs of some sort.
- [Jake] Oh, they are,
dual 64 part. - Okay.
- Where are they?
Oh, they're in the top, they're in like the very top.
- Yeah.
- So there's the PCIe lanes for the A100s
that just sit under it, very cool.
- I don't know how to take this out 'cause kinda-
- I wonder if it got hit a little actually in shipping.
- Looks a little
worn, yeah. - Looks a little bored.
- Oh boy.
- Oh, there's a screw right here.
- Oh, there's your problem.
(Jake laughing)
- You got this?
Don't drop it. - I got it.
I got it, dual 64 core EPYCs,
I have to assume
that that slakes. - That means 713.
Is this two terabytes of Samsung memory?
- The spec sheet said 256
but I don't actually know who sent it.
Okay, here, let's just pop one out.
- Oh, boy.
- They're not gonna like this.
That's a 64 gig DIMM, dual rank, 3,200 megahertz Samsung,
well it's Supermicro memory, so-
- Yeah,
- This is a qualified-
- So 64 gigs, that would make this yeah-
- 32, 2 terabytes.
- 2 terabytes of RAM.
- Ah, look at those,
these things must scream. - Fricking real.
- [Linus] Yeah, these fans-
- What are those, 40 mills?
- Gotta be absolutely ridiculous.
- [Jake] This is a lot of hardware in one box.
- On fricking real, love the air flow management here,
gotta put basically all of it through the CPUs.
- The RAMs,
I could have a little bit. - The Rams are slightly,
yeah, come on, maybe there's a little bit of cooling.
- Look, its like a tiny little spot there.
- Yeah.
- These VRM heatsinks are like, "I don't need much."
(both laughing)
- There's business level hardware
(hardware clattering)
and then there's enterprise level hardware.
This thing is unbelievably heavy.
There is a lot of fricking cooling in it,
it's like a solid ass block
of aluminum. - Okay, it's very heavy,
yes I understand, let's put it in the goddamn thing.
Huh?
- How many watts-
- Oh, look at that, they got flappers, what the hell?
- Holy crap, the fan itself is this thick.
- Yeah, wait now, is there two in there?
- Crazy.
- May be two in there.
- So there's two blades, see this?
- Yeah,
there's two blades. - So you've got this blade
is swept this way.
- They spin independently.
- And then this one is swept this way
because if you want more static pressure,
you can't put two fans that spin the same way,
there's some kind of aerodynamics thing that makes it
so that doesn't work very well,
they have to be spinning counter to each other.
We didn't really consider
that when we assembled this up here,
we were going to have to bring it down.
- Yeah.
- Okay,
plan. - Where are we gonna put it?
- We're gonna put it on top of the rack for now.
- I'll take the light side.
- I see, we ready?
- Oh my God.
- Okay, if we drop this, like, it's game over.
- No, no, we're good, here's what we're gonna do,
this thing already got beat up to hell in shipping,
so we're gonna bring it down like this,
go ahead and put it on, yeah.
- Here you go little guy.
- Oh.
- I dunno if little guy is the right word for it?
- Absolute unit guy.
This is when you have all the budget
'cause it's not just powerful, it's dense,
you gotta remember in the data center,
density has a ton of value because building a data center
is the most expensive thing about building a data center.
- So each of these one use servers
are gonna be our storage hosts, there's six of them.
- Six?
- Six.
We gotta put 72 drives in here.
- Now you might think, "Hey guys, this is Austin."
(recorder beeping)
You might think, "Hey guys, why didn't you just pick servers
that could take more NVMe SSDs?"
Well, here's the thing,
once you've got high performance enough SSDs
and these are all Kioxia CD6 PCIe Gen-4x4 NVMe drives,
you run out of CPU power
as well as PCIe for networking as well,
so you can't just put as many drives as you want
in each server, you gotta add more compute,
you gotta add more networking.
- Yeah, this dual processor, so these are.
(Linus groaning)
oh gosh.
- 32 cores each and a far more reasonable,
I'm guessing this is 512 gigs of RAM.
- [Jake] It says 256 but I think this is wrong.
- Yeah, only 256, and then of course we've got
the Nvidia connectX-6, 200 gigabit network cards
that are gonna handle our NVMe over,
I guess actually they're not fiber, are they?
'Cause they're so close.
- I think we're gonna be using direct detect,
but you call it NVMe over fiber.
- It's NVMe over fiber, just with fiber, you can go farther.
- Is that the boot SSD
or what? - Oh, it's adorable.
- Oh my God it just slots right into the-
- It's a little, see there, thumb drive, come on out.
- [Jake] I think there's probably a lock on it.
- Oh yeah, there is, look at that,
that's so cool. - These are super cute.
- Yeah.
Oh look, they even included a USB Type-B port
so you could run and rate on this thing.
- Yeah, so the reason that people use these
instead of an internal USB drive
is just that the grade of non-flash
that gets used for a device like this tends to be higher,
and while honestly, in a real enterprise deployment,
a boot drive failing is probably not the end of the world.
- [Jake] A lot of the time they're booted over fiber too.
- Yeah, it's just, why, right?
Just use something better.
Now, you might be wondering why we went AMD EPYC
which means you probably haven't been paying close attention
over the last few years
but basically these will boost up to 3.7 gigahertz
giving us tons of compute, 64 total cores
and they have lightning fast
not to mention a ton of PCIe connectivity,
so every device in pretty much this whole rack I think
is gonna be running at-
- [Jake] Full speed.
- Maximum speed.
- [Jake] Definitely will.
- We don't want any bottlenecks, we can't have bottlenecks.
Before we move on
to a couple of more fun things I noticed about this,
these are 20 watt fans,
each of them (both laughing)
80 watts,
that's 150 Watts in fans. - Wait are these,
is that just, oh yeah, they're like the same type.
- Yeah, EPYC has so much PCIe
that you can just put a PCIe 16x slot
just chilling in the middle of your server,
(both giggling)
just hanging there.
Yes, this is a 200 gigabit card, one of two.
Each of these is gonna have
a 400 gigabit per second connection to our head unit.
(Jake giggling)
- The best part, there's that slot,
there's another one back here.
- Oh, wow,
well that one's at least accessible from the outside.
- Yeah, yeah, yeah.
- This is just hanging.
- They're like, "Man, you wanna put some NVMe in there?"
- You only live once man.
Now for the next part,
you guys saw this crate we were using as a table?
(Linus groaning) - Oh, it's very sharp.
- What the heck is all this?
- Wait, hold on, go down.
- What?
- I think there might be more servers in here.
- Kioxia, oh, these must be the drives.
Oh, what's this from Mella?
Oh, excuse me, excuse me,
the box says Mellanox but I mean to say Nvidia.
Nvidia acquired Mellanox
somewhere in between this box being printed
and us doing this project.
(Linus chuckling)
- What could be in this box, hmm?
Oh. - Wait, what?
Whoa, what?
- We originally asked Micron to send memory for this project
but Supermicro was like, "Oh, it's not qualified yet."
(Jake babbling)
- Wait, what?
- I think these are just eight gig DIMMs.
- Eight gig 3,200.
- Can I take some of this home?
Is it all eight gig?
- I don't know.
- Well, thanks Micron, what's behind this door?
- In fairness though,
there are some really cool cables in here,
there's power cables, those aren't the cool ones,
and then there's these guys,
these will do 200 gigabit per second
and these are the direct attached copper cables
that we're gonna be running between our head unit
and our SSD units.
Now, there's networking gear that's like pretty cool
and then there's this, this is a Mellanox,
well, Nvidia 3,700 series. - Oh (beep).
- 200 gigabit switch, that's right,
each of these ports, 200 gigabit per second.
- And it can do it all at full speed at the same time.
- This thing is bananas.
- [Bystander] Sorry, what?
- They will all do
200 gigabit per second.
- It's always fun
when you start measuring your network switches in terabits.
- Yeah, the fans are 30 what?
31 fans.
- I wanna see how big are the power supply?
(Jake giggling)
(hardware clattering)
- Really?
- I just looked at the spec sheet of that switch,
12.8 terabit.
- Ah, oh, they are, oh, that's hilarious, we were look...
Kioxia, are you guys new to working with like-
- [Jake] No, it matters, the position matters.
- Oh, the position matters.
- [Jake] Yeah, don't switch 'em around.
- Oh, I was gonna be all like
"Kioxia, come on, what's the fun of unboxing all the SSDs
if you don't put them in a box," but, okay,
the position actually matters a lot for this configuration,
so we'll just take a look at this drive
and carefully put it back in here.
Each of these is 15.36 terabytes
and can deliver a total throughput
of six gigabytes per second
which is impressive considering that we're gonna have
12 of them per server,
so that's 72 gigabytes a second per server,
absolutely mind-boggling numbers
but there's even better stuff,
so it's obviously using a completely custom solution,
everything from the controller to the flash
is all Kioxia through and through
and this puppy can do a million IOPS per drive,
so by the time we're done with this,
we are going to be putting
everything we have ever done before to absolute shame.
If you don't watch our channel regularly,
you might not have heard of Kioxia
but we've actually worked with them a fair bit,
they were the first to deliver a U.3 capable drive,
the first to deliver PCIe gen-4.0 drives,
they are formerly Toshiba's flash division,
so they've got literally decades of experience
working with this kind of technology,
and I'm really excited to see what these drives can do
at their absolute peak.
Of course speed's not the entire story
and I'm obligated to tell you
that they are of course using the same high quality flash
that they use in their high-end enterprise SAS
and other NVMe products with the same reliability features,
and of course also they've got support for everything
from nonencrypted drives to sanitized instant erase,
self-encrypting drive, SCD FIPS to level 2 certified,
if you need it and it's an SSD,
Kioxia has something for you.
- [Jake] Or something better
than you were thinking. - Or something better.
The final piece of the puzzle is our server rack sort of.
(server rack clattering)
Let's talk about how we're powering this.
(server rack clattering)
Oh my goodness, what the heck is going on here?
Is this the most whack power cord you guys have ever seen?
Here it is, next to a human hand for scale,
okay? - Well, I mean,
I don't know about human, there's a normal hand.
- Fricking ridiculous.
- These are L6-30s, so that's 2-40 volt, 30 amp,
each of 'em.
We didn't actually have a place to plug these in.
- Ah, yeah.
- We're gonna be stealing,
I think one of the plugs from the welder
and then one of the plugs from the mill,
I got infinite cables to make these giant extension cords,
they're like just as thick, maybe thicker,
so yeah, we can't use the welder or the router
when we run this rack but I think that'll be okay.
Infinite cables also provided a hundred meter 100 gig cable
so we can plug it into the switch back there.
- No.
- I guess we'll have to like run it outside, but yeah.
- Guys, this is gonna be a really fun series,
we're gonna go through the hardware, the software.
(Linus giggling)
- We're gonna show you AI benchmarks.
- Yeah, it's gonna be great.
4 times 12.
- 384.
- 300
okay, plus another 128. - Plus another 128,
so 512 cores.
(Jake laughing)
That's the same amount of RAM
we have in our main editing server.
- We're gonna have 512 cores and a terabyte of flash.
- It's a little bit more than that.
- This is gonna be-
- A terabyte of flash?
- Petabyte,
sorry, my brain broke. - Like 1.1 petabytes, yeah.
- That's not possible.
- Yeah.
- Petabyte of flash.
- 12.8.
- Petabyte of flash, that's crazy,
no one would be crazy enough.
Final shout out to everyone who helped.
- Kioxia, Nvidia slash Mellanox,
whatever you decide you wanna be, AMD.
- Micron.
- Supermicro, fs.com,
Infinite Cables. - Infinite Cables.
- Whoever-
- And Jake Tivy, Jake Tivy, who's been cool
and all of our contacts
at those places- - I'm taking all this home
afterwards.
- You guys have been so patient, this has been-
- It's gonna be a trade show at my house.
- If you guys are enjoying this series,
maybe go check out the original petabyte project series
that's pretty cool too. - I Actually don't think
the panel in my house has enough capacity to run this.
- I don't think mine does either.
(Jake wheezing)