- Okay, hold on, Oh, God.
After being gravely disappointed
when some of my fellow YouTubers showed off their
30 to $60,000 Jellyfish network storage devices,
I set out with exactly the same budget
to build something that could absolutely
run circles around the thing.
Now, given that we could afford SSDs
for what they charged for hard drives,
it wasn't surprising when we, fried their Jellyfishies.
But for some reason,
apparently people are still buying
enough of these jellyfish things.
That OWC recently purchased LumaForge,
the company that makes them.
Did we not make our point, people.
Do you have any idea how much money we could have saved
if we didn't go full stupid,
and build the thing purely from solid state drives?
The answer is, $45,000.
That's right,
on this table is everything we need to build a machine
that is better than the Jellyfish
in just about every possible way,
and costs one fifth as much.
So, let's fry the Jellyfish once and for all.
And we're gonna do so with the help of our friends
over at Pulseway.
Big, thanks to Pulseway, not only for sponsoring this video,
but for building such fantastic remote management
and monitoring software.
With Pulseway, you can manage
your whole organization's IT infrastructure on the go
with their mobile apps,
and even create and deploy custom scripts
to simplify your job.
Check them out at the link below.
(dramatic music)
I'm sure you're thinking,
Linus, you can't possibly make something
for one fifth of the price,
without some pretty big compromises on the hardware.
I mean, when it comes to a gaming PC,
yeah, you can save a few hundred bucks building your own
over getting someone to do it for you.
But if you try to cut 80% of your budget,
you're gonna end up with something like this.
Where is it going?
- Really impressive.
- Ah, I'm stuck.
No professional would use that.
Well, you're right oh.
(clears throat)
Excuse me, wrong, so very wrong.
LumaForge has a few different models.
But, we're gonna be targeting
their Tower/Rack models specifically, because
well, partly they're the most ridiculous.
And also partly, we like the idea of rack mounting storage
and we actually intend to use this thing in the longterm.
So we're gonna replicate the 240 terabyte Rack model
with the optional boost package.
We'll talk about what that marketing nonsense
actually means later.
For our case, we've gone with iStarUSA E4M24HD,
which includes even more drive slots
than the Rack, or Tower Jellyfish,
and should fit the rest of our hardware, nicely.
One challenge though, is that
LumaForge is not very forthcoming
about the specific components of their systems.
Lest, somebody Googled the part numbers
and figure out how hard they are being fleeced.
But, there are some things that we know
from last time around,
and some others that we can glean from their product pages.
First off, LumaForge's big selling point
is that it's all direct attach.
You just grab your network cable,
plug it directly into the machine
you wanna edit on and bam,
it works just like USB, with no network switch required,
even for multiple users.
But, being the reasonable person I am,
(screaming)
I just can't fathom why anybody would want
a setup where you have more than, say,
two or three people editing off of the same box
and you don't just run a network switch.
Mikrotik even has a 12 port 10 gig switch for only $500.
So for the amount that I'm paying for LumaForge's
special sauce here,
I could buy a pallet of 10 gigs switches.
But okay, I'm gonna put myself in the shoes,
or sandals, as it were,
of the person who thinks that that's an ideal workflow.
And I'm gonna grab two Intel Quad Port
10 gigabit network cards.
That gives us a total in this system, of 10 10 gig ports.
For storage, LumaForge uses ZFS,
a file system and volume manager wrapped into one
that they've got running inside CentOS,
a Linux-based operating system.
Like last time though, we are taking a different route
and we're gonna go with TrueNAS Core,
formerly known as FreeNAS,
which also uses ZFS as its file system,
but has a more user-friendly management interface.
Based on their specs page,
it appears that LumaForge is using groups or VDEVs
of 10 drives running RAID-Z2.
What that means, is each group of 10 drives
in your Jellyfish can sustain
two drive failures, before you actually lose any data.
And since they have 20 12 terabyte drives
on the 240 terabyte model that we were trying to replicate,
we could suffer up to four total drive failures,
before we actually lost anything.
That is, as long as they failed to in each group.
If we lost them like this,
we'd be plumb out of luck on all the data on these drives.
We're not too worried about that though,
because Seagate sent over 22
of their EXOS 12 terabyte enterprise drives,
which boasts two and a half million hours
meantime between failure,
low power consumption,
and, excellent resistance to vibrations
from nearby drives, which
(clears throat)
there's gonna be a few of them.
Also, the five-year warranty is a nice touch.
That's a Seagate part number.
But HP Enterprise model, HP Enterprise.
Now, funny story.
We would have actually used Seagate's IronWolf Pro
consumer grade NAS drives,
but, at the time of filming
these Exos Enterprise drives were actually cheaper, so
weird flex, Seagate,
but, okay.
Now, according to LumaForge's website,
the performance boost package
doubles the stock 128 gigs of RAM.
So, we're gonna be running 256 gigs
of 3,200 megahertz crucial registered ECC memory
in eight channel mode.
That should give us
a really nice balance of cost and performance.
And, we haven't had a single stick of this stuff fail
in our last handful of deployments.
So we absolutely love it so far.
As for the, ingest capacitor,
well, that's a fancy name for a write cache
which ZFS calls a SLOG.
This limited amount of faster solid state storage
prevents the hard drives in the array
from getting bogged down when data
is getting written to the machine
at the same time that somebody is trying to edit off it.
Like for example,
when you're ingesting footage off of your cameras.
Now we don't know how big the drive they use is,
but we figured a two terabyte NBMe drive
from Sabrent would be plenty for most injust workflows
given that camera media larger than that
is pretty unusual.
As a read caching, you see I'm holding two drives here.
ZFS utilizes something called ARC,
which is a portion of the system memory,
that acts as a live cache for frequently
or recently used files.
Problem is, when it comes to media work,
they're full of large files
and you can have enormous projects.
So even our 256 gigabytes of RAM could be easily filled up.
Thankfully there's a special feature called level 2 Arc,
that allows us to add a second NDMe drive.
That will be a well slower than memory
but still faster than hard drive caching layer.
Before you start reading directly
off of the spinning rust.
Lumaforge doesn't actually include this in their systems
but since the motherboard that we chose
has an extra M.2 slot,
and we have lots of budget leftover to play with
you figured, Hey, how many times do you live?
However many times you live
it's not enough times to risk downtime
or data loss due to a power supply failure.
That's why we actually deviated
from the Jellyfish's original design.
This right here, is the twins pro 700 watt from FSP.
And it is super cool.
Like the 1000 watt consumer grade unit
that LumaForge uses it fits in any standard ATX case,
but unlike that one, it can sustain a complete failure
of one of the two redundant power supply modules
that goes into the back like.
So, another nice benefit of this thing
is that it actually allows you to plug
into two independent power sources.
If you had say, for example,
a UPS and a just direct into mains power,
one really you should just have two UPS systems.
And the best part about it this thing ,
is that we were really concerned about noise.
But it ended up being not quiet,
but reasonable enough that you could have
it in an open office space.
And it wouldn't be a huge nuisance.
Now, obviously if you're going to have your Jellyfish Fryer
like on your desk right next to your head,
you might want to choose something else.
But never fear, because you can afford literally
the most expensive enthusiast power supply on the market,
with all the money you saved.
And then you can go get it gold-plated at a local shop.
One option we'll upgrade
would be to pull out these hot swappable sleds,
and replaced the Sunon fans that ship with this case.
Something like a Noctua's industrialPPC,
might have a longer lifetime.
But for our purposes,
it's not like Sunon is a bad fan manufacturer.
And given that we're going to be putting it
in a server rack anyway,
we're just not that concerned about noise.
And the convenience of having these plugged right
into these little daughter boards
that go into the drive backplane is pretty good.
One thing that's a little funky about our configuration
is that this power supply has fixed cables on it.
And unfortunately, it's only got I think like,
two molex connectors.
Well, we need a total of a
(clears throat)
10 for all of the hard drives that we have
installed in this thing.
So, what we'll be doing is adapting SATA to Molex
because it's the only connector on here,
that's still got five volt built into it.
Otherwise we love to use the PCI express connectors,
which you don't need in a storage server.
Now, that doesn't sound great,
but as long as you pay attention to the max amperage,
that these cables and these connectors are rated for,
it should be okay.
As for the rest of the machine,
we chose AMD EPYC platform
due to its excellent price performance and features.
Which is basically everything.
Specifically, we've gone with a 24 core EPYC 7302p
cooled by Noctua adorable little NH-U9 SP3,
which is one of the only cooler specifically engineered
to cover the massive heat spreader on an EPYC CPU.
For our board, we've chosen ASRock Rack EPYCD8-2T
This board is great for a number of reasons.
I mean, just look at all that expansion.
It's freaking crazy.
Have I mentioned by the way,
how much I love EPYC, because thanks to EPYC,
we've got 128 total lanes of PCI express to tap in
That gives us all the slots we need,
for our HBA which is what we plug
all our drives into.
Our 10 gigabit NIC, including the two
that are already on board.
Our dual M.2 for those caching drives I mentioned before,
it's even got too many SAS HD ports over here,
that gives us another eight drives.
So we only need a 16 drive host bus adapter,
in our case a LSI 9305-6i.
Check this out too, it's even got a couple of oculink ports.
So that is eight more lanes of PCI express down there.
This thing's awesome.
I think the only thing that could make it better,
is if it was PCI Express Gen 4,
which naturally ASRock Rack has,
it's just not something that we would get
a performance benefit from.
So we are using the Gen 3 one.
This is fun, while I was plugging power
into the motherboard,
I noticed that this power supply
has a USB connector coming out of it,
turns out that FSP has a little tool.
It only runs on Windows, unfortunately,
so we can't use it.
But they have a little tool, that allows you
to monitor temperatures, fan speeds,
all that kind of cool stuff.
Just off of a USB header.
For boot drives, we used a couple of Kingston 240 gig SSDs.
And you can use basically anything for this.
The one thing I'd recommend is having two of them,
so that in the event of a boot drive failure,
you have a live backup of it.
Ready to go, okay no back up, but copy there.
And I guess that's pretty much, it isn't it?
- [Jake] Yeah we just gotta boot up TrueNAS.
- Let's do it, 24 cores, 256 gigs of RAM,
10, 10 gigabit network ports,
24 drives, 700 Watts of redundant power,
and a mirrored 240 gig boot drive.
For one fifth of the price.
Now, you just got to boot this puppy up.
There's probably another cable...
- Oh look at another cable.
- We've already installed TrueNAS.
So we're just gonna boot right up to that.
If you guys want to know how to install it,
Patrick from Serve the Home, did a great guide
last time we collaborated.
So we're gonna link that in the video description.
Now that's a lot of network interfaces.
Look at that.
- 10. - 10, that's 10 gigabit,
10 gig... - We could put them
all together for a 100, maybe.
- Yeah, we could.
- 100 gigabytes. - Oh, yup,
Pretty connecting machine...
- That'd be way smarter.
If we just put a hundred gigabit NIC in here,
and then a switch and still had lots of money leftover,
but whatever we're direct attach.
Now that we're fired up,
we're in that beautiful little web UI.
Actually, this has improved a lot.
Since the last time I looked at it.
There's all those interfaces.
(chuckles)
- There's those power supplies going
(reeee )
- By far, the loudest fans in the system
are the ones in the power supplies
cause they're just these little tiny things.
But, if you're not into it,
just swap it out for something else.
Cause hey, you built it, you designed it.
You can do it the way you like.
So we've got two groups of 10,
and then like I showed you guys earlier,
we could lose up to four before we actually lose any data.
- [Instructor] And in the TrueNAS dashboard of course
we can see our two RAID-Z2 both with 10 drives.
And on the bottom, we have our log drive
as well as our cache drive.
So cache would be the level 2 Arc.
And then, log is the SLOG like we mentioned before.
- I like calling it a SLOG. - SLOG.
- It's got a SLOG through those index entries.
(chuckles)
- All right, - SLOG through them.
What we're gonna show you guys now,
because really the main point of this
isn't how to set up TrueNAS.
We wanna show you guys,
how we're gonna do the direct attach,
special magic, that Jellyfish ships.
So, we've got a test system right here on our test card.
That's got a 10 gig network card in it.
We're going to go ahead and the, Oh my God.
That's not screwed into anything.
- Did you just pull it out?
- Well, I didn't not pull it out.
- You didn't not pull it out.
What does that even mean?
- It means we're about to find out smart guy.
- Ah, it's probably fine.
It's hard swap, right?
You tried PCIe hot top, right?
Is that the right HDMI?
- I think so.
- Well, it was on before. - Oh crap.(beep)
Well, you didn't screw the thing in.
- Well it didn't have a thing.
And we're just trying to do it.
- The thing is right here you could just move it.
- I wasn't expecting you to just yoink on it.
- Well I didn't--
- He's a yoinker man, you just--
You gotta be careful around this.
While Linus tries to fix that,
we're on the TrueNas side, and basically
all you need to do to set up a direct attach setup.
Like the fancy Jellyfish software would do,
is essentially just assign an IP address
to both interfaces on either side.
And then the computer will facilitate transfer
between the two machines.
And what's known as an unrounded sub-net.
There's no router that's controlling the packets.
It's just the systems telling each other where
it needs to go.
So we're in TrueNas,
I've set up 10.96.0.10 for this computer.
And we got to make sure our submit is the same.
So dash 24.
And then on that side, we'll do 10.96.0.20.
And you can set these to realistically, whatever you want.
Click apply there, test changes.
Oh man, TrueNas has a super nice thing.
Basically when you're doing network changes on TrueNas,
it will like try them for 60 seconds.
And if you don't confirm it in the dashboard,
it will just revert back to the way it was.
You can't screw it up and like make it
so you have to boot it into recovery mode
or something to change the settings.
And then on our Windows computer side,
we have set up the same thing,
but 10.96.0.20 same sub-net 2552.525
which is the same as /24.
There's going to click, okay there
close, that will start to do its thing.
And then we'll just apply on this side
and we should be pretty much good to go.
- Funny story, that machine doesn't have premiere on it.
And it's actually faster for us
to just go get a longer internet cable,
and plug into one of our editing stations.
So we're just gonna redo the IP setup over there.
One moment, please.
After a quick reboot, to get the network share
remapped properly, you can see we've got our J drive,
which is our Jellyfish Fryer, and boom.
We are video editing off of our Jellyfish Fryer.
Of course, what we need to know now
is how's it handling it.
And for that naturally we turn to Pulseway.
Unfortunately, Pulseway does not yet have support
for free BSD based operating systems like TrueNAS,
but we use it for all kinds of server related things
here at the office, and actually even ones that
aren't necessarily server related.
With Pulseway, you can monitor all of your systems
through a simple dashboard
that's available right on your mobile device.
From here, you can see I've created a little sandbox account
with the machine behind me.
And one of our servers, I can monitor everything
from system resources like available memory,
or if we're running out of storage,
to patch policies, to even remoting into these machines
and taking control of them.
With Pulseway, you can also create custom scripts
that simplify your IT management.
And if Windows updates are a pain in your butt,
you can even manage those remotely
and schedule them as well.
And if you look around here in the editing then,
we've had to do a lot more remote management
than usual with a bunch of our staff working from home.
So try out Pulseway.
It's available for most operating systems,
including Windows, Linux, and Mac.
And it's so powerful that I'm always
learning about new functionality.
Apparently, you can use it to send a wake-on LAN ping,
like a magic packet, to one of the machines on your network.
If for whatever reason, it's stuck in sleep,
and you have no way of waking it up,
cause you're halfway around the world.
That's cool stuff, even for home users.
So it works great.
And how much did we pay overall,
before tax, including everything we've currently
got deployed, around 12,000 US dollars.
That's a lot of money, but it is nowhere near
the $58,000 that LumaForge charges for their equivalent.
And before you ask,
yes, we also accounted for the two cold spare drives
that aren't even installed.
That LumaForge does include with the Jellyfish.
We could even comfortably step up to IronWolf Pros,
which aren't enterprise per se,
but do have three years of data recovery included
if you're into that kind of peace of mind thing.
So, yeah, I can understand the appeal of a solution
that is all pre-set up for you.
And has some fancy software
for connecting to it or whatever.
But for the price difference,
you could quite literally,
hire an IT person for like half a year,
to spec this thing build it for you,
deploy it and show you how to use it.
I mean, you'd probably have some money leftover
to send your team on a nice vacation.
I see how you snuck that in there, Jake,
that's good, that's a good one.
You know, we can't do vacation this year.
As for us, this machine is destined
to become the new local backup server,
for our NVMe editing machine, AKA new, new wanik.
So, go check out that video,
if you haven't seen it yet.
And go check out Pulseway.
Pulseway is absolutely fantastic.
Whether you want to simplify your IT management tasks,
monitor your servers, remote desktop in,
apply patches Pulseway has got you covered.
And you can do everything from their mobile app.