Nvidia gave me access to play with their data center.
Not sure why they did that, but while I was in there, I found something,
something that I think might change the future of virtual machines. Oh my gosh,
that's so stinking fast. Did you see that? We have a CPU and we have a gpu,
but now we have a dpu. What is that? It turns out it's amazing.
It's called the data processing Unit. And let me tell you,
this thing is intense. I got a chance to play with it. Test it.
I'll show you later in this video. But first,
let's talk about why we need this thing , this DPU thing,
because it turns out we actually really did need it. By the way,
shout out to Envidia for sponsoring this video and allowing me to mess with our
servers. I didn't hack anything. Promise. Now,
first let's talk about servers and how they kind of changed a lot.
And if you're like, what's a server? Well,
it's kind of where the internet lives. Everything we do, every picture,
every website is gonna be on a server somewhere in a data center.
And back in the day, we kind of had too many servers. It was becoming a problem.
You see, every time we wanted to deploy a new website, a new app, a new thing,
we would have to deploy a new physical server filling up our racks,
putting out a space using tons of power. That was crazy inefficient.
This is where virtualization came in. You probably saw that coming, right?
And using something like VMware,
and we can take all of those servers and put them onto one server,
which sounds kind of crazy, right?
And each of these servers is now a virtual machine. Now,
this was actually a game changer.
We could reduce the number of physical servers we had cuz we only have one
physical server.
And each of those virtual machines would share the resources of that one
physical server. But that's where our problem begins.
And I wanna focus in on Mr. Cpu, our central processing unit,
doing the general stuff. You see his job got a lot harder.
He went from managing just one server, one operating system,
to now managing a bunch of operating systems, a bunch of servers.
He's being spread thin and used across multiple machines.
That's literally what we do. Like seriously,
some of our physical servers now have 30 to 40 to even more virtual machines
on them all sharing these servers resources. So yeah, Mr. CPU is getting taxed,
but we made him bigger, better,
and he could do more to where a lot of our CPUs can handle so many virtual
machine. It's crazy. So you're probably thinking, Chuck, where's the problem?
Here's the problem. We didn't stop at virtualizing physical servers. No, no, no,
no. We thought, hey, what else can we virtualize?
The networking you see in a data center, we're gonna have switches, routers,
firewalls, and all kinds of security appliances.
These once upon a time were their own physical boxes, just like servers.
And you probably see where this is going. We thought, hey,
we have all this stuff taking up space, why don't we virtualize it?
So that's what we did in came software to find networking,
all the cool networking stuff we could do on our physical boxes.
We then threw it onto virtual machines on a server. And funny enough,
the same people, VMware that helped us virtualize our physical servers,
they created nsx,
which you gotta be really careful with saying you gotta annunciate that.
So now NSX has helped us virtualize a lot of our networking stack and
security cybersecurity stuff to where they're no longer physical boxes handling
their own business,
but now they're sharing space and you guess to resources on one server along
with all the other machines and servers that we already virtualized.
So now at this point, Mr. Cpu, he's getting pretty stressed out.
He was okay with handling servers, let's virtualize those,
but now all the networking and security and stuff, back to stress mode,
he's done.
He went from a simple life of managing one OS on one server to now doing all of
this. And even though we made him really big and strong with tons of cores and
threads, he started to become the bottleneck slowing things down.
And one of the main reasons for that is that he wasn't really built for
networking and security. Like he can do it, but he's more for general purpose.
He's good at a lot of things,
but when it comes to specialty things like encrypting and decrypting traffic,
transferring lots of data and analyzing that data, he's like, he's okay.
And he really started to sweat when our network started getting faster,
going from one gigabit per second to 10 to 100 to 200, he couldn't keep up.
He threw up his little CPU hands and said, I can't do this. I'm gonna quit.
And we said, don't quit. We're gonna bring in some help, don't worry.
We brought in what's called a smart nick. And no,
I'm not talking about my video editor Nick,
even though he's the only smart nick, I need . That was for you buddy.
Now here is the smart nick. It is similar to a regular general nick,
which stands for a network interface card.
Every computer has one including yours.
And it typically allows you to connect the ethernet connection to it and get
internet servers have that. And in many cases,
the smart nick is designed purpose-built to handle things at the cpu.
MR CPU isn't really great at. So Mr.
CPU is greatly relieved because now he can give things like network traffic
inspection and cybersecurity, things like IDs and ips.
And it's kind of a beautiful situation, right? Like we just take this nick,
which pops into the server's P C I E slot,
and then takes some of the workload off the CPU and give it to the smart nick.
And in many cases, that's kind of our solution. Now,
I know some of you're probably wondering, where's the gpu? What's he doing?
Tell mcg, Miguel's lazy button help out. I wish he could.
He's kind of pretty boy. All he cares about is looks graphics and some AI stuff.
He wasn't designed to help with data traffic forwarding.
Now our journey doesn't end here because as you know,
technology keeps increasing . We want to add more servers,
do more network traffic. And now the smart nicks looking kind of dumb.
Nick. No, I'm just kidding. But seriously,
we're at the point now where the smart nick, while he's great,
isn't quite enough.
So sure he is helping out our cpu and it's kind of like we gave him an extra arm
a little bit, but he needs more than an extra arm. He needs an arm processor.
You'll see where I'm going with this. It'll make sense in a moment.
I just could not help but make that joke.
So here is where the D P U comes in the data processing
unit and where the CPU and the GPU both have their things.
They're good at the dpu, he's good at data processing as the name might imply.
Now the DPU has command to pretty much do the same job as a smart nick to
offload the stress from the cpu. But he's a bit different. He's not just an arm,
he's a whole body. He's a server being put inside a server. And seriously,
that's where the case is. Let me show this thing to you by the way.
This right here is NVIDIA's Bluefield three dpu. It's the latest and greatest.
Look how beefy this thing is.
They refer to it as an S O C or a system on a chip and the chip. You know,
it's right there.
Notice I kept the drawings for smart nick because that's essentially what this
apparatus is. In most cases, the Bluefield three DPU will be on a smart neck,
it'll be part of it. So they actually call it a dpu Uen enabled smart neck.
And again, it's doing a lot of what a typical smart nick might do,
except again, it's its own person, it's own. And I keep saying person,
that's kind of weird. It's its own endpoint device.
Like it actually is a server and side of a server.
And in the case of VMware, which I'm about to demo here in a bit,
this is actually kind of crazy.
You've got the server right running VMware ES XI with vSphere eight to VMware
eight, however you wanna call it. And their distributed services engine,
they're actually installing ES X I on this Bluefield dpu.
It has its own OS server inside a server.
So what that means is instead of just IT helping the CPU with certain tasks like
intrusion detection, network traffic inspection, all that kind of stuff,
instead of just helping, it's doing it like, you know what I mean?
Like we're installing the firewall software,
the networking software onto the DPU or the CPU is just like he just took the
job. I don't even know what he's doing. I mean he knows, but it's separate.
The d p is handling the load network traffic isn't coming to the CPU U,
it's coming to the dpu. He's handling everything.
Now let me show you the specs on this thing so you can understand the scope
here. Pick this up. 400 kickass per second of can win.
What? Try to do that cpu try. You can't.
This thing has more RAM than your gaming pc. It just does. Sorry.
And here's where my arm joke came in. , it's got a arm CPU.
Now ARM is fantastic because it's power efficient and it's built for high
performance processing stuff like data. I mean with a name like Hercules,
how can you not be powerful? And speaking of power,
one of the main powerful things it does or allows for is programmability
doca,
which I believe is the NVIDIA SDK for their dpu allows extreme
customization and programmability for the dpu.
And you may have noticed here on the right, all the things that it can do,
it can handle, which I forgot to mention, we also,
decided to virtualize our storage software defined storage.
So the CPU was already just gonna die.
So we had to just throw the DPU in there and look at all this networking stuff.
It's insane to me,
especially as a networking engineer who worked with physical hardware to see all
of this be virtualized and accelerated with a dpu VXLAN
accelerated in inside a DPU layer four firewall connection tracking.
So that really leaves me with one question,
when am I gonna get one inside my computer? ? Seriously,
could you imagine having a DPU on your computer that allows for that kind of
power? I don't need it, but I want it. But I'll also,
for starting with one of my servers one day now,
there's a lot more DPU than how powerful they are and what they can do.
Awesome white papers below.
But now I wanna show you hands-on what it kind of looks like performance-wise.
And actually, I forgot, mention this. You may still be wondering why dpu. Sure,
we can add a DPU to the server,
but wouldn't it make more sense just to have another server and another one and
another one just so the CPU doesn't get stressed out, right?
We'll have just a bunch of cpu. Well, we're going back to the original problem,
aren't we? We're filling up our data centers, we're running outta space,
we're using too much power and we're inefficient.
A dpu on the other hand with this arm processor, is much more power efficient.
Instead of expanding across multiple servers,
add some more horsepower to your one server that's specialized to handle the
traffic you're trying to optimize.
And by offloading that from the rest of your resources,
you're seeing some crazy stuff, like 30% efficiency increases.
Let me show you. So I got the example right?
Nvidia gave me access to their data center and their VMware V center
environment. And yes, with VMware vSphere eight, you can use dpu.
You can assign them to your virtual machines to be used. Now,
as part of my lab and plan with this, I had to set up some things.
This included virtual machines and even configuring some switches and stuff in
nsx, which okay, NSX ,
which includes their, you know,
their virtualization of networking and firewall security stuff.
This was super fun. Now,
the first thing I got to do in the lab was test something that wasn't possible
before. You see, traditionally when using a smart nick with VMware,
which you could do it involved doing what's called P C I E pass through,
which works for a lot of things except when you want to do things like vMotion,
which vMotion is so cool.
It allows you to move a virtual machine from one physical host to another.
So like one data center to another data center. I've used this a million times,
but with P C I E pass through, that breaks . When you try to v motion,
you lose connection to that smart nick. It just doesn't work out kind of a pain.
But with VMware eight, vSphere eight and their distributed services engine,
which was previously called Project Monterey,
they now have what's called U P T or Universal Pass through.
Actually U P T stands for Uniform pass.
Through. And you can use this with Nvidia dpu, which means,
and you're seeing this right now as I'm doing this,
I can assign the DPU as a nick to a virtual machine. Again, by the way,
the DPU is nestled safely in the P C I E slot of the server,
but it's also running its own os, which is kind of crazy. E S X I.
So that server has the DPU as its Nick and I can v motion that server over to
another host and it just works. I don't have to worry about it,
which is kind of killer. The VMware engineers eat your heart out.
It's a cool feature. But now let's test the speed of dpu.
Now Envidia set up a pretty cool test.
We're transferring data between Redisk servers, which redisk is a database.
Essentially they gave me a test and a script to like transfer data over,
first of all, example one, a regular Nick,
a nick that was using the CPU for its networking,
the CPU is having to deal with that stuff.
And then we switch over to using the dpu,
the machines that had the DPU installed. So here are the test right now. First,
let's run the standard nick test and see what the performance is.
So not too bad. Bandwidth was 64 megabits per second.
Latency 0.2 7, 2 3, 1 milliseconds,
548,000 operations per second. By.
The way, this entire lab was done with a 25 gigabit ethernet cluster.
If you wanna know more details about this setup,
we've got a link to NVIDIA's white paper below.
Now let's test the dpu.
Oh my gosh, that's so stinking fast.
Did you see that 90 megabits per second latency 0.19,
770,000 operations per second. Do the math. I'm not gonna do it,
but that's just really fast. Right now, what's even crazier about this test is,
you know, the DPU obviously performed way better,
but at the same time the CPU is still there. Dylan ,
like he wasn't stressed out.
So he's over there just being a powerhouse for what he's good at and the DPU is
doing what he's good at, everything's great. Whereas, you know,
just the standard Nick test, the CPU is stressed out. He's like, ah.
So keep that in mind. When you compare these numbers, it's not only better,
it's more efficient.
Cuz the CPU's still doing his thing with all the other servers and he's not
stressed out. We're using less power.
And this right here is the future of virtual machines.
It's how we're designing our data centers. Dpu are the future.
As we do more advanced things like, I don't know, ai,
we'll have specialized dpu. I can handle things like this. Now,
as Abby mentioned earlier,
this was just done on a 25 gigabit per second cluster.
So you might be wondering, Chuck, you said 400 gigabits per second.
Where is that? That's why you don't see it here.
This was one test with one application with one workload.
And the point was to demo what DPU can do for a workload,
which this was kind of crazy. Now,
if you do wanna understand the true scale of maxing this out,
Nvidia has a white paper below. Check that out. It's a fun read. Grandma.
Go subscribe to network. Chuck.
What are you doing?