Hi there, my name's Lee Bushen, and I'm a Solutions Architect for NVIDIA.
And today I'd like to talk to you about some of the GPUs
that we've released in 2021 in the new Ampere range of GPUs.
Today, I'm going to be concentrating
specifically on this GPU here, which is the A16 GPU.
So what area of the market is this GPU aimed at?
Well, we've got a number of GPUs that we're releasing in 2021.
And this one is aimed fairly and squarely at the knowledge
worker / office worker type of workload.
Now, for some time, we've had another GPU in the market
that's been aimed at the knowledge worker workload,
and that is this guy here, the NVIDIA M10.
This is still a great GPU, has 32 gigabytes of memory on it.
So it's very good for high density VDI deployments.
We also had another GPU in between the M10
and the A16 that we have been recommending
for the last couple of years for knowledge worker workloads.
And that's this guy here, the NVIDIA T4.
This is still a great GPU.
We continue to produce this GPU and will do for some time to come.
It's got a really good niche in terms of what it's aimed at.
It's really good for the knowledge worker workloads as I said.
It's also very good for Edge deployment.
So Edge deployments, where you're doing things like
artificial intelligence, inference, that kind of thing.
And it's also good for maybe low end
professional visualization workloads as well, things such as maybe,
if you're using AutoCAD or some low end
applications like that that don't need too much graphical grunt.
So what are some of the things about the NVIDIA A16
that makes it so relevant for office worker workloads?
Well, the first one is that this isn't a single GPU.
Actually, this is actually four GPUs
all connected together using a Mellanox PCIe switch.
And what that's really good for is firstly, contention.
So I'll explain what I mean by that.
So the way that virtual GPU slices up
that GPU between multiple users is based on time.
So take this GPU, for instance.
This had got a single GPU chip on it.
And if you have 16 users on this GPU,
they each get a slice of this GPU.
They don't get a slice of a few of the cores each.
What they have is access to the whole GPU for their own time slice.
So effectively, the more users you put on this, the longer
the queue there is for processing on the GPU.
16 is fine on this GPU.
It's been tested and it works really well.
But anything more than 16 say when you start getting up to 24
users on the GPU, this can actually become a problem.
So, the great thing about having 4
GPU's is effectively you end up with four queues down to that GPU,
which means now we have four queues there is much less contention on this
card, which allows you to put more and more users onto here without there being,
conflicts between all of the users on here.
Another good thing about having four GPUs, is the fact
that you can run different profiles on each GPU.
So one of the things that you might know about our single GPU cards
is that once you put a profile on it, say you put a two gigabyte profile,
you can only host two gigabyte profiles on that card.
Until you clear that card down of virtual machines and maybe load a four gigabyte
profile VM onto it, in which case it can then handle four gigabyte profiles.
The great thing with this is because we've got four GPUs.
we've actually got four sets of memory as well.
So, GPU1, for instance, you could be running one gigabyte
knowledge work profiles, GPU2 you could be running two gigabyte
or four gigabyte GPU profiles for higher power users.
You could even be running different licenses.
So you could be running a RTX Virtual Workstation license on here
and maybe on another GPU on this card,
you could be running standard office workers.
So the choice is really yours and it gives you a lot more flexibility
in Day-To-Day, management of the GPU and your users.
Now, the next thing that makes the GPU really, really important for high density
office workers is the fact that this has 64 gigabytes of memory on this.
So in theory, it can host 64 users per GPU card.
If you compare that to the M10 before it, that has 32 gig of RAM.
So it's got twice as much RAM as the previous generation that we had.
The T4 has 16 GB of RAM.
So in theory, 2 x T4s equal 1 x M10
in terms of the number of density you can get.
This A16 though, you can get double the density on this GPU.
So if you put a couple of GPUs a couple of A16s into your server,
you're talking 128 users on that server all with 1GB profiles in that case.
Now you could potentially go up to three
GPUs, 3 x A16s in a host but then you start
going up against the limit of CPU contention on that server.
So probably a sweet spot for us that we see is going to be 2 x A16s in a host.
So it's worth mentioning that it's not always about
one gigabyte profiles and 64 users on this card.
As workloads become more and more heavy
and people are needing more and more screen real estate.
So more monitors, higher resolution monitors the need for
maybe a two gigabyte framebuffer or even maybe a four gigabyte framebuffer
per virtual machine has actually gone up over the last few years.
So bear in mind as well that you'll be able to put 32 users
with a two gigabyte profile on this card or, 16 users with a four gigabyte profile.
So the choice is yours.
Now, there are two things
that are really important for an office worker class GPU to have.
One of them is lots of memory, and the other one is lots of encoders
and decoders. And the A16 has four encoders on it.
Now, why are encoders important?
Well, the frames that come from the data center from those virtual
desktops need to be encoded in some fashion and then sent down
to the endpoint for decoding by the user's endpoint machine.
If you do that in CPU, that actually creates a load on the CPU.
So if you can offload that onto the GPU, you can actually make
latency a lot quicker and make the user's desktop experience much better.
So the four encoders that we have on the A16 here are the latest generation.
They're about three times as performant as the encoders that we had
on the M10 back here.
So they offer great performance for our end users.
So decoders are also really important to have on a GPU, especially for knowledge
workers, where they're doing lots of video based content, lots of video
conferencing, lots of CEO broadcasts, training videos, that sort of thing.
And the A16 has a massive eight decoders to decode traffic.
We decode all the standard stuff like H.264 and H.265.
We also support VP9, and the A16
actually supports AV1 as well decode, which is really important.
If you look at most YouTube videos today that you play,
if you right click on the video and say "stats for nerds".
You can actually see it was encoded using AV1.
So the ability to have that decoded on
the GPU itself is going to give much better performance for the end user.
Also, having lots of encoders and decoders on this GPU
means it opens up a whole new load of use cases for transcoding
for other applications other than knowledge worker workloads, too.
Now, another thing to
point out about this Ampere Generation GPU is that has the same Tensor Cores
and the same Ray Tracing Cores as the A40,
which you can see over my shoulder behind now, just fewer of them, so
it can actually open the door for maybe having some mixed workloads.
So during the day, you could do VDI.
During the night, you could potentially do some sort of A.I.
workload or maybe some rendering workloads as well.
So in terms of performance, the GPU hasn't been released yet.
So it's kind of fairly early days.
But we've done some rough benchmarking tests in between the M10 here
and the A16 using the VP2020 benchmark.
And we've actually found that the A16 is twice the performance of the M10,
something we'd probably expect in between those two generations.
However, just to point out that in terms of GPU, pure raw power,
the A16 isn't really designed as having, a massive amount of raw power.
It's not something that a lot of knowledge worker workloads will need.
What it is designed to have,
though, is the main attributes that knowledge worker workloads do need,
and that is a lot of memory and a lot of encoders and decoders.
So you couldn't really compare the A16 performance, for instance,
to something like an A10 or an A40.
So the next question is,
when can you get your hands on this A16 GPU in your server?
And that's the question that I'm being asked a lot by our customers.
We actually announced this earlier on in the year.
And the answer is, to cut a long story short,
probably towards the end of the year (2021 that is).
When we release a GPU, the first thing that needs to happen,
obviously, is, the GPU gets released into the market.
We then we have to support it with NVIDIA Virtual GPU.
That's something that's coming in the next release.
So that'll be there in fairly short order.
The next thing, of course, is that our hardware manufacturers
need to certify these GPUs in their chassis.
As you can tell from all of these GPUs that I've got in front of me here,
they're passively cooled.
So the cooling for these GPUs is actually done from the server chassis itself.
So those server chassis need to be certified to provide
enough power and enough cooling for these GPUs.
Which is why it takes a little while for servers
to be certified after a GPU is shipped.
So one thing to point out is that in between
the Turing and Ampere generations, we actually changed the way
the virtual GPU addresses and splits up that GPU.
We use a proprietary method in the Turing generation. But for Ampere,
we've actually moved to using SR-IOV technology to talk to the GPU.
What this means is that for the hypervisors that support
Virtual GPU, they need to also support NVIDIA GPU with SR-IOV.
In terms of VMware that actually started support
as of vSphere 7 Update 2.
So vSphere 7 Update 2 is what you're going to need if you're
going to deploy an Ampere generation GPU for Virtual GPU (on VMware).
So another thing to point out
is that the A16 is going to be supported as part of our EGX platform.
So this will be NVIDIA certified for EGX solutions, too.
So just to summarize, the A16, a great GPU for knowledge worker
workloads, it has all the attributes needed for those knowledge workers,
multiple GPUs for low contention and lots of memory.
64 gigabytes of RAM also it has lots of encoders and decoders
of the latest generation to get the best performance possible for your users.
Check out the NVIDIA website for further information and hopefully
see you back here soon, where I'm going to be talking about
the big brothers of the A16, which are the A10 and the A40!