NVIDIA has some of the best tools in the industry for data science.
You can use things like BlazingSQL and RAPIDS to process a large amount of data without having
to have a big Spark cluster. Just have a couple of high-end GPUs on your computer.
However, these technologies are very difficult to install. Typically that's where Docker images come
in. You can get a Docker image; it's an appliance you don't have to worry about configuring the CUDA
drivers, CUDNN, KUBlas, all kinds of stuff on top of that. You just get your docker image and go.
This is where NGC comes in. NVIDIA GPU cloud, it's a series of Docker images,
and you can choose to bring them down to your local computer and run them with your
own GPU. I'll do a video in the future on how to do that. However, in this video,
what we're focusing on is how to use NGC in AWS. I did a previous video giving you just a basic
crash course for data science in AWS. You may want to look at that video first if some of the topics
in this seem like I'm blowing through AWS quicker than would normally be there. So this is kind of
the video that introduces NGC. I'll show you how to get one of their containers up and running.
In this case, it is going to be the RAPIDS, one that allows you to process data. Then I'm going
to do a much deeper dive into RAPIDS in the next video, and we'll get this going as part of a whole
pipeline where you use S3, which is essentially like GDrive and CoLab. This is where your data
lives. You bring up the container to process it. Which is temporarily, and that goes away, and
then your results are back in S3. That's the next video coming up. So if you're interested in seeing
more of these data pipelines, definitely let me know in the comments, and if you're interested in
this sort of thing, definitely give me alike. That is how I know what videos to produce. I go down,
and I look, oh, that topic that got a lot of likes, let's double down on that one.
Okay, so let's go ahead and start NGC. I always find it by just searching the NVIDIA NGC catalog,
and it will take you right to it. These are all of the Docker images. I use TensorFlow, PyTorch
a lot. We're going to be working with RAPIDS, and there you see the various RAPIDS images.
I'm going to just go into RAPIDS here, and you see the instructions for this. This docker pull
would launch this on your local computer; we're not going to do it on our local computer.
I do want to keep this open because this gives me better examples really
than NVIDIA has on the Amazon site. But they work both on either side. So if you search for NVIDIA
NGC AWS. Now you're in the Amazon marketplace, and this is where we're going to really pick
which one we're going to use. They're the same images, so you can very easily transpose between
local and cloud, which is one of the real powers of this. So I'm going to search for rapids.
There it is; click on this one you want to use, these orange buttons. So I'm going to continue
to describe subscribe, yours will say subscribe, and that usually takes about an hour to actually
subscribe to it. There is no cost for it; you can see its dollar sign zero other than what you're
going to pay normally for these EC2 instances, so there's no markup. If you're doing something
like Oracle, there's a markup. If you're doing something like Teradata, there's a huge markup
because those are commercial software packages. So continue to subscribe and then continue to
configuration. I like to use the runtime for this example. Base has not as much installed,
so you might have to install additional things. Just for this demo, we're going to make it nice
and easy and use runtime, and we'll use the latest version. Continue to launch usage instructions;
there's some interesting stuff there. But you're going to want to click on this view container
image details. You'll need this in a moment when we come back here. I'm going to launch another AWS
page. I'm launching just the console, and we're going to hop into EC2. We'll launch an instance,
click the orange launch instance, go to AWS marketplace, and we're going to search for
NVIDIA. Like gaming PC, I've heard that these GPUs developed for deep learning can actually be used
for games too. I have to try that someday. This is the one we want, in all seriousness,
NVIDIA deep learning AMI; this is a very useful AMI even if you're not using NGC. It has a base
drivers, CUDA and Docker everything that you need to run NVIDIA Docker. So I'm going to select it.
Run it. These are the costs; we're going to use p3.2xlarge. I think that's a great one because it
uses V100, and there's just one of them, so you can start to try this out. We'll click continue
and that price. $3, we're gonna do spot because I'm a cheap, you know what. So that will get us
that for one-third of the cost, and this lets you just pick the hardware if you scroll down
to where it's checked. There it is. You can see it's selected. Don't go to review and launch;
we're going to go to instance details. This is where I can make it a spot instance. Notice
those are about a third of the price. You also have to pick the right one a there's no A, B, C;
this is defaulting to A. So we need to switch it to B, so that that lines up. Very critical to do
this. If you want additional information on this, I did do a previous video which I put a card in;
you should see it up there now; run through that I'll show you how to set up the IAM,
other things that. You need to actually authenticate this. But this is the only thing you
need on this page. So i can go straight to Review and Launch. Not eligible for the free tier, that's
fine. Click the launch button. Key pairs this is important. I talk about this more in my AWS intro
video, which I had the card on before, and there's also a link in the description. I acknowledge that
if I don't keep this key pair, I won't be able to access it. I'm going to use the test one that
I set up before, but you could easily create a new key pair, and it'll download a PEM file for
you. You've got to convert that PEM file to a PPK file so that Putty can make use of it. But let's
go ahead and launch the instance. Okay, this is now launching. Setting it up as a spot instance,
I'm going to now click on view instances, and we'll see this one starting up. So here are the
instances. I'm going to click the instance ID. We want this public IP address because that's
how we're going to interact with it. And i will launch Putty now. There's a number of
things we have to do here. I'm going to put the IP address in, and then I have to go over here to SSH
auth and browse. This is where we're going to put that PPK file that I generated and converted
using PuttyGen again, look at my previous video for a tutorial on that or just google
it. It's a pretty simple process. Then we're going to go to tunnels. This will allow us to
access the Jupiter notebook through a normal web browser; this is 4 8's this is localhost
4 8's. colon 4 8's. The source port is just what we're going to use on our side.
This is what we use on the Amazon site, and we're keeping the ports the same just for
simplicity. Always "add it," I swear I do that all the time. Log in as we're going to put in "ubuntu"
sometimes, this is ec2-user. The passkey I created when I generated the PPK file, again
see my previous video to see how to set that up if you've not worked with Putty much before. And now
we're at a prompt you can see how current NVIDIA keeps all of these. There are no warnings about
running "apt update" and stuff like that. Now the first thing you need to do is run "aws configure"
and you're going to need an IAM that has access to the EC2 registry. The ECR registry here,
we can see the user that I created for this. Again watch my previous video to see how to do
this. I basically just set up a user, and by the way, this is just a demo junk AWS account
that i create so you can actually see my AWS ID there. AWS ids they're useful if you want to maybe
social engineer attack me like you'll give me my own AWS account id and say, hey, I'm Amazon, but
really there's nothing you can do to exploit me on that, so I don't even block those out Amazon just
plasters them everywhere and even if I create a sub ID, it still has the same ID, so I don't even
try to protect those, especially on my tutorial account. You'll want to give that IAM user access
to the Amazon EC2 container registry full access. And when you create that user, it is going to give
you a file. So just to see some of the files that I do have here. Some of these are files that are
for me. I mean, you don't care about my CUDA drivers and stuff like that. But this is the
file that I downloaded from Amazon. The PEM file that is my private key to download into there,
and if you open these, it's just a private key. If you want to steal my private key, go ahead and
type every single one of those characters, I dare you, and in all honesty, this is the dummy account
that I will delete before this video even hits. It PPK is what you convert from PEM into PPK. This
is what Putty needs. This is what we downloaded from Amazon. This is a very secure file. Do not
show this to users on YouTube, like I'm about to do. But again, I'm going to delete this user. What
you're going to do is go into Putty, and you'll type "aws configure" and for that, I am that you
created, and if you notice my strategy here, I create lots of IAM users that have very limited
capabilities. So if one gets exploited, if you exploited this one, you have the ability to pull
from the Amazon ECR container images from me. I guess you could DDOS me and run up a big bill,
but you'd hit one of my alarms, and I would shut the whole account down. But we'll put that there,
and this is the really part definitely don't broadcast this to YouTube. That one there that's
my secret; these two don't matter, and I'm back at the prompt. Now I can go back to this page,
and I'm going to run these. What you see here is basically ECR login, and what you see here,
I believe that's an ARN. Oh no, that's actually that's actually a URL to my current EC2 instance,
and by the way, right here is, I believe, that account ID. Again I mean Amazon just plasters that
everywhere. I really don't even try to protect those and block them out at every point in the
video. So we need that first, then we'll need one of these back to Putty. We'll just run that
directly that logs me into ECR so that I have the rights to pull the NVIDIA NGC image for RAPIDS;
while that's taking a while over there, I'm going to go ahead and copy this, and again, there's my
account number being plastered for everybody to see. But we'll copy that, and we'll basically do
Docker pull. Now I have access to do it; this comes in very quickly. While it's doing that,
I am going to go over here; remember this page. I said it gives us nice examples of how to use it.
I want to look at this doctor run and copy it; I'm going to do this in a notebook. I use a notebook,
kind of like my notepad, like my operating table. Just lay it out there. I'm going to do Docker run
blah blah blah blah. But not this part. We're going to do this using the ECR instance from
this page. We need this, so we copy it again. That's the thing we just Docker pulled and
put it here. Now notice what's going on here. "GPUs all" we're going to use all one of them.
"rm" means remove the container. That's important; you may not want to do that if you don't want to
remove your container. If you want it to still be there because any files that you've worked on
will be in that container, so you've got an EC2 instance inside of there. Is Docker, so this is
kind of like Matrix in the Matrix, but in the next video, when we do the more formal version of this,
I'm going to use S3 to store all of this. Because normally, I want these containers
to be very transient. Throw them away. This is the 8888 that we tunneled. There are these two other
ports. If you need them, tunnel them. If you can, I think one of them is dash, and I forget what the
other is, but you can create as many of those tunnels as you need. I'm copying that
now. This is not quite done. That's a big file. I'm surprised even running inside of AWS. That's
taking a while. We'll fast forward, and by the way, with all of this running, this is
basically copying that Docker image into your ec2 instance. Once this is done, you do not need to
redo this unless you terminate your EC2 instance and lose that backing image that's attached to it.
So let's move this command that I created here. I won't actually put these commands
into the description for you to copy and paste because it has my account number, and that's okay;
these are semi-public; it's about like giving somebody your phone number actually not even that
bad you can't call me on that, but the point is it won't work for you so you'll need to generate your
own. Let me paste, and we're going to run this, and this actually runs the Docker image. It's up
and going in a moment. Fast forward, notice the prompt. Change it says RAPIDS. We are now inside
of the Docker image. So this is Docker running in EC2. This is the Matrix inside of the Matrix.
It'd be like in the old movie Matrix. If Neo decided to have built his own Matrix inside of
the big Matrix. All right, I'm not going to do much actually at the command prompt here because
Jupyter is already running in the background. So you can copy and paste this URL, and we're
going to run it here. And now I go to localhost, and this works much better. We're in Jupyter lab,
and DASK is running, actually not DASK RAPIDS; it's kind of the same thing, but I'll do another
video where we go more in-depth, and I show you how to actually do it some stuff with DASK.
Thank you for watching this video, and definitely subscribe so that you can see the next video where
we put this more into a production type use. How I typically run RAPIDS in the cloud where I can
spin it up, use more GPUs, and get the results in and out of S3. I'll show you how that all works.
Thank you for watching this video, and if it was helpful,
please give me a like and subscribe to the channel.