what's going on everybody and welcome to
a video about kind of an overview of the
current high end cloud GPU providers as
well as a bit of a tutorial for how to
actually make efficient use of these
servers so you're not wasting time and
money doing the pre-processing or even
data set download or upload to the
server which can take a very very long
time and just waste like hundreds of
dollars so with that we've got a lot of
information to cover let's get into it
so the first thing that we'd want to do
is just kind of quickly as quickly as
possible compare all these providers
there's so many numbers that you got to
think about and look at it can get super
confusing hopefully I can make it really
easy for you first of all the every
provider has released most providers
have a cluster of GPUs or
different GPUs that they offer but the
best the the flagship GPU that pretty
much everyone is offering is the v100 so
the tesla v100 from nvidia this GPU has
16 gigabytes of vram per GPU there is a
32 gigabyte variant to which you can see
an example here
and I think AWS is the only one offering
it I'm not positive well I'll take I'll
try to remember to look as we go through
the prices but you'd have to be spending
a lot of money this is a $30 an hour
machine so I don't think many of you
guys are looking for that but it is I
think also AWS has the most GPU vram
person singular machine that you could
have so this is before you might think
about doing distributed tensorflow or pi
torch or whatever which is always a huge
mess so if you want the largest models
possible AWS is still going to be the
victor but in terms of v1 hundreds per V
100 GPU three dollars and six cents you
can sign up for one year in three year
reserve prices which lowers it but you'd
be a fool to sign up for a GPU for three
years or even probably one year so I'm
not really gonna look at those prices
moving on so on - azure also V 100 GPUs
one thing start you know paying a little
bit of attention to is is CPU and RAM
for a lot of deep learning tasks this
might be irrelevant to you like you
might not actually need very much at all
like maybe the only thing you're
like IO or something like that from like
a data set but if you're doing something
like reinforcement learning the more
vram the better and often also with
reinforcement learning depending on what
sort of environment you're using the
more CPU the better but might not also
always be the case that more CPU the
better so so that is going to be a per
use case scenario that I can't possibly
dive in for you guys but basically Azure
is exactly the same price but they do
have different very variable very
varying amounts of RAM storage and how
many cores of CPU mm they all vary so
pay attention if any of those things
start to matter to you
pay attention moving along a Google
cloud also has the v100 also has well
actually they're cheaper I first
emersion in my head I thought there were
three dollars as well
2:48 I wonder if they were always 248
anyways 248 per GPU so actually cheaper
than both AWS and Azure and they have up
to 128 gigabytes let me see what they
had only four times so less Wow not very
many so actually Azure is kind of losing
in terms of how much possible although I
wonder if this this one's a little more
expensive
so what also we same amount of RAM same
cores I wonder if this is your 32
gigabyte variant v100 I'm not sure
anyway moving on to paper space my
longtime favorite cloud GPU provider
simply because you spin it up it's ready
to go it's just super simple you've got
a virtual desktop already it's just
super easy right to get going on paper
space there v-102 30 an hour so even
this is the cheapest so far another
reason why I really enjoyed going with
them
also they offer other GPUs like the
p6000 which has 24 gigabytes of GPU
memory which is fun to play with so then
moving along oh the one thing that's
kind of a downside for paper space
though is you can only have one GPU of
her machine so you're maxing out it
either 16 gigabyte
of VRAM here or 24 on the p6000 but you
can't come you can't have multiple GPUs
per machine so that's kind of a bummer
moving on to Linode which just started
their GPU plans they have rather than
the V 100 the RT X 6000 which is kind of
an interesting move so I first I had to
take some time to look in what the
difference is so the V 100 GPU obviously
16 gigabytes of vram the RT X 6000 is 24
gigabytes of vram so quite a bit more
vram also the other thing that matters
is how quickly can we process tensors
basically or arrays so the V 100 has a
hundred and twelve tensor T flops the RT
X 6000 has 130 tensor T flops so more
operations per second more memory for
half the price
at least of AWS and Azure it's a little
closer to paper space but again you can
have up to four of these RT x six
thousands so that's a pretty amazing
offering to be honest with you I don't
know if they're even that you got to be
operating at a loss at this stage but
but anyway super cool so this is who
we're gonna end up using I do have
referral link for Linode sometimes
people get a little fidgety when I have
referral links honestly I have a
relationship with Amazon a sure Google
paper space and Linode all of them they
all want the airtime it just so happens
that right now Linode is offering the
most absurd deal possible so we're gonna
go with Linode so that's what I'm gonna
use but if you have a different provider
you're watching this later and someone
else has better offer you can you can
you can use the same methodology that
we're going to use on any of these
providers so so hopefully it will still
be of use even after maybe Linode is not
offering the best deal possible so first
of all you'll go ahead and need to
create an account login again you can
use its Lindo com if you got a little
comm slash syntax
you should get here and you can just
sign up for an account there you'll get
a twenty dollar credit but I think you
actually have to spend twenty dollars
and then you'll get your credit so so
keep that in mind but you'll still at
some point get a twenty dollar credit so
uh once you do that let me go ahead and
login to my account you should look at
something like this when you get to your
dashboard so how do we how do we want to
efficiently do this so your GPU servers
going to be many dollars an hour as
opposed to like if we were to look at
the other types of servers we're talking
like pennies per hour so so the first
thing we want to do is we want to have
like a simple a simple virtual private
server that is going to serve as our
sort of house of data so at least on
Linode if you go to create we're gonna
create a limo this is their name for a
VPS so we'll go there we want a boon to
for this we can use 1904 but hopefully I
don't forget but we want to use 1804 for
at least our GPU server and in fact I'll
just I'll just get in the habit so
eighteen where is 1804 here it is and we
want to do that for the long term
support makes updating later down the
road much easier also it has the long
term support so 1804 a boom to pick a
region in my case I'm gonna go with
Dallas Texas I haven't checked all the
regions but some actually not Dallas
Texas because Texas is a problem Newark
New Jersey so with all of these
providers they don't offer GPUs in the
same region like like they have more
regions of like CPU RAM storage than
they do have like GPUs at those regions
so some of these places have like 40
locations around the world that you
could choose from but not for GPUs and
so in Leonard's case I happen to know
they have GPU instances in Newark New
Jersey so I'm gonna put everything in
Newark New Jersey so what we're building
right now is our like where we're going
to store our data so you could either
have a data storage VPS you could have a
data pre-processing
VPS and so on I wouldn't recommend it at
least on Linode the thing that makes the
most sense would be to do your storage
and pre-processing probably on the same
machine but we'll talk about that
maybe later if I remember but for now
I'm just gonna go with a standard Linode
this two gigabyte you could even go with
an anode for even cheaper but I I kind
of don't like one gigabyte of memory
because sometimes that's not even enough
to install certain packages so I'm gonna
go with this one and then we'll come
down here I'm gonna call this data
server and then we're gonna make a
password
jellyfish it's a great password and you
might want to think about backups I've
never regretted having backups but this
is just a quick machine that I'm just
using for an example so I think we're
ready so we're gonna go ahead and spin
this one up so again if you're following
along I would strongly recommend you you
probably use Newark New Jersey and then
later you can look for regions that are
maybe closer to you although it
shouldn't really matter where it is for
for cloud GPU stuff I don't think it
matters unless you're uploading the data
set from your local machine maybe but
either way it's not gonna cost you very
much money in this case because this is
a $10 a month server as opposed to you
know a dollar 50 is a dollar 50 times 24
so you know you're paying more than that
per day more than double that per day
anyway uh let's go ahead and create did
I forget something do I need a punk
how about capital jelly jelly fish
exclamation mark what do you think about
that it's not a good password if I
forget to destroy this machine so we're
I me because you all know now know the
password to my lovely IP address we're
right here
okay so we're gonna wait for this to set
up so basically what this is going to do
is we're going to have a volume that
we're going to attach to the server and
again this is a this is a concept that
exists definitely on AWS definitely on
Google cloud I haven't used Asher to any
serious degree but I'm willing to bet
they do it to basically all these
providers have this sort of structure
where you've got your you know yes you
could have a VPS but you can also spin
up totally separate storage container
type things because often you need more
storage than you need on that is
available on some of these like just
kind of prepackaged Virtual Private
servers so uh did I get lost okay so
this looks like it's good to go
here's your IP address we can copy that
and I actually don't think we need to
login just yet the other thing the next
thing that we want to do is we want to
have storage like in this case I'm
actually not going to need more than 50
gigabytes of storage if you don't need
more than 50 gigabytes of storage then
you don't have to do the next step right
and you could also spin up like as you
go bigger on your linens you'll get more
storage in the Linode also your GPU
machine probably has a pretty hefty
amount of storage but that machine we
just want to spin up and use it as
quickly as possible so we don't want
we're trying to avoid the download or
upload of a data set to that machine so
that machine has to be on and you're
being billed so long as those machines
exist whether it's on AWS Azure Google
if those machines exist you're getting
billed for them so even if you turn them
off that GPU is still being dedicated to
you and you're still paying a dollar
fifty an hour or three dollars an hour
or thirty dollars an hour so you want to
use it as quickly as possible
so so we've got our data server I
believe it's probably online now what
we're gonna do is we're gonna go to
volumes and we're gonna add a volume I'm
gonna call this ml storage size 20 is
fine again this is this is just an
example but you know you'd probably want
like a terabyte or like 10 terabytes or
something how freakin big did I go Oh a
thousand gigabytes yeah
but you probably want a 10 Tara but it's
that's a lot of money per month anyway
you know you might want 500 gigabytes or
something reasonable anyway
you also could in theory have your
storage on some other provider but part
of what we're trying to do is we want to
have our volume and our data source we
want to have that in the exact same
region so when we go to transfer that
data it has the fastest transfer rate
possible so hopefully you can upload and
download data at a hundred plus
megabytes per second if if it's like
from your local machine you might only
get like 5 or 15 or something terrible
and it's going to take forever so we're
trying to get this to be as quick as
possible and at least here there's an
even better method that I'll show you
guys near the end but we'll go with a
hundred region again we're gonna go with
Newark New Jersey we can just
automatically select a Linode you can
change this later but I'm gonna say data
server because that makes sense I don't
really care about tag I don't even think
you have to do it so let's go ahead and
hit submit there and we should get our
volume so now what we want to do is we
need to run these commands and I guess
I'll just make another window here
windowed calm go login what I want is so
there's our volume click on the server
I'm trying to get our IP so here's our
IP address Boop so now coming over to
either if you're on Mac or Linux just
open your terminal if you're on Windows
either download a program called putty
or I'm using bash on a boon - on Windows
you can go to like the App Store I think
and get it or you just enable it I don't
even know might even be there by default
now I honestly do not know I just know I
have it and it makes it easier to do
stuff like this so again use whatever
there's a million options for Windows
basically but if you're on Mac or Linux
just open terminal if you're on Windows
and you're having a problem with this
step feel free to either post comment
below or join us on discord that's
discord GG / centex we can definitely
help you out or go to the text-based
version on this tutorial there's
instructions there so we're gonna SSH
root at can't paint there we go right
click to paste it in that address
and then the first time you connect
you're going to get a message like this
basically it's just like hey we've never
seen this fingerprint before so if this
is the first time you're connecting to
that server then this is totally fine if
you don't think this is the first time
you've connected to that IP address
something is wrong this would be a red
flag but for now yes is correct
cool now the password capital jellyfish
exclamation mark see if we can get in
that any type of that oh my goodness if
I don't get in maybe ok just took a
second like no okay so now what we want
to do is we want to run these at least
these three commands and then we could
also add this as well this is a little
less essential
so first the this command here actually
makes the file system on the actual
volume itself because we've attached
this volume to this blue node let's call
it so first we want to make a file
system on that volume so it's almost
like it's its own drive right now right
and so first we're gonna add that so I'm
gonna come over here and just right
click Boop done and if I forget to okay
if I forget to say it later we're gonna
I'm gonna show you guys two methods for
kind of moving your data around and this
is a command you would only want to run
one time per volume just remember that
okay so and then this one make dirt so
now that we've got the storage or the
filesystem on that volume here we're
just gonna make a directory for how we
where we want to access that storage on
our actual machine here so I'm gonna
just copy paste that in makes that
directory and then now we want to
basically mount that volume to that
directory so now copy paste again so
it's just like pointing this location to
this location so now our data server has
access to all that 20 gigabytes of
storage
or 500 or 10,000 or whatever so cool so
now that we have that done we also could
you could just nano FS tab here here so
so nano FS are actually was et CFS tab
or fstab whatever you want to call it
come down here paste that in confirm
that cool yeah all set control X yes to
save
enter done so now if you reboot this
machine or whatever this will always be
there for you so so now we have the
storage and we can get there by just
doing change directory into mount ml
storage in there we are now move this
aside and in fact hmm I don't want to do
this go the next thing we want is the
okay so I could google it I bet I could
find it that way we're gonna we just
need a data set just to kind of show you
guys how it works
cats verse dogs might grow soft but I'm
sure I can find it
yes okay so click on that and then
here's your download I think this will
take us to a new page and actually what
I want to do is right-click this copy
link address and then I'm going to come
over here and inside of mount ml storage
what we're gonna run is you can't see
that here we're gonna say W yet oh
really
I'm pretty sure I copied that let me try
that one more time
copy link address come over here OOP
cool so that will download again to our
data server
so hopefully here you can see yes even
here we're getting about I don't know 30
40 megabytes a second it make it bits is
it anyway so feel free to bring
so now once that's done we basically
have this data set on our our server now
the next thing we could do is we can
either we could keep it in zip form or
we could like unzip it and so I think
I'm actually just gonna unzip it
I always forget the tar xfz the command
to do that so instead we're gonna say is
sudo apt-get install unzip wait for that
any day now I guess while we wait on
that shuts out to the following channel
members let me Boop that over cool
one-year Theano Lima Rodrigo Silva Abbe
Ajit thank you guys for one year of
membership it's a very long time
omni crux Miguel Latour Oh pirata and
how yan guo six months of membership you
guys are amazing
okay we've got our unzip so now we want
to hopefully over here
unzip kaggle cats and dogs very good uh
wow this could take a while okay well
while we wait on that I'm trying to
decide so the next thing that we're
gonna do is once we have this this is in
theory our data set like we're pretty
much done we're just gonna wait I wish I
could remember if this started on cats
or started on dogs I guess we'll find
out I think it's about 12,000 images per
thing oh we're done okay let me make
sure that that worked as intended
I cannot read that this is pet images
and then I have no idea what's supposed
to be here I've picked up a poor color
scheme so okay so we've got basically
this directory that is it looks it's pet
images like that
that's what contains basically all of
our training data so now
see all this time that we spent luckily
this didn't take too long this isn't
actually a very large data set right I
think it's like 400 megabytes or
something the typical data set that
you're gonna actually do deep learning
on though is often like 50 gigabytes or
500 gigabytes right it takes a very long
time to get that data to your server
either through downloading or even worse
if you have to upload that data almost
everyone has a pretty at least in the
United States other parts of the world
are blessed and then other parts of the
world aren't blessed they don't have any
internet so so depending on where you
live this might vary for you but in
general I find my upload rate is
terrible and it really is very painful
as I upload to a server that I'm paying
many dollars an hour for so anyway we
have all our of our data and it's on the
data VPS so now what we want to do is we
can actually spin up our GPU server so
I'll come over here okay we're going to
actually go to stack scripts and then
community stack scripts and then
probably if you just type send text
you'll find it maybe yes so what we want
is the syntax tensorflow GPU and pi
torch setup now like I said well first
let me do this okay first of all click
on it like I said you can do this on any
of the providers you don't have to use
Linode but you should because right now
they're the best but this basically it's
it's called a stack script on Linode and
stack scripts on Leno tor basically
shell scripts with a little bit of UI
and variables involved or okay with and
that's it and this stack script is truly
a shell script that you could run
there's no like ten or there's no Linode
variables being passed here so if you go
to the text-based version of the
tutorial you you literally can just copy
and paste this it's a shell script and
then whenever you get onto your server
you would first just chmod plus x the
name of the script and then you would
just run it with period slash and then
you know script SH okay simple again if
you're having trouble with that you
don't really understand any of that come
to discord GG / syntax we be happy
to help you out so so many ways coming
back over here it's just convenient on
Linode that we can just save this and
it's kind of like like on paper space
you can just spin up the machine it's
ready to go
because again setting up tensorflow can
take a while because you got to get qu
DNN CUDA toolkit you've got to get tents
flow GPU you've got to get every all the
other dependencies you're gonna need to
get I think Python 3 6 is now the
default on Lynda or on a boon to but it
probably should be on 3 7 at least
that's what I would prefer to be on so
you know getting that all set up getting
pip getting just there's so many things
you gotta get so you just wasted a bunch
of time and it's not only the server
time it's your time as well so anyways
this will just automatically do all of
it for you so I'm gonna say deploy new
Linode from this stack script you also
could have clicked those like little
three dots but this is what I'm going to
do it's already selected a boon to 1804
for us again region do Newark New Jersey
if you do another region you might see
something so let me just say well I'll
just say Newark New Jersey for now and
I'll try to remember to switch it but
then come down to Linode plan choose GPU
I'm gonna go with the the thousand
dollars a month option but you could use
the other ones if you if you have
something that needs that much a little
label I'm going to say our TX 6000
password jelly-fish exclamation mark
please don't forget about that uh coming
over here what else do we need I think
we're good to go so the only other thing
I want to show you guys is if I was to
select Dallas Texas which I'd have tried
before and I click create if this works
I'm gonna be mad but I don't think it
will yes you'll get this this little
warning here it says you are not
authorized to take this action what they
mean to say is we don't have any GPUs
there so anyway coming back to Newark
New Jersey they do have GPUs there
although if enough people follow this
tutorial on release they might not
so careful anyway
Norton New Jersey cool I think we're
good to go again you could do backups
but $40 a month is pretty expensive and
our methodology will kind of have almost
backups fully created you would just
need to do a couple of basic things and
you'd have your own backups so yeah
anyway I'm not gonna do that great hope
that's good the Lindo GPUs are build I
want to say prorated to the hour so some
places are prorated to the second which
is like AWS will prorate your bill so if
you use 30 seconds on Linode you will be
billed for an hour so a dollar fifty
let's say whereas on Amazon if you use
thirty seconds you would be billed for
whatever a dollar fifty or actually
three dollars and six cents and 30
seconds of three dollars and six cents
an hour you do the math okay so
currently our server is spinning up it's
busy it's got a it's there's actually
quite a few things that it needs to do
because first it has to spin up get
allocated everything and then it
actually has to run the stack scripts so
even though we don't have to run this
that anymore it still does have to run
and actually set things up so things are
getting downloaded and stuff like that
so I actually don't remember exactly how
long this will take so probably what
I'll do is pause the video for now and
then see it does say booting but it will
first boot then run the stack script and
then reboot so it could take it could
take a little bit so we'll see
let me think you're we could connect to
it has been created booted I just know
it can't possibly be ready it'll have to
reboot after the stack script runs
anyway I'm gonna pause for now and then
pick back up once I'm confident the
machine is ready so I'll just test to
make sure tensorflow is is up and all
that okay and we are back
it has finally rebooted I just I knew
that it would reboot at the end of that
stacks grip so I was just kind of
waiting for the reboot it took ten
minutes
honestly to set up all that stuff myself
I usually take about 30 to 45 minutes
because you have to download like coding
in the CUDA toolkit drivers all the
other things and then I usually end up
forgetting certain dependencies when I
go to run my scripts for the first time
it takes me a really long time so 10
minutes is really good you just start it
you walk away it's all good
so anyways yeah so you'll have to wait
10 minutes if you're following along
otherwise you're good to go so now here
is a fresh batch on a boon to bash on I
don't know anyways go yordle it is what
I want is my IP address for the GPU
server I want make sure everything
worked so SSH routes at that IP I've
already connected once but otherwise you
you'll see that same warning as before
it's it's your first time connecting
password what is le fish jellyfish don't
forget to nuke okay so the first thing
I'm gonna do is check python 3.7 that
does indeed open python for me let's
make sure we're on view for everybody
and import tensorflow as TF despite
future warnings it does work and you can
ignore these these are just you have
numpy is involved with tensorflow as you
might expect and there are certain
things that are going to have to change
before the next update to numpy which
will eventually deprecated certain
things that tensorflow is using of numpy
but right now it's totally fine and
don't worry it will be fine in the
future as of python 3.7 various
deprecation and future warnings are much
more visible than they ever were before
which i kind I don't know how I feel
about that but anyway that's why you
probably if you've been using 37 you're
seeing more of these deprecation
warnings than you've ever seen before
anyway looks like it works now we want
to test it but we first we need some
data also our data is not on the server
yet right so the
first thing that we're gonna do is uh I
guess I'll just make derp and then
forward slash deep-learning cools and
then we'll change directory into deep
learning cool and now we want our data
so I'm going to use secure copy and
paste to move our data set from here to
our machine now this is one of two
methods that you can use I'm going to
use this method first because this
method will always work and then I'll
show you the method that I would use
anywhere you have this moveable volume
like we have right now on Linode because
it'll make way more sense but for now
we're currently in this on the data
server which has currently the volume is
mounted to it we have our zip file which
extracted two pet images switches I'm
sorry about the colors but anyways pet
images is there and now what we want to
do is where I've already meted this now
what we want to do is move pet images to
our GPU server and the way that we're
gonna do that is with SCP so let me pull
up yeah so to do SCP you'll start with
SCP and then what do we want to move we
want to move pet images but because pet
if it was just kaggle cats and dogs you
would just say SCP this okay really
oh it just took us at yeah hit tab you
can hit tab and in autocompletes but it
was taking awhile anyway so SAV so you
could if it's just a single like zipped
up file you can do that but I actually
want to move this directory that we
extracted to save time so I'm actually
just say SCP - R for recursive pet
images and now we're going to SCP that
entire directory and then where do we
want to SCP it - we want to SCP it to
the root at then the IP address for our
can I get it with this possibly is it
here yes right here for our GPU server
so copy come over here root at
45 sending that okay cool at that
address : and then the location on that
server so the location was like deep
learning or something may break
so just slash deep learning yes PWD is
print working directory but also set it
right here anyway so just slash deep
learning slash deep learning cool I
don't think you would need that trailing
slash but I'm gonna throw it in there
that's a beat
Arpit cool I'll hit enter hopefully this
goes again this is the first time that
the data server has connected to the GPU
server so again we get the same little
fingerprint warning thing yes password D
jellyfish and now we wait for this all
these things to transfer this operation
it would be it would have been it's
questionable which would have been
quicker actually to do and I also wonder
if SCP has a distributed option because
that would also be much quicker this is
going to take quite a while actually
this is going this would not be the most
efficient it would be more efficient to
move the zip file which I think took 30
or 40 seconds to download we'll move
that one at the end of this as well
because I think that'll be kind of a
curious thing it's still going
the extraction was definitely quicker so
here's what I'm gonna do I'm gonna just
break that just ctrl C to break it I'm
gonna come over to our GPU server I'm
gonna do LS there's pet images I'm an RM
- R for recursive pet images
see you later freak and instead what
we're gonna do is rather than SCP
recursively which will be useful if
you're training like especially if
you're trying to move the information
from your GPU server like you want to
move model files log files other things
back you still want to know the the SCP
recursive but that is taking way too
long
so instead what we want to do is let's
do SCP and in fact let me just do kind
of just do this and then rather than pet
images let's SCP and then we'll just
move cago cats and dogs zip to the same
location
now this should go quite quickly
hopefully even now it's a 30 megabytes
per second it's terrible I wonder if I
didn't do I'm pretty sure both of these
are in the exact this should be way
quicker it should be way quicker
both of these are in Newark New Jersey
or volume it gets attached but it's also
in Newark New Jersey these are not
typical numbers that you would see it's
going so slow I can't decide if it's
going so slow because of our previous
Trant like that might be like a firewall
issue like we should be getting a
hundred at least I'm curious if anyone
else is following along post below what
number you're seeing especially if you
don't do this initial this initial thing
but this it's got to be a firewall or
something I've done this quite a few
times and usually when you're
transferring locally like that because
it's it's from in the same region so
they should all be on the same network
so we should be getting somewhere
between 100 and 300 like megabits a
second like this should not take so long
so it's this is kind of a bummer
make it bits or make a bite someone
correct me there ain't that's too bad
that took a while that really did
luckily I have an even better solution
for you
but first let's finish up the SCP
version because even I can't I still
can't upload data at a solid 12 megabits
a second or megabytes a second so many
people are gonna be like it's this idiot
okay so now this is our data server
see you later freak now we have the zip
file which again we're gonna have sudo
apt-get install unzip and then we'll
unzip it much quicker than that now
though I do wonder us to unzip Kaggle
I do wonder if I just wonder if we were
throttled kind of before we started
moving that data or if there's some sort
of network issue right now what cuz
that's just it that's a terrible local
transfer rate anyway that's already done
so cool so now we have that directory
the next thing I want us to do is I will
post a link in the description basically
this is the text-based version of this
tutorial if we scroll down to here we
get a just a quick example script that
will run it's just gonna train this
model basically so coming over to my
bash on a bun - on Windows on Linux on
Mac OS oh dear I didn't want to do that
we want to nano ml example dot PI and
then right-click in there and this is
just our quick script I don't really
think I need to explain it but basically
it's just going to do some of the
pre-processing again this is something
you might want to actually do on like a
pre-processing server rather than on
your GPU server that you're paying a
bunch of money for but it does just so
happen to be the case that your GPU
server has a lot of processing power so
I'll let you make that decision but for
now we'll just run it here but the same
thing applies you could just SCP you
know run the pre-processing then SCP it
over I promise the SCP rate should not
normally be 11.2 mb/s per second so
anyway yes to save cool then we can just
run Python 3 point 7 ml example dot PI
which again will run the pre-processing
step first so that could take a little
bit I don't remember how well I
okay so first we're printing out some of
the errors so for one of a reason in
this training data set some of the
images kind of like don't actually load
for some reason so anyway what you're
seeing right now for the errors is just
right here for some reason we're unable
to read it with open CV
apprise you to use tiki DM or something
to show where we were in the process
that would have been smart huh but we
can court we can see at least where we
are in dogs although it's ads out order
that's weird hmm okay I think we're yeah
okay so we're now where I can see we're
loading things so let me move this over
so this is training our model for a few
epochs and it is exceptionally fast
this is hilariously fast just in case
you've never trained this model I highly
encourage you to take this exact code
and train it locally okay so we trained
a cast first dog model the accuracy in
sample access trees 91 that validation
accuracy is 74 basically so we
definitely over fit but we weren't
really trying to do anything special we
just wanted to run it but that's in
sanely fast to do a two second epoch -
for some reason when I wrote this script
locally I was getting like 24 second
epochs which is very odd because I run
an RT X Titan GPU which has the exact
same tensor T flops and the same no more
same or same same V Ram 24 yeah I think
it's the same vram so anyway not sure
why it was so lightning fast on Linode
but definitely curious to dig more into
that because this GPU is very comparable
to the GPUs on paper space so it sound
like this GPUs like super slow like it's
about it's a little faster than the V
100 so I don't know why these are so
fast but anyway I'd love to look more
into that and also if you want to see me
compare this RTX 6000 to a V 100 like on
paper space let me know the other thing
I wouldn't mind checking at some point
is many so the V 100 was meant to be a
server GPU that you would just have a
bunch of so I do wonder if for art
six thousands still significantly
outperform for v1 hundreds but that's a
very expensive test so depending on how
many people really want to see that let
me know because I'm curious but it's
also expensive so we'll see anyway so
that's that that is basically the entire
workflow that should work everywhere now
because of what you just saw with that
SCP delay and all that sometimes these
things can take a while to to transfer
for who knows why I'm gonna wager I
don't know I just don't think an SCP
should have hit any sort of firewall or
something I don't know what's going on
there maybe that's something else you're
sharing the network with people so maybe
the network's just under super heavy
load I don't know it should be faster
than that though to be honest okay so
the other thing that I would show is at
least on Linode and other places that
have these like separate volumes like
this one thing you can do is detach and
reattach pretty quickly and that is the
case on Linode so rather than doing all
the SCP nonsense you can have an ml ml
storage right that first you've attached
it to your data server you do your
pre-processing so in this case let's
call extracting pre-processing and then
we can attach it to our GPU server so
actually what what I'm gonna do is I'm
gonna CD out of the deep learning
directory and then
okay I'm gonna RM our deep learning
see you later freak and now what I'm
gonna do is come over here and it would
probably be wise to pseudo shutdown on
his age now and I'm gonna shut down this
server as well pseudo shut down now
because we're gonna move the volume and
if we move the volume while the server
is on we could like at least right now I
am under the impression that I have like
nothing is modifying that volume but you
might make the mistake one day and then
that volume gets corrupted or whatever
and if you don't have backups you're
gonna be really sad so just go ahead and
shut them both off and then what I'm
gonna do is I'm gonna pop over here I'm
gonna wait until this says that they're
offline okay we're gonna come to the
volumes and then what I'm gonna do is
click this little arrow here I'm gonna
say detach so currently ml storage has
all that training data already on it
so now I'm going to detach it from from
that that data server but the data is
still going to be on this volume so
detach and then we'll wait for that to
get attached to be pretty quick okay now
we want to attach and we're going to
attach it to our GPU server save now
once it's attached we want we have to
like re kind of sort of set it up but
don't run that first command because you
will recreate the file storage and
you'll basically just kind of like
overwrite it because it's the exact same
command so if we go show configuration
don't run that command this is the one
we want to run so so now what I'm going
to do is I'm going to come down whoops
do do to do and reconnect again to our
GPU server jellyfish and we're going to
run this command here loop and then the
second command here
I'm not gonna run the other command just
to save time but you can run that as
well if you do the I'm gonna call it f
stab okay so now the mo storage is there
so if we change directory into mount the
forward slash mount ml storage and then
we list out that directory guess what
there's our pet images so now what we
could do is we could train a model save
the model right to this this storage
here and then again what we would do is
we'd come over here once we're done once
the the model and logs and all that is
saved to our ml storage volume click
here don't click on that one
detach I crack myself up
and then you'd move it back to your your
you know your data VPS right to do you
know your pre-processing or just to
store it so then you go over to your
linens and actually oh yeah I just like
tripped myself out there because I was
like food
I never booted back up those servers but
whenever you attach the attach or detach
it runs a reboot sequence so I removed
it from data server so it rebooted data
server after it was removed and then the
GPU server when I attach it it booted
the GPU server but for a second there I
was like wait I never turned it back on
anyway okay so now what I want to do is
before I forget actually I want to turn
both of these guys off power off power
off and power off but basically at the
end of your training a model you would
detach that volume power off the GPU
server and nuke it so basically you
spent 10 minutes really spinning it up
setting it up and then the attaching and
detaching of the volume is nearly
instant it's pretty quick or you can use
SCP which is variable that's not the
first time it's not just Linode that's
not the first time SCP has been quite
slow for me even locally so some
time's it can really bite you in the
butt and that kind of sucks but anyway
at least on Linode that's how you can
even avoid that you just simply detach
attach the entire volume and yeah so
okay that was a ton of information
hopefully you guys have learned
something useful if you want GPUs in the
cloud at least right now Linode is the
place to go again if you haven't created
an account up to this point you can sign
up for loan owed and get a twenty dollar
credit by lynda.com slash sin tax
questions comments concerns whatever
feel free to leave those below if you
have also you can join the discord let's
this cordage e slash sin tax if you've
got questions that you want to answer in
there as well so anyway that's all for
now I'll see you guys in another video