what's going on everybody and welcome to
a deep learning with Python and of
course pine torch tutorial series this
tutorial series is gonna start from the
very very basics assuming maybe you
don't know anything about deep learning
and then we'll be getting into more
complex types of neural networks and
applications for them so anyways this to
get started of course you're gonna need
pide torch you're also gonna need Python
I am assuming you already have at least
Python and my hope is that you at least
understand the basics of Python as well
as object-oriented programming so if you
don't have either of those two things
probably you should stop and either find
someone else's tutorials or I've got
tutorials on both the basics it's a
pretty short series it's like 13 parts
or something like that and then also
object-oriented programming again it's
not that many videos you should be able
to get through it pretty quickly you
should have a basic understanding of
Python because I'm just assuming you do
I'm not gonna be talking about basic
Python concepts and then object or in
your programming because in general your
normal network is going to be a class so
if you don't understand that you're
gonna be fuzzy about certain things and
then you're also probably fuzzy about
neural networks it's just not a good
combination
you should be solid on how
object-oriented programming works first
and then jump into this next up I'm
assuming you guys don't at least some of
you guys probably don't know about
neural networks or at least don't know
much so I'm gonna go really really
briefly into how neural networks work
you you can get by here with a pretty
high level understanding of neural
networks and be just fine and then you
could if you want to be very successful
you're gonna need a pretty deep
understanding at least for this series
we're not going to get too deep too
quickly because again you don't really
need that but if you're curious or you
want to get into more cutting-edge
research and stuff like that you're
going to need to so as this series
progresses we'll be getting a little
deeper into things but then also and
we'll be talking about concepts so I'm
just not gonna hit immediately because
there's just no need
and then also at some point I'm gonna do
in neural networks from scratch video
and by scratch I mean we're gonna use
numpy we need something to do like array
math or something like that but no
helper functions just because the list
would be really slow anyway it's not
important but if you guys want to get a
little deeper at you will be able to
both in this series and in the future
anyways assuming you don't know anything
about neural networks
I'm gonna go briefly over them but then
again if you're still kind of fuzzy
about just even sort of how neural
networks work you can look up videos by
three blue one brown that there's some
great visuals there also there's just so
much information online as far as like
how neural networks work at a basic
level but I'll do my best to explain
them really quickly so this is a basic
neural network you've got your input
data here and let's assume you've got a
Nisshin where you're trying you've got
images of dogs cats and humans and you
want a neural network that can
differentiate between an image of dogs
cats and humans so over here in your
input you've got features and these are
going to have to be numerical valued in
some way it might be the case that
they're pixel values and pixel values
are already numerical values right 0 to
255 for RGB or if it's gray scaled image
it's just going to be 0 to 255 or some
other number it's usually 0 to 255
anyways so they'll already be in
numerical form but your features could
also be so the input we call that
features this could also be some sort of
categorical type of feature so it's like
has four lengths has two legs
it could be has body covered in hair or
something like that or what kind of
color how tall is this thing that kind
of stuff is gonna be like descriptive
features and at some point we have to
convert that to numerical so you could
just do like a like just simply the
first thing is a 0 the second thing is
the 1 and so on and we'll talk much more
about that and how to how to convert
things to numbers later but just
understand it's always going to be a
number
that is being input it has to be a
number so anyways that's your input so
let's assume their pixel values because
that's a pretty common task so and it's
great scaled let's say so you have this
input and then it gets passed to these
things called
hidden layers they're called hidden
because we don't have control over those
layers the machine has control over
those layers we can read them we can see
what's going on but we're not changing
any values there after they come through
the hidden layers they get passed to an
output and again in general I'm just
gonna be super super vague they're not
vague but hopefully nothing super
general here let's just say this neuron
this is your dog neuron this is your
cat's neuron this is your human neuron
and the value that winds up here in
general we do an Arg max function that's
just going to ask which of these is the
greatest so dogs cats humans let's say
dogs is five cats is seven humans is
twelve right so that would mean the
largest value is humans so we would say
this neural network has predicted it's a
human okay so how does the machine has a
neural network even learn it's not
complicated at all we can look at really
complicated imagery like this it's you
don't even really have to think of it
this way but basically each of these
connections you can see how there's a
line drawn from every input to every
neuron here and then every neuron here
has a connection to every other one
right as you continue going forward here
this is what's referred to as a fully
connected neural network they don't have
to be fully connected and we can talk
more about how to do different things
later on but in in general this one's
fully connected and each line here each
connection is really a weight so you've
got this input value and then it gets
multiplied by a weight a bias is
optionally added in and a bias is just
an extra you know bias could be three so
it's you know the input times the weight
plus three okay and then it becomes a
value in here and then x2 the same thing
is done and it's and all these are added
together into this one neuron so
I've got another image here and like
this is basically what goes on in a
single neuron so you've got all your
inputs coming in those inputs are being
multiplied by a weight of biases
possibly added all those are added
together so they're just summed together
and they're passed through an activation
function so the activation function kind
of mimics a neuron in your brain right
is it firing or isn't it
and it could be a firing or isn't it so
it would be like a stepper function but
in most cases we use a sigmoid function
so it's actually a range between zero
and one in that again it's just kind of
a mimic both a neuron firing or not and
also it serves a purpose of keeping
values from just exploding in your
neural network it just kind of keeps
things again between zero and one one of
the key concepts that just keep coming
up time and time again is that in
general normal networks appreciate
values between zero and one or negative
one and one in some cases so always keep
that in mind so your input usually
you're gonna do what's called scaling
and scale it down to be between zero and
one and then yeah even through your
network once values start exploding into
these gigantic values something's
probably going wrong but way more on
that later the one thing all you have to
really think about is what a neural
network does all it does is each of
these weights is what we call a
parameter so same with the biases so
weights and bias even this really really
simple neural network that I've just
handed drawn here look at all of those
parameters every single line is is a
parameter probably two parameters if we
consider weights and biases so what the
Machine does is it has a it has the
ability to modify every single one of
those weights independently right so it
just keeps tweaking all of those so that
when we pass data when we train a neural
network we're passing in this input data
and we're telling it let's say it really
is a picture of a human we would say the
desired output the output I wish you
would make in this case is a zero zero
one right please make that and it makes
its output
and then we determine how wrong that is
with loss and then we have an optimizer
that goes back and attempts to update
these weights in such a way that it gets
closer to predicting the targeted output
and we tend to do this over batches over
millions of samples and slowly over time
the neural network learns what exact
values to put for each of these weights
to hopefully generalize over your data
and predict but at the end of the day it
just it's like this gigantic function
with like a million variables sometimes
more like you like I'm not even being
I'm not exaggerating like three million
variables is a pretty small neural
network 30 million variables is a
regular neural network so it's so many
things that it gets to tweak and change
and of course with that comes things
issues like overfitting and stuff like
that and we will talk about that moving
forward but really all a neural network
is is just this huge function where we
allow the machine to slowly tweak
variables and parameters to output
desired results okay so now hopefully
you have a general and basic
understanding of how neural networks
work again I I think it's completely to
be expected that you're somewhat vague
on how things work I think one writing
it out in code would help but also to
there are just vast resources online as
far as depicting how neural networks
work and if that's how you learn I would
strongly recommend the three blue one
brown just type of like to YouTube three
blue one brown neural network and I am
sure you'll find the series he is
wonderful at visualizations way better
than my hands are on photos so anyways
moving along we're gonna be using PI
torch now chances are you've heard of
for sure tensor flow which is another
machine learning library and there's a
quite a few others I want to use PI
torch one I just start I just decided
one day to learn PI towards myself and
as I was learning it I was like hmm this
would be a really good language to do a
you know beginners deep learning series
because PI torch is like super friendly
to the
grammar especially a Python programmer
as opposed to tensorflow and in one way
it is is with the graph and the great
thing about pipe torch is you don't even
need to know what the heck is the graph
what is he talking about you don't need
to know with PI torch it's great with
tensor flow you have to you have to
learn about the graph because you have
to work around the graph and it's very
challenging
whereas in PI torch you really you just
don't need to know about it and so and
then also people call it like more
pythonic and like what does that
actually mean in general like with PI
torch you write Python programming like
you write the object you deal with it
like it's an object you you its eagerly
executed and there is tensorflow eager
but it's not the main tensor flow and
there are certain issues and limitations
with using it also is much slower than
regular tensor flow and and then the
next question of course is how does pi
torch compared to tensor flow and speed
and they're pretty close actually but PI
torch is faster than eager tensor flow
and that would be the more fair
comparison because PI torch is eager and
what do I mean by that I mean you can
write a line of code in PI torch like
you can have your network and then you
run like one line or you run through one
layer and then you could check the
output you can see it or also with PI
torch you could you can branch out into
different layers and you write pure
Python code to do it it's just it's just
easy to make complex networks there are
some tedious things about PI torch and
we'll talk about them when they when we
get there but anyways long story short
we really use PI torch and you're gonna
love it so so let's click on get started
there's like a million ways you could
install this so I'm gonna let you
install it by yourself all this stuff
should be super simple for you to
understand
except for maybe CUDA if you don't know
what CUDA is go over here click on none
if you have an NVIDIA GPU of decent
quality you could enable you could get
the cuda version and you put operations
on your GPU and when you do that it's a
it's in general somewhere between fifty
and a hundred times faster which makes a
big difference
in training times to begin this series
at least for the first few videos all
the code that I'm gonna write is one I'm
gonna run it I'm gonna be running it on
my CPU and it's like totally capable of
being run on a CPU at some point you are
going to need access to a mid-range or
better GPU either locally or in the
cloud now if you have access to one
locally you can install it we'll be
talking about how to do that and then in
the cloud there are lots of options this
series is sponsored by Linode who also
happens to have the best prices in terms
of bang for your buck performance in the
cloud so you can check them out I've
done a video on exactly how to set
everything up on them as well I'm not
saying that because they're a sponsor
they're a sponsor because they are
simply the best right now anyways right
like over time all these all these cloud
providers are duking it out so at some
point someone might beat them on price
and I'll let you know when that happens
till then Linode has the best cloud GPU
prices by far it's like twice as good as
anyone else right now so anyway check
them out but like I said if if you don't
know what CUDA is don't worry about it
right now
just get the regular CPU version you're
gonna be totally fine you can learn the
basics without needing to run on a GPU
but at some point you will need to run
on a GPU so anyway once you have that go
ahead and do your install I am going to
pause for a moment and assume you guys
when I when I'm done pausing willing to
assume you have PI torsion stalled great
congratulations on your install so the
first thing I'm going to do is figure
out dude I have it oh I do already open
it up nice so I'm gonna be coding things
in the Jupiter lab but you can use
whatever editor you want because
remember you guys have done you're at
least not basic programmers anymore so
I'm assuming you guys can have your own
editor and be fine I'm going to use
Jupiter notebook just because one I can
yay for eager execution but also because
I just kind of make
since at least when you're trying to
learn the very very basics running line
by line is just useful also if you
needed to bug things again it's just
it's super useful so I'll be starting in
a Jupiter notebook at some point we'll
probably leave the notebooks behind but
for now cool
so first of all like what is pi torch
right like what is that
so again assuming you know the basics of
Python you're probably familiar with
numpy which is the numerical processing
library of Python now the problem with
known pi is what I just explained a
moments ago is that it doesn't run on
the GPU
so whatever deep learning first came out
there was really it was very difficult
to to really run these things because
the the your CPU the reason why things
are so much faster on your GPU is with a
normal network like I said the the thing
that is happening is each of these
weights is getting updated so your
machine is needing to run thousands and
thousands of operation of small cat like
not even that like millions of small
calculations basically and so so the GPU
trying to think how I exactly want to
describe this more order your let's say
your CPU so your CPU was made to do
these not really huge it's not like
these huge calculations but the CPU is
really there under the assumption that
most calculations that are going to be
made are gonna be me are going to be
large calculations like very much more
challenging calculations than modifying
little weights ok your GPU is your
graphics card right so it's competing
graphics and graphics there's a lot of
very small calculations so we run on the
GPU for that reason so so running on the
GPU is where we want to be for something
like deep learning because what we're
trying to do is a lot of simple
calculations and your jeep-like your CPU
probably has something between I don't
know 4 and 12 cores
ok a nice GPU has like thousands of
cores so it's just it makes a big
difference so let
come over here and so so like I said PI
torch is just numpy on the GPU with some
nice helper functions because a lot like
like you don't have to write your
optimizer and your backpropagation and
all these things that probably might not
mean anything to you yet but well soon
but at its basic core let's go ahead and
import torch and can I approach to
figure this one out first wouldn't mind
it making my font size larger it's got
to be a way besides zooming in but I'll
just zoom in for now so import torch and
then let's say you've got two two
variables so we're gonna say x equals
torch dot tensor and I mean this could
go many ways but let's say like a five
and a three and then you've got y equals
torch dot tensor and we can make this a
2 and a 1 we could then say print x
times y it's gonna take a second to
import torch there but normally that
operation will be much faster well yeah
we just we just multiplied these two
arrays so a tensor is initially kind of
a scary sounding word but basically a
tensor is an array like that's all you
need to think of right and so in a
basically a multi-dimensional array so
yeah so so it's that's as simple as it
gets right and what's nice about torch
is we can put this on the GPU and this
could also be on the GPU and then when
we multiply them together the GPU is
doing that math but again what just
happened just now was actually on the
CPU we weren't really asking very many
calculations to run so it was like no
big deal to actually run that the other
thing that we can do is like we can say
x equals torch zeroes and then we can
specify a shape let's say say it two by
five and let's go ahead and well one
thing we can say is what we could print
X then we could also say X dot shape and
that gives you like the size and and
before anybody's like it's just really
necessary why are you showing me all
these things basically all these
or things that we are going to use but
also if you're already familiar with
numpy one thing you might be starting to
realize is hey they've got zeros in
shape and all these calculations that
you tend to need to do or you've done in
numpy there's a torch variant so again
we could say y equals torch not R and
and then we could say 2/5 and then we
can output Y and there you have just a
random initialization of a 2 by 5 tensor
so the next thing I want to show is
reshaping so the one thing that for
whatever reason they have decided to
call something that you are not familiar
with is like when you want to actually
reshape something and numpy you would
say dot reshape and I believe in tensor
flow that would be a dot reshape in
torch it's not a reach a
you use a function called or really a
method called view so this is a two by
five so let's say we want to do what's
called a flatten operation so a two by
five in general you're not gonna be able
to put a two by five into a neural
network so let's say it's an image in
the images two by five pixels it's not
gonna be the case but let's just say it
was the first thing you have to do
before you feed this image through let's
say a basic neural network is you have
to flatten it so it would be a 1 by 10
right because there's ten total elements
if there's two by five if we flatten it
it would be a 1 by 10 well how do we do
that though so the way we would do that
is with a reshape and like numpy but
here in torch it's not view but one
thing to keep in mind I'll show in a
moment we would say 1 by 10 right so
there's your 1 by 10 reshape there one
thing to keep in mind is if we print out
Y again you'll see oh it hasn't actually
been modified so keep in mind that you
would need to reassign so you would need
to say y equals y dot view and I guess
you know the view sort of makes sense
like it's like show me this view show me
why
viewed as the shape or something like
that I don't really mind the name but it
is different from what we're used to in
terms of reshape but it's it's honestly
not that bad people made a big deal
about that whenever I was reading about
updates to tip I torch people are
complaining about that I'm like really
I mean views a good name so anyway okay
so so as you can see it's just doing
simple math with arrays basically now
again there are lots of little functions
that we can also import to help us
specifically with neural networks but at
its core it really is just a library to
help you do array math so anyways I
think I'm gonna cut it here and then in
the next tutorial we're gonna start
talking about the actual data itself and
then probably that'll consume a tutorial
and then we'll talk about building the
neural network and then training the
neural network so as strange as it might
sound we're gonna dedicate the entire
next video to data alone and even then
it's pretty simple but if you think
about it as a machine you know as
somebody who's doing machine learning
your biggest job is like the one thing
you have control over is data and then
you know you can you sort of have
control like in other ways like you can
change the shapes and sizes of your
neural network you get to dictate what
the structure is but in general the
biggest thing for you to do is provide
good data it's kind of like with a lot
of things like audio or something where
you say garbage in garbage out if you
put in bad data you're probably gonna
get bad data out so anyways in the next
tutorial we're gonna discuss data and
we're gonna go easy mode on the data
first and then on the next neural
network we build we'll get a little more
complicated but anyways we've we've
covered a lot of basic stuff here but
we've covered a lot of stuff if you have
questions comments concerns if I said
something wrong feel free to correct me
you can leave those below also if a disc
or that's disc or GG slash centex if
you've got any quick questions you need
help on something come check out the
disc court also special shout out to my
channel members who have been with me
now for a year
horse auth angel font although mr. gene
jeans and edward mccain thank you guys
so much for your support you guys are
amazing if you want to support the
channel you can click on that beautiful
blue join button get early access to
content shout outs a special rank in the
discord that's pretty much covers it I
think anyway that's it for now I will
see you guys in another video