i need to explain this ai concept or else i have to eat this lemon by the end of this video i need
to explain it to my non-technical girlfriend in eight minutes or less kind of like you're
falling down a slide you go from this part of the slide to this part of the slide and then if you're
still falling then you have to keep falling until maybe you you don't know marble sorry 20 seconds
you have a roll of marble and it sometimes goes up the other side right you ever do that so then
do you know anything about gradient descent is that a video it's a video game in the mind of ai
i am going to start this eight minute timer right here if i don't explain this concept correctly
it'll show from a pop quiz that i'm gonna give her at the end of this video and then if she gets even
one of those questions wrong i'm gonna have to eat this lemon are you ready for my sake oh yeah
this is professor meatball my name is andrew and let's get started all right eight minutes
so the first thing you'll notice is that this is a blank screen one sec hold on black marker
black marker okay so 10 seconds gone let's say that you are trying to understand god just holding
the board is half the battle okay let's say that you're trying to understand this marker is dead
okay so let's say that this is um sanrio these are sanrio characters and this is going to be like uh
money that they make right and how and how old they are right so hello kitty is probably
one of the oldest so it's going to be like like this probably like the actual like uh
amount so you're gonna have uh like dots here so this is probably like hello kitty this is probably
like bats all right and then i don't know who the newest one is gudetama is pretty new right so
goods is probably over here so this goods makes a lot of money but it's not very old well you
get the point right so there's going to be like a variable here and a variable here and that makes a
um a correlation should only already a minute has gone bad if you try to draw a straight line
that will basically predict how likely you can use like oldness or money to tell how old a character
is or how much money your character makes right so the the way to be able to find the best line that
predicts these two variables is with gradient descent so here is how gradient descent work
a gradient is just a fancy word for a derivative and a derivative is the relationship between how
fast something is moving for example uh you know speed on your speedometer right so like speed is
a derivative of position so like uh your house to the store right so how fast you are moving like uh
versus how fast you're moving is a derivative of position and a derivative of speed is acceleration
that's the easiest way to understand what a derivative is right
so here there are only do you remember uh uh drawing graphs right do you remember mx plus b
so m is the slope here so it's how tilty the line is and then b is uh the intercept so that's like
where on the the the map do we want to start off like zero for example so this is b right there
so this is going to be like a thousand dollars or something
no at zero dollars this is like gonna be like 30 years old right that's how old the intercept is
and then how tilty this line is is m so we need to find both m and b in order to find
the perfect line here all right so m is what again slope remember that slope that's gonna
be part of the quiz m is gonna be slope and then b is gonna be intercept so in order to find the
the derivative of one of these variables you need to use a partial derivative and a set
of partial derivatives is called a gradient okay so gradient's just a fancy word for a derivative
so remember what derivatives are again this this stuff so it's speed acceleration
acceleration is uh a derivative of speed and speed is derivative of position right so so like this is
like your house like this is like a google maps location this is like how fast you're going and
this is how like fast you're changing your speed acceleration is like breaking is also acceleration
because it's negative acceleration i understand that makes sense okay good so just to say that
you're fine when you're not first you need to be able to find the derivative of let's say the
intercept so let's say that each time we have an actual data set we need to measure its distance to
a line that we make here right and then we're going and we need to make uh the further away
a point is is worse because that is that is like wrongness like if i make a line and then
and then the point actually is this far off that's how wrong my prediction is so that's called loss
does that make sense it's like a cost of how bad my prediction is so if for our loss function
becomes our our derivative here we want to be able to have this penalize further away points
more than points that are closer so that's uh the best way to do that is square for example
if you square three that becomes like nine right but if you square two that becomes four so four
is a smaller much smaller number than nine even though two is basically only one away from three
so this is a bigger punishment for having a worse prediction does that part make sense
so okay listen it's this this line with uh and all of these points have these dotted
lines okay so these are all the residuals so these are called residuals remember this
these are residuals and we're gonna find all these residuals so this is just this
number minus this number and then square them so that's called the sum of squared residuals
and that's our loss function okay two minutes left as we as we track each of these residuals based on
me changing b so if i change b it's going to look like just going up and down because
the slope is the same so it's just me going up and down this chart and so the residuals
are actually going to create a different chart the sum of square residuals as i change b
it's going to get more accurate or less accurate so it's actually going to look like this more
or less right so a bad b is over here but a good b is all the way down here with the least
sum of square residuals possible we can't see your board okay there so you see this is gradient
so this this is the sum of square residuals and this is um b so this is the intercept as i move it
back and forth okay so we need to find this point so how do we find this point well we randomize
randomly select one of these b's and then we look at the slope of this chart right
so the slope of this chart is the derivative of this chart but as we see the slope is going to be
like this value here that's m and then we multiply that by a very small number that's gonna be called
the learning rate so just remember these these things okay learning rate is just a random small
number maybe like 0.1 and that multiplies by m here as we move to a place where m equals zero
so when m equals zero then we have descended the gradient all the way down to the bottom
so the way to calculate where our next part of a gradient is is we multiply this learning rate
by the m so m times learning rate
plus the position that we're already in so this is this position so this is like plus b
uh and that gets us to the next location so since it's going in this direction it's kind of like
you're falling down a slide you go from this part of the slide to this part of the slide and then if
you're still falling then you have to keep falling until maybe you have a roll of marble sorry
20 seconds you have a roll of marble and it sometimes goes up the other side right you
ever do that so then it'll roll back down right so keep rolling until it gets to the very bottom
that's what we want when it gets to the very bottom we have the perfect b we do this multiple
times and that's multiple partial derivatives to get a great intercent at the very end okay
i was eight minutes i'm gonna keep the board up here let's just ask a soft question of do you
think you understand gradient descent now maybe okay well let's find out so the first question
so she can't get any of these wrong otherwise i have to eat this lemon it's right here it has like
moldy spots hold on yeah oh yeah i'm gonna have to it's also there's a pointy part here which i'm
afraid is gonna go inside the skin i'm eating the whole and we have to eat the lemon i promise um
okay so the first question is what is a derivative
let's see what your notes look like to the camera it's pretty good notes one page of notes i'm a
slow learner so what is the derivative here you can even point to the part of the board that
describes what a derivative is
this is the derivative here what about this
or his acceleration speed in position so derivatives are oh um
but can you explain it in uh in english no all right let's one question wrong a derivative is how
fast a variable changes based on another variable usually time one wrong one wrong
okay looks like i'm eating that lemon the second question is what is a learning rate
um it's a small random number yeah it doesn't have to be random but it is a small arbitrary
number the next question is what is a loss function i don't know there's two questions wrong
we went so fast man we went like this
all right let's let me ask a very general question what is the point of grading descent
it's this one right yes this is a gradient descending but what does this actually
tell us i took away the marble um example so the answer to this question is a gradient descent
helps us find the best line that fits two or more variables so the answer is gradient descent is a
optimization function that tries and finds the best line that fits a data set it has to
be between two variables or more but that is currently trying to find the best line
of fit between like amount of money and how old uh a sanrio character is all right that's
three questions wrong how many questions are there i'm gonna ask uh two more questions all
right we're already failing we have we've already failed what does ssr stand for oh
sum of squares what does the r stand for residuals yes yeah we got one question right someone scored
residuals i'm right now it's not attacking them okay the question about summon square residuals is
it does it penalize data that is more far away or less far away than the line of test fit
can you repeat that yeah the sum of square residuals penalizes our line of best fit
based on the actual data so how far away each residual is the residuals can be pretty far
away or are very close to the line of best fit does summer square residuals penalize data that
is further away or closer a farther yes farther i guess okay all right last question this question
has to do with marbles if your learning rate is too big what kind of risk does that bring
to your model over here with my drawing here or the marble rolling around and stuff
if this small number is too big i didn't actually get to explain this but you didn't
so how am i supposed to answer this can you see what could happen marvel is not even on there
no the marble is actually um an analogy there is no real marble but what could happen is that this
never stops moving and it just keeps getting like it goes yeah it gets kept getting thrown around
and that's called divergence which i never explained so i guess like there's no way you
could get that question right all right so we've gotten uh four out of six questions incorrect
so therefore i now have to eat the lemon would you like half a lemon with me no okay here i go
i've never actually eaten a lemon before you should get a napkin let's give you a plate
honey all right uh this is for the youtube content thank you guys for watching uh remember to stay in
school i'm still in school remember to stay in school if you have plans to not stay in school
all right here it goes ai or consequences episode one episode
one it's so much worse than i expected
and yet it kind of like no kind of expected this is a big lemon otherwise i would just like
swallow the whole i gotta take another bite oh there's seeds in this lemon you spit at the seeds
drop a like down below if you like this content i'm not even sure if i do
oh my god
i feel like matt stoney he's from san jose my teeth ow
i will take half a bite of this okay solidarity what happens when you are just born stupid i guess
it's actually getting better nothing was worse than that first bite
my teeth are really sensitive though
is lemon skin good for the colon
oh my god
wow
is water not lemon juice
well
there you have it my name is andrew tune in next time for more ai or consequences
wow i didn't throw that up goodbye
you