Hi everybody!
Welcome to the third episode of AI Development 101,
In this one we are going to classify all the things.
Remember you can follow me here for more videos
at aka.ms/davidc.
So, by now you know how the basics of a regression,
and we also saw the math
behind the main regression algorithm,
which is called the linear regression.
Using a regression, you can get the machine
to learn a function h of x
that guesses an output number based on an input.
For example, it could take the square footage of a house
and try to guess the selling price.
In this video we are going to see
a different type of algorithm called classification.
Classification algorithms try to guess
a category instead of a number.
For example, it can take some feature of a flower,
like the length of their petals,
and it can try to guess the type of the flower,
for example virginica.
This is actually a real dataset of flowers
which is also considered
like the hello world for classification.
Here's another flower,
and in this case let's say that it's a Setosa.
The dataset has several flowers and features,
but let's imagine that we have only two types of flowers,
Virginica and Setosa,
and we are going to use only the length of the petal as the feature.
We can represent this graphically
just like what we did for the diabetes example.
Let's say that the x axis is the petal length,
and the Y axes is going to be the type of the flower.
Because I have only 2 types
my Y axes is going to have just 2 values,
let's say that virginica is value number one,
and setosa is zero.
Let me draw some examples in this chart.
Random fact of the day for you,
setosas have short petals,
so they will all be on this left side of the chart.
And then I will have a bunch of examples of virginicas
mapping to the Y equals 1.
And in this case on the right
because they have bigger petals.
These videos are not only educational for machine learning,
we are now expanding to new horizons, like botany.
OK. So now I have to think of a function h of x
that can map to these points.
You know this saying that when you have a hammer,
all you see is nails.
Well, we can try to use the linear regression
That we already know and love.
And the equation was, if you remember,
theta zero plus theta one x,
the resulting line would look more or less like this.
I meanů not the best h of x in the world,
but it can work.
Imagine that I get a petal length right here,
the output of the model would be here.
An obvious approach now is
to identify anything below 0.5 as a setosa,
and anything bigger than 0.5 a virginica.
So this one would be a setosa, which makes sense,
because it has a small petal.
And you can see that actually all the examples
would be classified correctly by this model,
because all the setosas are
to the left of this line,
and all the virginicas are to the right.
But unfortunately this was just luck.
Because imagine that I have now
another example of a virginica, right here.
Just because I happen to have
this additional example that is quite big,
my linear regression would move.
The 0.5 line would move too,
and the model wouldn't work well.
For example, it would classify this flower as a setosa,
which is wrong.
The good news is that there's a solution for this.
We are going to use a different h of x.
Instead of a line we are going to use an expression
called sigmoid function or logistic function.
The equation for this function is a little bit more complicated,
but it's going to be worth it, I promise.
It's one over 1 plus the number e
to the power of minus the same equation
we have before, so theta 0 plus theta 1 x.
The cool thing of this equation
is that it's going to look like an S,
that's why it's called sigmoid.
It fits the dots in my chart much better
than the line that I used before
and it works better for extreme cases
because you see that it flattens over time.
This approach is called logistic regression.
And I know that the name is super confusing,
because it has regression on it,
even if I'm doing a classification.
It's called regression because the output is still a number,
between 0 and 1,
and not just a category.
But you can also interpret that number as a probability.
For example, if you get this input,
imagine that you get an output of 0.8,
you can say that this flower has a
80% chances of being a virginica,
which is quite cool.
And just like the linear expression,
I can control the shape of this curve
by changing the parameters theta 0 and theta 1.
So you can get different offsets
and different inclinations
to get other functions like these other two.
So, how do we find the best curve to fit my samples?
How do I pick the best theta 0 and theta 1?
Remember how we did that?
We are going to use a cost function,
measuring the distance of my predictions
to the real values, so these distances here.
Let's call them c i.
Let's use now the same equation
as the one we used for the linear regression,
so y minus predicted y squared.
Then I can define my cost function
as the average of all these distances,
so the summation of all my c i's.
Now I want to get the parameters theta zero
and theta one to minimize this function,
and for that we can use our friend,
the gradient descent.
But we are going to have a problem.
If we define the cost function just like I just did,
the shape of it is going to be
more complex than the nice bowl of soup
that we had for the linear regression.
It's going to have several valleys instead,
and that is going to be a big problem.
Why? Because you can end in the wrong valley
if you apply now the gradient descent algorithm.
Imagine that you start here
and you move down the mountain
using the gradient descent algorithm.
So first step you can end here, then here, then here and so on.
Oups, you see what happened?
I ended in the wrong valley,
I should have ended here which is deeper.
This problem is super common in machine learning
and it can give you a big headache.
Fortunately, there's a way to fix this
for the logistic regression.
Instead of using this formula for the cost function,
we are going to use a different one.
And it may look daunting at the beginning
but don't you worry, it's not that complex.
It's going to use logarithm functions
instead of squares with some funny math.
The trick to understand this formula
is remembering where the Y's are coming from.
Y's are the examples in my dataset, right?
So these two Y's are going to be the examples,
Therefore, they can only be two values:
one and zero.
When Y equals 1, so if the flower is a virginica,
this whole thing would be zero
and the whole part,
the second part of the equation will go away.
So the resulting function when Y equals 1,
so when the example is a virginica,
is just this component, a logarithm.
Let me show you this component in a plot,
so you can understand what is happening here.
Let's represent c i in one axis
and the predicted y in the other.
A logarithm is going to look like this.
Now imagine that my guess is 1, so the guess is correct,
because this was the function when Y was 1.
That would be this point here.
So when my guess is correct
the cost function is zero.
And that is perfect.
If my guess is incorrect, so if my guess is zero, a setosa,
then it would be this point here,
and at that point the cost function will be super high,
actually infinite.
If it's close to zero it would super high,
for example this point here.
So my cost function gets really high
when the guess that I get of my example is really bad.
The other part of the equation is for the other case,
so it's going to look exactly the same
but the symmetric of it,
so I keep penalizing the model
when the guessed number is different from the example,
that's the whole point of the cost function.
The beauty of this new cost function is that
the resulting function is another nice bowl of soup.
And that's wonderful,
because I can now apply my gradient descent smoothly,
and walk down the mountain
and end in the minimum value of this cost function.
Actually, here's a fun fact.
The formulas for the steps are going to look very similar
to the ones that you know for the linear regression.
So, if you calculate the derivatives for the cost function,
it's going to take some math
because you have exponentials, and logarithms,
so it's going to be hard,
but let me show you the final equations.
Do they look familiar to you?
Because they are effectively the same derivatives
that I had in the linear regression.
Like tadaaaa.
This is the one with respect to theta 0,
and this is the one for theta 1.
So effectively they are the same ones
that we had for linear regression.
So the training that we saw before for the linear regression,
it's going to be exactly the same.
We will have the same loops with epochs and iterations.
And the only difference
is how we calculate the predicted Y
for the updates of the thetas
using this formula instead of the linear function.
And with this we just ended episode 3.
Now you know the ABC of machine learning.
In the next episodes we are going to introduce
other types of regression and classification algorithms
to end in the big star of machine learning,
the neural network.
You can't miss all this fun, so remember
That you can watch these videos at aka.ms/ai101,
and you can follow me here in LinkedIn at aka.ms/davidc.
Thank you for watching!