Hi everybody!
Welcome to AI Development 101.
In these videos I'll be using this whiteboard
to explain the key concepts behind AI technologies.
And we'll go step by step,
starting with the basics of machine learning
and ending in more complex concepts
like reinforcement learning or even
massive NLP models and AI supercomputers.
You can watch more videos like these here,
including the equivalent series for AI in business 101,
or follow me at LinkedIn here for new videos.
OK, so let's start with the foundational element behind AI,
which is of course machine learning.
Today we are going to do like the Hello World of machine learning.
The essence of machine learning can actually be summarized
with this math expression.
Y equals H of X.
That's it, that's the most important math that you need to know.
Why? Because at the end of the day, a machine learning model
is just a function that is going to take an input,
that we will call X and an output,
that we are going to call Y.
The input is going to be a number or a series
of numbers, so a vector.
For example, it could be your sales history
or the pixels in an image.
The output is going to be a number too,
a vector, or even a list of categories,
for example the objects identified in a picture.
Our job in machine learning is to find that function H,
that can transform the input X into the output Y.
For example, it's going to transform our past sales history
into an output with a guess
on the forecast for this month.
It's called H because it's the hypothesis function,
so it's guessing that output.
So now, this is the interesting part.
How would you approach this in traditional programming?
In traditional programming you would develop a program
implementing that function H.
You would need to find out an algorithm
to predict your output,
whether is a sales forecast or identifying pictures
of dogs and cats, which is quite difficult.
Then the machine would take the input,
the program implementing H that you developed manually
and it will create that output y.
But in machine learning,
you won't be creating that function by yourself.
The machine is going to take examples of inputs X
with their corresponding outputs Y,
and then with those it will guess the hypothesis function, so H.
In a sense, the machine is learning by itself
based on the examples,
instead of being programmed manually by you.
So let's see this in an example.
Imagine that you have values from a research
on patients of diabetes.
This is actually a real example that is often used as the dataset
for the hello world of machine learning.
So in the y axis you would have a number
representing the evolution of a patient
with diabetes after one year,
and in the x axis you are going to have some attribute of the patient,
for example the body mass index, the BMI.
Now let's imagine that we have only a few patients,
for example patient 1 with the values x1, and the output y1,
patient 2 with x2, y2, patient 3 and patient 4.
With this data my goal now
is to find a function h of x
that can predict that evolution of a new patient
with a new input BMI that I haven't seen before.
The easiest way of doing this
is called linear regression.
And it's super simple,
what I'm going to do is assuming
that the relationship between these two is linear,
so my function would be something like this.
Hopefully you remember the equation of a line
it is something like this…
h of x equals theta 0 + theta 1 times x.
Theta 0 and theta 1 are called the parameters of my model.
In this case I have only two, in future episodes
we'll see crazy models with billions of parameters.
Now the key concept in machine learning
is finding the right values for these parameters,
in a way what they represent is,
I'm trying to represent my samples as good as possible.
So that process is called training.
In the case of a linear regression is quite simple.
You want to find the parameters
theta 0 and theta 1
that create a line like this,
instead of a line like this.
An approach for this could be to get a measure of
how good my function is
guessing the samples that I have already.
So, what I'm going to do is to apply the line
to the values X of my examples,
So that would give me a predicted value for each example.
Y1 predicted, Y2 predicted, Y3 and Y4.
You can see by the way that I'm using that symbol over the letter Y,
that's a common way of referring to the predicted output.
Then, how do I measure how good is my model?
What I can do is taking the distances between
the real output and my prediction.
So that would be this distance right here,
this distance here, this one and this one.
I need to come up now with a formula
bringing all of these distances together in a measurement, so in a number.
This is a critical concept in machine learning so keep it in your mind
because you are going to use it a lot.
Remember this, that measurement is called the cost function.
The cost function will use some equation, some formula,
to give you a sense of actually how bad is your model.
You will find this function to be referred as J,
and there are many types.
A simple way of defining it would be just adding all these distances.
So, y one minus predicted y one
plus y 2 minus predicted y 2, and so on.
And you can divide it by n so you have an average.
But this one is tricky because the distance could be negative,
so they will complement each other.
Another option is to have the absolute values.
So, that would mean having the absolute value of y 1
minus predicted y 1, and so on.
The most important one, the most popular one
is actually using the squares,
because that will penalize
the predicted values that are very wrong.
So, the formula would be y 1 minus predicted y 1, squared,
plus y 2 minus predicted y 2 squared, etcetera.
You can use the summation notation to make even this more compact.
You are going to use these summations
in most of the math for machine learning,
so it is good that you get used to it
if you haven't seen it before.
In this case it would be the summation from i equals 1 to n
of each of the distances squared,
all of that divided by n.
It's just a more compact way of saying the same thing.
So the averaged addition of all the distances squared, that's it.
This function is super popular in machine learning,
it's called Mean Square Error or MSE.
So, summarizing all of this, the cost function
is a function that is measuring how bad my model is.
And a popular one is the mean square error,
Which is just adding the squared distances of my predictions
compared with the real values
that I have in my sample dataset.
Now my goal is to find the values theta 0 and theta 1
that minimize the cost function J.
The good news is that
there's actually a closed solution,
there's a formula that will give directly the values
of theta 0 and theta 1 that are
minimizing that function, which is awesome.
However, I said that there's a solution but not that it's an easy one,
because the equations are quite long
so what I'm going to do is just putting here
so you can see here the final formulas
for theta 0 and theta 1.
If you want the details behind this,
If you want to see the math behind it,
You can find it anywhere,
the approach is called the normal equation.
Super popular, you can find it online.
The key thing of this formula is that
they will give you as an output
the optimal theta 0 and theta 1
for that line that we saw before.
The resulting h of x will be your first machine learning model.
Congratulations!
Now I can get a new value for the body mass index, the BMI,
and I can estimate the output
of that evolution of that patient.
So this is like the hello world of machine learning.
Unfortunately things are going to be
a little bit more complicated.
Because the output in this case
depended on only one factor, the BMI.
But in real life it will depend on multiple factors,
for example, the blood pressure or the age.
So instead of having as an input one number,
we are going to have many.
We are going to represent that with a vector.
So every sample will be now a vector with multiple numbers
x1, x2, all the way to x n.
Each of these numbers is called a feature.
So BMI is a feature. Blood pressure is another feature. Age is another one.
And the number of features is called dimensions or just n.
My training dataset will now be a collection of vectors,
one vector for every patient.
So I will have the vector x1 with the features for patient 1,
x2 for patient 2, and so on.
And the output y will still be a number in this case.
The good news is that the linear regression that we saw before
can also be used for these multi-dimension features.
We just need to add the additional dimensions
with a new parameter for each.
But here's a problem now.
As I increase the number of dimensions the equations
to get the values for each of those parameters
is going to be get more and more complex,
And eventually it's going to be very slow to compute.
In the diabetes dataset for example, which is super simple,
we have 442 samples and 10 features.
The resulting equations are going to be humungous,
they are not going to fit in this whiteboard.
In other cases it's just directly impossible
to find the parameters
for the model with a closed solution.
So how do we solve this challenge?
We'll see that in the next episode,
but here's a spoiler,
we will use a technique which is absolutely fundamental
to machine learning which is called gradient descent.
Remember that you can watch these videos at aka.ms/ai101
and you can follow me in LinkedIn for new videos at aka.ms/davidc.
Thank you for watching!