hey Ingo yeah hey do you know what time it is of course it's time for five
minutes with Ingo today we will talk about linear regression which is one of
the most basic data science and machine learning algorithms available some
people even claim it's not even machine learning it's around since about 200
hey you can predict values so I just treat it as any other machine learning
algorithm so the idea of linear regression is actually super simple
you have data points in a space and you want to fit those data points with a linear
curve just like this stick here so let's have a look how this can work
you have a stick and you have a couple of green data points over here and we would like
to get as close to those data points as possible well one way to achieve this is
you could take those rubber bands here and connect those rubber bands to those
data points just like I'm doing this right now and you see those rubber bands
actually they pull the curve the line around until it has a stable position
and this is really the basic idea of linear regression you want to basically
fit the linear line as close to the existing data points or basically while
minimize the distance between the data points and the linear curve okay let's
see how this algorithm works then so here again we have our our data space
two dimensions just for now we only have like one dimension X at the bottom here
and this is the dimension Y we want to predict here are our green data points
so for any given position on the x-axis this line then will tell us what is the
corresponding value for y well that's the whole idea of the of the linear
regression curve okay so the curve itself can be described as a formula
here Y is defined as alpha this is the intercept the position
where it actually crosses the y axis here plus beta which is the slope of the
curve times the value for X so you probably learned that already like in
basic math in school like many many years ago for every single data points
you can make the following calculation so you can basic calculate the value yi
so this value here with the same formula alpha plus the slope beta times Xi
the corresponding value but you see this is only an approximation so it's
not perfect so there is some error you making we call this epsilon i so you
need to sum this up as well those error terms they are also called residuals so
the algorithm now is super simple the whole goal is that we actually minimize
those residuals epsilon i in fact we want to minimize the sum of all of them
we want to be good for all the data points well instead of just minimizing
the sum of the epsilon i you can also minimize the square of the well of all the sum
of the squares so the only thing we can change are the alphas and betas so this
is basically what we can play with and then for all our existing training data
points we want to minimize the the sum of the squared errors this whole
idea is by the way called the least squares approach so now we're doing some
mathematical magic so I'm not going into all the details there but it's well not
super super difficult and that actually delivers a very surprising result so we
can see that we can calculate the slope better here just as the correlation
coefficient between the values x and y so that's just a regular Pearson
correlation and then we we basically normalize this correlation by this
factor here which is just the standard deviation for Y divided by the standard
deviation for X so that is something we can easily calculate and the intercept
alpha is equally easy after we got the slope beta we just take the average of
all the Y's minus beta times the average of all the X's so now we have
our two values alpha and beta and with those two values we have basically everything we
need to to really calculate our linear regression curve so that's working very
easily in one dimension so what can we do for multiple dimensions
well for multiple dimensions it's very similar it's a simple technique which is
called ordinary least-squares you can look it up if you want to know the
details but the basic ideas are pretty much the same so um yeah well those are
the basics of linear regression so Ingo are there any pros and cons of linear
regression oh sure so first of all the pros I mean
it's a very fast method typically and in terms of number of data points it's also
easy to understand I mean all those factors like the slope if you have
multiple X and you have those coefficients which sum up that actually
gives you some idea what is more important or less important for your for
your regression analysis it's not really great for many dimensions if you really
have a high number of dimensions techniques like support vector machines
for example are much better and also another thing which is problematic are
outliers outliers can really screw up the curve I can actually show this
easily to you here on the wooden board again so let's let's actually go over
here if I have just one more data point so I was hiding it in the beginning it's
here at the bottom right and I just add another rubber band look what happens it
really changes the slope of the curve like like big-time
so only one data point actually can change the whole slope the whole
regression coefficients again this is something where other regression
well techniques like for example the support vector machines are just much
better and more robust against outliers but it's a simple tool it should be in
your tool belt like naive bayes, K nearest neighbor, decision trees those are kind
of the basic methods you always have in your tool belt easy to understand and
as such it's a great method thank you for today