In this segment, we'll talk about that many times you might be asked to do polynomial regression,
but you would not know beforehand what kind of order of polynomial regression to use. So,
we would like to use a polynomial which is optimum nature, it's not too high it's not
too low. Not too high that it defeats the whole purpose of doing regression in the first place,
and we don't want to be too low there it doesn't really capture what the data is trying to tell us.
Do is when it has this question: what polynomial model should we choose if one needs to be chosen?
And when we do we do it to an example. So, here are here is a typical example given to us where
we're given thermal expansion coefficient as a function of temperature, and there are 22 data
points given to us. So, if 22 data points are given to us that would make us believe that we
can use a polynomial order of 21 or less to make to go through those 22 data points. So,
the question is that hey what order of polynomial which will be 21 order or less should we be using
for addressing to this particular data? So, for example, if we go ahead and take these two data
points and here, what I have done is that I have taken a first-order polynomial and message to
that, and that's the straight line which we get which explains that critical data. Now,
what I do is I take a second-order polynomial and regress it to the 22 data points and this is
the kind of curve I get. Now, visually this might seem to be appealing, but what stops us from going
to a third-order polynomial or a fourth-order polynomial and say hey that might be a better
model to choose. So, want be able to make that decision and not based on looking at our curves
visually, although that can be one of the checks, but also to be able to do this scientifically.
Let's go and see how we can go about doing that. If you look at these two graphs keep in mind that
these two graphs are exactly the same they are drawn as a function of the order of the
polynomial. This particular left graph which is seen here starts with M equal to 1 and this right
graph here starts with M equal to 2. So, I just want to make that clear right in the beginning so
that you guys don't think that these two are two separate, separate graphs. So they are exactly
the same graph, however the starting point on this one is m equal to 1 which is right here,
and the starting point on this one, here, is 2. So, what we are plotting here is the sum of the
square of the residuals as a function of the order of the polynomial. We know that we have twenty-two
data points and we can take a first-order polynomial, a second order, a third order,
and so on and so forth all the way up to 21st order polynomial to do the regression. And,
as we can see that, if we look at this particular graph, which starts from m equal two, mainly to
illustrate that, at 21, when I choose 21st order polynomial which will use all the data points that
means that you pass through all the data points, I should say, and in that case, Sr will be exactly
equal to 0. So, for m equal 21 Sr is equal to 0 for any other value of M, which is less than 21,
the sum of the squared residuals will not be equal to 0. So, if we use that hey if we use
the criteria, criterion to choose the optimum polynomial to be hey let's pick up a polynomial
for which the sum of the squared residuals is as small as possible, you'll find out that the
polynomial which goes through all the data points it would be the optimum one to use and we really
very well know that that's not regression. We are trying to simplify, to explain the data by
a simple model. Here we're going to take to get 22 data points and make it go through a
twenty-first order polynomial. So, there should be some mechanism of choosing an optimum polynomial.
Let's go and see what we. So here again I want to make sure that you understand that both of these
graphs are exactly the same. This one starts with M equal to one, and this starts with M equal to
two. That's the only difference which. So, left one starts at M equal to one, the right graph
starts in two. The two graphs the same. However, what we are plotting here is this quantity called
Sr divided by n minus M minus 1 which can be also written as Sr divided by n minus M plus 1. So,
that's what we are plotting we are plotting this quantity here. We're taking the sum of the squared
residuals and dividing it by the difference between the number of data points you have and
the number of constants we have in the polynomial model because if it is an mth order polynomial
it has n plus 1 constants like a first-order polynomial has two constants, the second one is
three constants, and that's what this n plus one is coming from. What you are going to find out is
that this numerator here which is the Sr decreases as a function of M. It goes down as M increases.
So does n minus M plus 1 right here. This one also goes down as M increases. You can see that if n is
constant because it's the number of data points, as you keep on increasing M this particular
quantity is going to decrease, as M increases. The same is the case here there the sum of the
squares the residuals is going to decrease as M increases. So, what are you basically doing is you
are mating, making the two then the denominator and the numerator compete against each other to be
able to choose that optimum polynomial order which you want to choose. So, here what we are finding
out for this particular example we are finding out that hey maybe m equal to two seems to be enough.
Because after M equal to 2 what happens is that the sum of this particular quantity right here,
which is the sum of the square of the residuals divided by n minus m plus, n minus M minus 1
turns out to be not having a big change after that, and we're not necessarily looking for the
optimum for a minimum value of this quality right here because we expect that this quantity will do
something like this, and this is the optimum polynomial M which we'll choose. But, that's
not necessarily going to happen all the time in this case here it is continually decreasing. But,
as you see, that after two, the values for three, four, five six, and seven, for example,
are of the same order for all of these quantities right here, for this particular quantity,
which is right here, so the same order. So, why not, why not stop it too - in order to be able to
say hey that's the optimum order of polynomial to use. And that's the end of this segment.