In this segment, we will introduce you to nonlinear regression just
to give you an overview of what nonlinear regression is all about.
So, the nonlinear regression models work just like the linear regression models
so far as the fundamentals are concerned. So, we are given n data pairs right here and what
we want to be able to do is we want to base fit some function f to it, which will have
the constants of regression model in it. So, let's suppose if f was a straight line, then
it will be the intercept, and the slope will be the two constants of the regression model.
So, it might look different, but it is the same way as we did over linear regression models.
So, the philosophy is still the same, is that we have these n data points which are given to us,
and then if we try to draw a curve which best explains that particular data there will be a
difference between what is observed and what is predicted; this is the observed value and this
is what is predicted by the regression model. The difference between the two;
this part here is called the residual. So, your residual is y_i minus f of x_i. Observed value
minus predicted value, and what we want to do is we still want to do the same thing; we want to
minimize the sum of the square of the residuals, and that'll be summation I is equal to 1 to n,
y minus f of x_i whole square. And so, what that simply means is that we are taking each residual
squaring it and then we're adding all of them up, and what do we want to do? We want to be able
to minimize this Sr with respect to what? With respect to the constants of the regression model
of this function f. So, as I was mentioning that if it's a straight line, it will be
the intercept and the slope and similarly, for the nonlinear regression models you'll find out that
it has similar kinds of constants with respect to which you want to make Sr to be as small.
Let's look at some of the common regression models which are used in science and engineering.
So, one of them is the exponential model so as you can really see here there is that it
has two constants of the regression model a and b. So, you are given data y is a function of x
and what you are doing here is that you are trying to regress this data to an exponential model.
A good example of an exponential model is if you have heard of somebody getting a CT
scan let's suppose because they want to figure out what's wrong with the person,
so what they do is they give you a radioactive dye and that radioactive dye goes into your body
and then they're able to when they do the scanning they're able to see that hey if there is something
wrong with that particular person, but if you're going to put a radioactive dye in somebody's body,
there is going to be a radiation intensity which is going to be in that person. And this radiation
intensity is going to decrease as a function of time and that's and that decrease which
is taking place with respect time is given by an exponential model such as this one.
So, as time increases, you'll find the radiation intensity is exponentially decaying
and in order to be able to figure out that if you want to figure out the regression model based on
if somebody's measuring the radiation intensity as a function of time and then you want to regress it
to an exponential model that's the one which you would use based on the physics of the problem.
Let's look at another nonlinear regression model. It's called the power model,
and here what we have, again a and b are the two constants of the regression model. A good example
of that is that you might have seen a parachutist sometimes, and the parachutist has a parachute
which is using you know to come down safely, and then you can very well see that on this particular
parachute there's going to be a drag force. So, this drag force which we have here drag force
it is going to be given by some constant c times v raised power b, where this v is simply the
velocity with which the parachute is falling down. So, you can very well see that if you somebody is
giving you the data of drag force as a function of velocity at different points
then you can calculate what these constants of the regression model are.
So, for example, this is a measure of the drag coefficient of the parachute.
Let's look at this another example here of a nonlinear regression model, the saturation growth
model. The reason why it's called saturation growth model is because at x equal to zero you
find y is zero but the reason why it's called saturation is because as x approaches infinity
y approaches a and you can see that just by doing some calculations or by just some
manipulation here. So, if I divide by x both the numerator as well as the denominator,
I will get a divided by b by x plus 1 and you can see that as
x approaches infinity, the y will approach this value a. So, that's like the saturation
growth model and the way it looks like is something like this if this is y and this is x
and these are the data points for the saturation growth model, it will look like this so it's
eventually going to saturate. I'm going to start with 0, and eventually it's going to saturate to
a constant value, which will be a. A good example of this one is something called
I'm sure that you have seen an animated picture in your life and if you look at the if you try to
plot the performance; performance let's suppose we can measure it by how good does the animation
look. So, that can be a measure of performance and then on this side, on the x-axis can be the frame
rate; frame rate meaning that how many frames are being used per second. So if you are using
many frames per second you're going to find out that you're going to get a better performance,
meaning that hey how does it look you must have seen some of the movies which have been made
lately animation movies you find that they are almost as good as real life and real-life movies.
But, at the same time that if we increase the frame rate too much, we may not get any more
performance from there, any more big change in the performance, so you want to keep the
frame rate to be at a certain percentage of the maximum performance you might expect
which will be let's suppose 100 percent and you don't want to use very high frame rate because
that costs money, that costs computational time. So, you want to come up with the frame rate which
will be the best one for a particular scene. So, in this case, you'll have a saturation growth
model the performance is zero when there is no frames being shown to you right, but as the frame
rate increases, more and more frames are shown per second, you will find out that the performance of
the animation which you will see turns out to be a good one. So, that's the nonlinear regression.
So, let's um go back to the exponential model and just repeat what we saw for the general
regression model. The reason why we do that is because in the general regression model, we use
a function called f of x and then people say hey what are the constants the regression model. So,
here they are for an exponential model right here a and b. So, all we are doing is that
given the end data points which we have here, we want to be able to find the difference between
what is the observed value and what is the predicted value. So, your observed value is y_i
and your predicted value be a e to the power b x_i, and what you want to do is you want to again
do the same thing as we did for linear regression. We want to take each, we want to take each of the
residuals, we want to square it and then what we want to do is we want to add all of them together
so this will be a e to the power b x_i and we want to add all of these residuals, so the square of
the residuals and we get the sum of the square of the residuals. What do we want to do now?
We want to minimize with respect to a, and we want to minimize with respect to b so that we can find
out what these constants of a and b are and that's something which we'll do in the next lesson,
and that is, of course, the end of this segment.