I'm going to start with parametric regression, which is a way of building
a model where we represent the model with the number of parameters.
Let's start with a simple example.
Suppose we want to build a model that will predict how much it will rain today
based on changes in barometric pressure.
So as you may know, if barometric pressure declines,
that usually means there's bad weather coming and it's going to rain.
And when barometric pressure increases,
it typically means that we've got good weather coming.
So on this scatter plot each individual point represents one day.
So let's consider a particular day, say here.
And what this means is that on this day, the barometric pressure decreased by 10
millimeters and we had 50 millimeters of rain, about 2 inches.
So let's consider that over time we collect data for many different days and
that's what this looks like.
So again, each one of these dots represents one day's worth of data and
we have multiple day's worth of data here.
And as you can see there's a general trend of as barometric pressure
decreases, we typically have more rain and as it increases we have less rain.
We'd like to create a model based on this data that when we query it at any
particular point it'll give us a prediction of how much is going to rain,
so we would measure barometric pressure or
its change and then estimate how much is going to rain.
The classic solution to this problem is to fit a line to the data.
So let's give that a shot.
As you probably know, this approach is called linear regression.
And the model looks like this.
So if you remember, from algebra in elementary school, or high school,
wherever you got it.
[LAUGH] The equation for line is simply y equals mx plus b.
So x is our barometric pressure change variable here,
and m and b are the parameters of our model.
Our model now is fully described by these two parameters, and if we want
to estimate or query how much it's going to rain at any particular point.
We measure the barometric pressure.
Let's say the barometric pressure today increased by 5.
We would then plug that 5 into our model here and
multiply it by m and add b.
And that's our estimate for that day of how much it's going to rain.
And the linear regression approach is how we arrive at m and b.
This model is decent, but
it doesn't track the actual behavior of the data, for instance,
in this region and in this region, so we can make a more complex model.
Instead of fitting a line we can fit a polynomial, we can add one more term x
squared and now we've got to also find this additional parameter m2.
So, when we find our model it's now represented by three parameters
mm2 and b.
This polynomial model will look something like this.
It fits the data pretty well over here, but not so well over here.
Now, we can add more terms.
We can fit an x cubed term and have another parameter and so on.
In general, these parametric approaches come away with
a number of parameters, and the more complex the model, the more parameters.
Still, three parameters is a pretty simple model.
All of these models, whether it's linear, with just mx plus b, or
polynomial, with a cubed or squared term, are parametric models.
In the end, after we learn these models, we have our parameters,
in this case, m2, m, and b.
And we throw away the data,
and the model's represented just by these three parameters.
But there's another way to approach it, and I'll show you that next.