- [Charlie] In the previous screen-cast
I showed you how we can use
something known as the matrix approach
to multi-linear regression to determine model parameters.
In this screen-cast I'm gonna build upon that previous one
and I'm gonna show you how we can analyze
statistical properties of least squared estimators.
In particular, the standard errors of those coefficients
in our multi-linear regression model.
So recall, we have our model here,
our multi-linear regression model.
This can be written in matrix form as follows
where we have our vector of observations,
our inputs that go into the model matrix.
These are our experimentally collected data.
It's really important that you're adding in
a column of all ones at the very beginning of your X matrix.
And then we have your model parameter vector
and random errors.
And using minimization of least squares
we can calculate a beta hat,
this is an estimate for our beta vector,
the coefficients in our multi-linear regression model
that minimizes the least squares.
Now what we're gonna do is we're gonna calculate
statistical properties of the least squares estimators.
The fitted model in matrix form would be Y hat,
this is our model prediction, is equal to our observations,
our X array multiplied by our estimate
for the coefficient vector.
The residuals are always the difference
between our observations, that's Y minus our predictions,
which is Y hat.
There's a distinction here between Y hat,
that's our model prediction and our observations which is Y.
We can calculate something known as sigma hat squared.
Sigma hat is also known as the standard error
by calculating the sum squared of the residuals
also known as SSE and then we divide by N minus P.
N is the number of observations.
P is the number of parameters in our model.
And again, P is equal to K,
the number of regressor variables plus one.
By the way, just as a reminder, sigma hat squared
is an estimate for the error term in our model.
So we have this epsilon at the very end.
Epsilon is different for every observation
and sigma hat squared is an estimate for that error term.
In order to determine the standard errors
of the least squares estimators
we first must determine this matrix known as C.
C is equivalent to XTX inverse
that we've previously calculated
in that previous screen-cast.
This is going to be a square matrix, a K by K square matrix.
The diagonals of this C matrix are very important,
so these items are really important.
They're related to the variances
of our least squares estimators.
Once we have C, we can determine the covariance matrix
by multiplying C by the estimate for the error variance.
That's sigma hat squared.
So our co variance matrix is just simply sigma hat squared
multiplied by this C array.
The covariance matrix is also a K by K square matrix.
Once we have C we can extract the diagonal components
to determine the standard error of any single estimator.
So the standard error of the J element
of our beta hat vector is just a square root
of sigma hat squared times CJJ.
And I'll show you how to do this in an example
in a few minutes.
So we're gonna revisit our example
from the previous screen-cast.
We have a bunch of data here
for the satisfaction of hospital visits.
We have 25 observations here and we're trying to create
this simple multilinear regression model.
So we're continuing from the spreadsheet that I created
in that previous screen-cast.
We've created our beta vector, and really this is beta hat.
These are our estimators
for our multilinear regression model.
What we're trying to do is calculate the standard errors
on each of these coefficients.
In order to do that, let's kind of work backwards.
We first need to calculate sigma hat squared, which again,
sigma hat is known as the standard error.
We also need to calculate this C array.
C we've already calculated.
C is equivalent to XtXInv.
So this here, XtXInv is the same as C,
our C array that we need.
In order to calculate sigma hat squared,
that's SSE over N minus P.
SSE also is just a sum square to the residuals.
So what we're gonna do is we're gonna calculate
our residuals, the residuals,
the difference between the observations and the predictions,
then we divide that by N minus P.
We substitute that into the standard error
and we extract the diagonal components of this C array.
So I'm gonna add a couple of columns here.
I'm gonna add three columns here.
We first need to calculate our prediction,
and this is Y hat.
That's our model prediction.
Once we have the model prediction
we're gonna calculate our residuals.
That's the same thing as E over here.
I'm gonna first go over here and name our beta hat array.
I'm just gonna name that beta.
In order to calculate our model predictions
all we have to do is take X multiplied by beta hat.
So I can just do MMULT of X times beta,
press Enter, and these are our model predictions.
You notice that the regression output, if I click over here,
the regression output provides the predicted
and so we can just check to make sure
that we have the same thing, which we do.
I'm gonna go ahead and name this, Control+Shift+Down,
I'm gonna name this vector Y hat.
And now I can calculate the residuals as Y minus Y hat.
Press Enter.
And these are the residuals.
Once we have the residuals, I can calculate SSE,
the sum squared of the residuals.
Let's first go ahead and name our residuals,
Res for residuals.
And the sum squared is simply sum squared
of our residual vector, which is 101,039.
We're also going to need to know what N is.
N is the count of Y, for example.
We're also gonna need to know parameters.
This is the number of parameters in our model.
I'm just gonna put in five
because there are five parameters,
beta naught through beta four.
We are ready to calculate sigma hat.
Sigma hat is also known as standard error.
Standard error going by this formula up here.
Before we can calculate standard error though,
let's go ahead and name all these things,
including standard error.
I can go to the formulas tab, create from selection,
click okay.
Standard error or sigma hat is the square root of SSE
divided by N minus P
and we get a standard error of 7.21.
We can check to make sure that both of these are the same
as the regression output, SSE and standard error,
1039 and 7.2.
So let's go to regression output.
Standard error is 7.2, sum squared to the error SSE
that's also known as the residual sum of squared is 1039.
Now that we have sigma hat, again, that's the same as SE,
I'm ready to calculate the standard errors
on our regression coefficients.
And I'm just gonna put that up here, SE of beta,
this is just a label.
So right here I'm gonna have the standard errors
of our coefficients.
This first one is beta naught.
That's just the square root of SE,
our standard error squared times,
we multiply this by the zero zero component of C.
And so that's down here in this first element.
That's our first diagonal component.
And when we press Enter we get a standard error
for beta naught of about 6.04.
Now I'm gonna do the same thing.
I can just drag this down, but I'm going to change
that diagonal component to either the one, one,
the two, two, the three, three or the four, four positions.
So all those diagonal components.
And we end up with the following standard errors
for our regression coefficients.
We can compare those.
So the first one is about six, the last one's about 1.1
to what the regression output gives us.
The regression output gives us the standard error
of those various coefficients.
In the next screen-cast, I'm gonna show you
how we can perform hypothesis tests.
We can determine if these coefficients
are significant or not,
and we're also going to determine confidence intervals
on the coefficients.
And in some subsequent screen-casts
you're gonna learn how to create confidence intervals
on the main response and prediction intervals
on future observations similar to what we did
using simple linear regression.
Thanks for watching.