Multiple regression - statistical inference and hypothesis tests Including Excel
Hi. I'm Dr Nic, and in this video I'm going to explain more about the statistical inference
involved in multiple regression. This video follows on from Multiple Regression, Overview.
This video will be about t-tests for the coefficients of the independent variables
and the F-test for the model as a whole. I will cover confidence intervals in another video.
Check in the description below for more videos about multiple regression and videos about
inference to develop your understanding. Some may be special content for channel members.
To start off with, I will talk a little bit about variation, population and sample.
We wish to find a fair price for a used Prius. If we knew the asking price for all used Priuses
for sale in New Zealand in January 2021, that would be the population.
We could find the population mean of the prices, and this would be
a simple model to use to estimate a price. There would be variability around that single
value of the mean, as not all used Priuses are going to have the same asking price.
We could calculate the sum of the squares of the distance of each price in the population
from the mean, which would give us a measure of the total variation in Prius prices.
It is called the Error sum of squares or SSE.
As a side note, if we then divided that by the number of cars, we would get the population
variance, and from that the square root gives us the population standard deviation.
Now we are pretty sure that the age and the distance the car has
travelled will have an impact on the price. We can account for some of the variability by
including age and distance in the model, thus improving our ability to predict a car price.
Using least squares we could find a population model that calculated
the price, using age and distance as inputs. This would reduce the variability in the model.
Note that we are still talking about the population of all the cars,
which is only theoretical. There would still be variability, but it
would be reduced by including age and distance.
Now, we get to the inference part of this.
We do not have data on every Toyota Prius for sale in New Zealand in January 2021
So we need to take a sample.
Ideally it should be a random sample in which every Prius has equal opportunity of being selected.
Samples are seldom ideal. See in the description below for a link to a blog post about this.
The data we have is a sample of 50 cars. We use the sample data to fit a sample model.
Using a sample introduces another source of variation - sampling error or variation due to sampling
This is the variation that occurs because this is only a sample and not the population.
You can imagine that if you took different samples of 50 prius cars, you would get different models. But they would be similar.
You can also think that the sample model you come up with is likely to be like the population model,
but not exactly. That is sampling error.
Statistical inference helps us to identify what we can say about the population model from the sample model we have calculated.
I will introduce some notation. When we talk about the model in the population, we use the greek letter beta.
The population model is that Y = beta zero + beta1 X1 + beta 2 X2 + and so on up to Beta k x k + epsilon.
Y is the dependent or response variable. x1 to x k are the independent or predictor variables.
Their values are inputs into the creation of the model.
Beta zero is the intercept or constant term. Beta 1 to k are the coefficients of the
independent or predictor variables, X 1 to x k.
Epsilon is the error of prediction.
For the Prius model, we could write it like this. I prefer to use meaningful subscripts where
possible, so I would write it as Beta zero, beta subscript Age and beta subscript distance.
The sample model is Y hat = B zero + b1 X1 + b 2 X2 + and so on up to B k x k
We use regular letter b instead of the Greek letter Beta to indicate that it is a sample estimate of the population parameter.
Sometimes people use the greek letters with hats, Beta zero hat, beta one hat etc.
This also indicates that it is a sample model approximating the unknown population model.
ttests are used to find if individual predictor coefficients are statistically significant
and an ANOVA test is used to test the model as a whole.
We developed a model for predicting the price of a used Toyota Prius in NZ in January 2021.
It has two predictor variables, age, and distance.
We wish to test if their coefficients are statistically significantly different from zero.
Computer output also gives t -statistics for the intercept, but this is seldom relevant.
We will work through the hypothesis test for the coefficient of distance.
If you are not familiar with hypothesis tests, check out the video, Hypothesis testing, step-by-step.
There is a link in the description below.
In the video "Hypothesis Testing" we gave five steps.
They are hypotheses significance sample p-value decide.
Hypotheses are always about the population parameters.
In the sample model developed from the sample data, the coefficient of distance is negative 25.
We wish to test if there is evidence from the sample that the coefficient
of distance in the population from which the sample is drawn is different from zero.
Our null hypothesis is that Beta subscript distance is zero.
There is no effect on price due to the distance travelled - in the population.
So H nought is Beta distance = 0
The alternative hypothesis, H1 is that Beta distance does not equal zero.
There is some effect on price due to distance travelled.
These are the hypotheses that the computer output is relevant to.
We will use an alpha value or significance level of 0.05 and
we have a sample of 50 used cars.
When we use Excel to perform the regression, we get an output that includes the p-values for the coefficients.
As the p-value for the coefficient of distance is very low - less than 0.0005, we can reject
the null hypothesis that the coefficient is zero.
There is evidence of an effect in the population from which the sample is drawn.
The same is true of the coefficent of age.
Say we tried fitting a model that included engine size as well.
You can see the output here.
The p-value for the coefficient of engine size is 0.236.
This is greater than our usual cutoff of 0.05, so we do NOT reject the null hypothesis that the coefficient of size is zero.
Though, for the sample there is a small effect due to engine size, there is no
evidence of an effect in the population.
Consequently we leave out that variable when choosing our final model.
See in the description below for links to videos about developing a regression model and choosing variables.
We can also calculate confidence intervals for the coefficients.
This is covered in the video on interpreting coefficients.
Ok, so that is each of the individual coefficents. But what about the regression model as a whole?
The null hypothesis is that all of the population coefficients are zero.
We write it like this: Ho: Beta 1 = beta 2 = beta 3 etc = zero.
The alternative hypothesis, H1 is that at least
one of the population regression coefficients is not equal to zero.
The test statistic is an F value and it and the corresponding p-value are
provided by the ANOVA table shown here.
It is called significance F instead of p-value.
You can see that the p-value for the F-statistic is 01.409 E-11, which is a
really really small number, so pretty much zero. We can say that there is strong evidence that
at least one of the coefficients in the population model is non-zero.
You could say that the model is statistically significant.
This video has been an explanation of statistical inference around multiple regression
In other videos I will explain more about different aspects of multiple regression.
Ask in the comments below if there is something in particular you would like.
Please like this video, subscribe, but most of all join the channel - especially if you are using our
videos in your teaching. Help the channel grow and help me help more and more people like you.
Joining the channel also gives you access to more helpful videos like these.
I am truly grateful for my channel members who help make these videos possible.
And I am particularly grateful for my super supporter, Ian Sim.
Thank you