In this video, I will be answering these questions based on this partial output for multiple
regression coefficients. The regression analysis is to predict sale
price based on house size,
number of bedrooms, age of the house, and distance from the city’s core.
Since we have 4 independent variables or predictors here,
the multiple regression model could be written as follows.
It could also be written in this form based on the definition of the X variables here.
beta_0 is the constant whose estimate is referred to as intercept here,
and beta_1 to beta_4 are the population regression coefficients,
or parameters, for the four independent variables. They are also sometimes referred to as population
slopes. This epsilon here is called the error term
or error variable. It is sometimes represented by u.
It accounts for the variability in the dependent variable Y,
that is not explained by the regression model. The values of beta are not usually known but
are estimated from sample data as we have here.
The estimated model or equation is therefore presented as follows
with the coefficients taken from the first column of the output here.
Note that the dependent variable now has a hat
to indicate that it is estimated or predicted, and the error variable is removed since this
is an estimate. Now, the estimate of the model’s constant,
565.3, is the expected value of Y or Price, when all independent variables are zeros.
In this scenario, it does not make sense for House size and number of rooms to be zeros.
Therefore, the intercept has no practical meaning or does not make sense.
Next, we interpret the coefficients for two of the variables.
For house size, the estimated regression coefficient is represented as beta-hat1 or b1
with a value of 0.113. It means for every square foot increase in
house size, sale price increases on average by about $113.
That is, because sale price is in thousands, we have to multiply by the 0.113 by 1000
to obtain the $113. And in contrast to simple regression, we usually
add “holding other variables constant” in multiple regression.
This means that, this per square foot increase in expected price
is for homes with the same number of rooms, same age, and same distance.
Similarly, for distance, the regression coefficient is -15.975.
It means, for every additional mile, price decreases on average by about $16,000,
holding other variables constant. Next, we construct a 95% confidence interval
for the Age coefficient. Typically, confidence intervals for the coefficients
are included in regression software outputs. Nonetheless, to calculate them, the formula
is given as: the estimated Coefficient +/- the t-critical
value * the standard error of the coefficient.
With 95% confidence, alpha is 1 – 0.95 which gives 0.05.
Recall that the number of observations, n = 34
and the number of independent variables here, k = 4.
Thus, the degrees of freedom n – k – 1 = 29.
Since confidence intervals are always 2-tailed, we divide alpha by 2 to have 0.025 in each
tail. With df = 29, the t-critical value is 2.045.
So, for Age, the coefficient is 1.3849 +/- the critical value * the standard error for
Age. That is, we are 95% confident that the true
regression coefficient for Age lies between -3.8 and 6.5.
Next, we estimate the sale price of a house with this given information.
Here again is the estimated regression equation. We want to point-predict sales for this house
by substituting the values in the equation. The house size is 1900 square feet and there
are 3 rooms. It is 96-months old, but note that Age is
in years and not months. So, convert the 96 months to years by dividing
by 12. And that gives 8 years.
The distance is in miles, so we simply enter 20 here.
And that is approximately 780. Since price is in thousands dollars, this
is $780,000.
Next, we move on to testing the significance of each regression coefficient.
For each independent variable Xi, the null hypothesis will be that the regression coefficient
is equal to zero. That is, not significant.
And the alternative will be that beta_i is not equal to zero,
meaning the coefficient is significant in the model.
That is, the corresponding X variable is linearly related to Y.
We are testing these at alpha = 0.05. As we saw earlier, the degrees of freedom
is 29, and the two-tailed critical value is 2.045.
In essence, we will reject the null hypothesis if the t-statistic is less than -2.045 or
greater than +2.045. In other words, if the absolute value of t
is greater than 2.045, we reject the null hypothesis.
Note that the t-stat here is calculated as Coefficient/(Standard Error)
Also note that, since our output here contains p-values,
we really do not need to use the critical region approach.
We could simply use the P-value rule. That is, we reject the null hypothesis (or
the result is significant) if the P-value is less than our alpha of 0.05.
Now, for the first independent variable, House Size, defined as X1 here,
we will use beta 1 in the hypotheses. The test statistic is 3.444 which is greater
than 2.045, so we reject the null hypothesis and conclude
that the parameter for house size is significant in the model
or we have enough evidence to conclude that House Size is linearly related to Price.
You can also see that the p-value is less than our alpha of 0.05
which corroborates our decision to reject the null hypothesis.
For Number of rooms, represented by X2, the parameter is beta2.
The test statistic is 6.268 which is greater than 2.045 so we reject the null hypothesis.
The p-value here is 7.66 times 10 to the power of -7
which confirms our decision. So, we also have enough evidence to conclude
that number of bedrooms is linearly related to Price.
Next for Age, with parameter beta 3. The t-stat is 0.535 which is not greater than
2.045, so we fail to reject the null hypothesis.
The P-value is also greater than 0.05 to confirm that.
So, we conclude that Age is not significant in predicting Price.
Therefore, we could drop Age from the model
For distance, suppose we decide to test if Price decreases as Distance increases.
We will need a one-tailed test. That is, a lower-tailed (or left-tailed) test
because “price decreasing as distance increases” indicates a negative relationship.
The alternative hypothesis will therefore be that beta4 is less than 0.
At alpha = 0.05 and degrees of freedom = 29, the one-tailed t critical value is 1.699.
And since this is a left-tailed test, the left-tailed critical value will be -1.699.
The test statistic is -8.355 which is less than -1.699.
Note that the P-value here is a two-tailed p-value.
So, for this one-tailed test, we divide the p-value by 2.
Either way, the p-value is less than 0.05. We therefore reject the null hypothesis
and we have overwhelming evidence to conclude that price decreases as distance increases.
And that’s it for this video. Thanks for watching.