Now that we know what the regression equation is,
let's learn how to use a regression equation for prediction.
Science can do better than a crystal ball.
We use the X variable to predict Y-hat, by the way, some books use the symbol called
Y' (Y-prime) that is exactly the same thing as Y-hat just easier to create on a computer keyboard.
Since the X is used to make predictions, it is the predictor variable, also known as the
explanatory variable, regressor variable, or independent variable.
The score being predicted for the Y variable is the outcome variable, also known as the criterion
variable, target variable, or dependent variable.
For example, SAT score is the predictor variable.
College grades are the criterion variable.
Scores on the job pre-test are predictor variables. Job performance is the outcome variable.
We use X to predict Y.
Although you can use regression equations for prediction, the predicted Y-hat value
is not perfect.
Unless, of course, your correlation is a perfect +1 or -1.
There will always be some error, or residual variance, in your predictions.
The amount of residual variance, error, will be related to the strength of the correlation.
Stronger correlations equal less error.
We use the Method of Least Squares to create the line of best fit.
The line of best fit is the line with the least amount of error in the sum of squares.
Method of Least Squares.
The line of best fit represents the improvement of the predicted model over simply predicting
with the mean.
So think of all of the variance in the model as a pie.
A variance pie.
Now the whole pie represents all of the variability in your data.
The total variance will be made up of variance that you can explain by your general linear
model, but some of the total variance is going to be due to residual error, or residual variance.
Now ideally, you want your model to explain all of the variability in the data, so you
would get the whole pie.
But in the real world, just like having a little brother, somebody got to the variance
pie before you did.
Somebody called Error, and little brother Error ate up part of the variance pie.
Bummer.
But if you're lucky, if your X is a good predictor then your model of a straight line will predict
a large part of the variance in the model.
That would leave only a small part of unexplained variance for that pesky error.
Now the thing to keep in mind here is that you want your part of the pie, the part you
get to eat, to be much larger than the part that error gets to eat.
If you each get half that's not good enough.
You want all of the variance explained that you can possibly get.
So to make this more mathematical, consider that the total amount of variance in the data,
the sum of squares total, comprises variance that you can explain (that's your part of the pie)
and the variance that you cannot explain (the part of the variance pie that
belongs to Error).
Now to be clear, the actual measure of variance is the mean square, but remember that variance
is the sum of squares divided by N (or in this case N-1, which is the degrees of freedom).
So both sum of squares and mean square are measures of variability in the data.
Mean squares equals the sum of squares divided by degrees of freedom.
So ultimately, we are going to divide the mean square variance of your model by the
mean square variance residual to get an F ratio.
Just like with ANOVA.
So if the sum of squares for the model, the SSm, is the same size as the sum of squares
residual, SSr, then you divide two things that are equal and you end up with a value of 1.
And that would mean that your model does not explain more of the total variance than error does.
Your ANOVA will equal 1 and be non-significant.
You might as well have used the mean for prediction.
Your predictors did not improve the predictive model.
Now for a contrast, consider the second model.
Now in this model the total variance is still made up of all the variability in the data,
however, this time the variance explained by the model, the SSm, is much larger than
the variance eaten up by error.
So when you do the division, the F value will be much larger than 1, and the model will
be significant.
You can predict more with your model than you could by just using the mean.
And keep in mind that the r-squared value that we're going to talk about in a moment
is the sum of squares for your model divided by the total sum of squares.
Ok, before we move on, let me warn you about using the regression line to predict outside
of the range of data.
Notice the X-axis for this scatterplot.
What is the range of these data?
The data range from age 4 to age 12.
So if we were to predict heights less than 4 or greater than 12, we would be predicting
outside of the range.
Why is that a problem?
Suppose that in February, I tell you that "if it keeps snowing like this, we will have
16 feet of snow on the ground by the 4th of July."
I'm making an assumption about the weather right now that may or may not be true in 6 months.
Predicting values not represented among your original X values is predicting outside of the range.
You should not use the regression equation to predict Y-hat values beyond the range of
available X values in the original data.
The regression equation was constructed based on a specific set of X values.
Those X values have a range.
If the X values range from 0 to 10, then the regression equation should only be used to
predict Y-hat values using X values between 0 and 10.
Here's what could happen otherwise. If I use my data for children to predict heights later
in life, I would predict that when you are 40 years old, you will be 12'8" tall.
Now that's not true because we know that growth slows down and stops in adolescence.
So the child data cannot be used to predict adult heights.
As this XKCD comic points out, extrapolation as a hobby ends in absurdity.
But you may ask, "What if I try to predict just a little outside of the range of data?
How far can I go?"
Assuming the range of the data values was 0 to 10, could I predict X = 11?
Well, probably.
It would be a little speculative, but perhaps ok.
How about predicting an X of 21?
No.
Nope, that prediction is highly speculative and likely invalid.
The further outside the range that you go, the more speculative your predictions become.
Once you have a predictive model, you can square the correlation value to create an
effect size.
The r^2 is the percent of variability in the Y variable explained by variability in the X variable.
For example, if the r correlation is .5, then the r^2 would be .25 which means that 25%
of the variability in doctor visits was explained by number of apples eaten.
See the correlation of determination in the correlation notes for more information.
You can also refer to the coefficient of determination in the correlation notes for more information.
We are now ready to learn how to create a regression equation by hand and then using SPSS.