Let me start by saying that it is pretty unlikely that you will be calculating too many regression equations by hand.
However, doing an example by hand is a great way to really understand regression.
I would recommend working through this simple example first, and
then you can use SPSS or some other statistics software in the future.
We will begin with the assumptions and the necessary settings for hypothesis testing,
and then I will show you a simple example of using the regression formulas.
This will get you acquainted with the regression procedure.
Simple linear regression is used to predict the value of an output variable from a predictor variable
Here are the assumptions for linear regression.
First, we are doing simple linear regression which means that we are using only one independent variable or predictor
However, we can also do multiple
regression using multiple predictors.
The independent variable always goes on the x axis.
The independent and dependent variables must both be
quantitative at the scale level
Likert survey scales can be used if the questions had five or more item options
There are forms of regression that use
categorical outcome variables, but for simple linear regression, we will only use scale variables
Regression is particularly sensitive to outliers you should identify and remove extreme outliers.
To identify outliers in SPSS check the residual statistics box
after you run the regression. The bottom row is standardized residuals.
The minimum and maximum values should not exceed 3.29 or -3.29 respectively. If they don't,
you don't have outliers. You should have no extreme outliers
Delete or Windsorize severe outliers
so that the outlier is the same as the next highest non-outlier value in the data set.
When looking for multivariate outliers you will use a Mahalanobis test
The residual variance, called the residuals in the regression model, should be normally distributed.
To check, create a P-P plot of residual errors in SPSS, or use a Kolmogorov–Smirnov test.
The residual errors should be independent, not auto-correlated.
The predicted residuals should tell us nothing about the observed residuals.
To check, use a Durbin-Watson test for independence of errors
The Durbin-Watson statistic, which ranges from 0 to 4,
should be close to 2.
Values that are less than <1 or >3 or are a problem
There should be a linear relationship between the two variables in the correlation. If there's not, then the regression
will not work, and the model will likely be non-significant
To check for linearity, create a simple scatterplot of your predictor and outcome variables in SPSS.
If the scatter plot is not linear, if it curves or if it's cone-shaped, then use
Nonlinear regression or transform the data.
If we are doing a multiple regression with multiple predictors we will also check for collinearity
Collinearity occurs when two or more predictor variables are highly correlated.
To check, we would look at the output table for the coefficients. We look in the boxes for Collinearity Statistics for Tolerance and
VIF. The VIF should be less than 10. Tolerance should be greater than 0.1
to meet these assumptions for collinearity.
If the data do have multicollinearity,
ask your statistician whether you can use multiple regression.
Probably not, and we'll have to figure out another solution.
Homoscedasticity, the variance of the residual errors, should be the same across all values of your predictor or X.
To check, scatterplot the standardized residual errors against predicted errors in SPSS.
They should be more elliptical not cone-shaped or curved.
If the Scatter Plot shows heteroscedasticity, transform the data.
If certain of these assumptions are violated, you may not be able to use linear regression,
but you still have options. If the outcome variable is dichotomous,
coded 0 and 1, you can use a logistic regression.
If the predictor variable is categorical you can use a t-test or one-way Anova, both of which are
variations on that same general linear model
And you should also know that I'm going to talk to you about checking these assumptions when I show you the output
for the regression that we are going to run.
We are not going to be doing hypothesis testing with regression in the same way that we have previously.
However, we will be examining a model for statistical significance, so we will use all of the skills that we have developed along the way
The null hypothesis for regression is that the slope equals 0.
In other words the predictor variable does not
significantly predict the outcome variable,
so we might as well just predict with the mean.
You do not need to write out a full null and alternative hypothesis, instead you will state that
one variable will predict another variable in the Regression Model
The Alpha level is typically set to .05,
But it can be .01 or any other level that is chosen by you, the researcher.
The regression model will be tested with an F test,
an ANOVA, to determine if the model predicts the outcome better than simply predicting based on the mean.
The degrees of freedom are interpreted from the SPSS output.
So just assume that the regression test is alpha equals 0.05, two-tailed test.
When we are conducting a linear regression we are trying to get to this regression equation.
We start with the general
regression equation Y-hat = a + bX, and then we solve for a and b.
We will calculate b first using this equation,
and then we will compute the mean of X and the mean of Y
to plug them in with a b value to a second equation in order to calculate a.
Both a and b will then be placed into the general regression formula to make the regression equation
specific to our research question.
Using Excel makes the math a little easier.
Here are our data.
X is being used to predict Y.
We need two additional columns. The X^2 and XY
We do not need a Y^2 column for regression.
We begin by squaring each X value to create the X^2 columns,
then we work across rows multiplying each X value by its corresponding X value to get the XY column.
The next step is to create the sum for each column.
If you are doing these calculations in Excel, use the sum formula as we learned about previously.
Finally, we need the mean of X and the mean of Y which we get by dividing each sum by N.
N is 6 because there are 6 pairs of scores, so the mean of X is 5, the mean of Y is 6.5.
So now we are ready to calculate b.
Here is the formula.
As we have done so many times before, plug in the values that you just calculated into this formula and,
after that we use PEMDAS, the order of operations, to begin solving the formula.
In the numerator we do the multiplication. In the denominator we square the 30 to get 900,
and multiply 6 times 160.
Next we do the subtraction
leaving us with
96 over 60 which we divide, and it comes out to
1.6. So b = 1.6.
For step two, we can use this 1.6 and the means of X and Y to calculate "a".
The formula for a is actually the general formula for a straight line, only
we have done some algebra to get a on one side of the equation.
We plug in the means of X and Y,
and the 1.6 for b.
All that remains is to multiply b and x and
subtract from a, leaving us with a -1.5. So a = -1.50.
Now that we know a and b, in
Step Three we return to the general regression equation so that we can plug them in. We know that
a = -1.50 and B = 1.6
So when we plug them in we get Y-hat = -1.50 + 1.6X
Remember that adding a negative, such as -1.50, is the same as subtraction.
Step Four. Use a and b for prediction.
We now have our regression equation for our data,
so let's plug in some X values and predict the Y values.
The first thing we want to know is
What would be the predicted value of Y for an X equal to 4?
Well that will be simple enough.
We plug in 4 where we have an X in our regression equation
Multiply 4 times 1.6 and subtract 1.50.
The predicted value is 4.9.
So if someone has a X score of 4,
we predict that their Y score will be 4.9. That is our best estimate.
So now you try.
Predict Y if X = 5.
Pause the video while you solve it.
The answer is 6.5.
Now predict Y if X = 7. Again, pause the video while you work
The answer is
9.7.
Great! Now that you see how regression works, let's use a larger data set and solve the regression equation using SPSS.