Hi! I am Mike Marin and in this video we'll introduce
how to check the validity of the assumptions made when fitting a Linear Regression Model.
While the assumption of a Linear Model are never perfectly met in reality, we must check
if there are reasonable enough assumption that we can work with them.
We will be working with the LungCapacity data that was introduced earlier in these series
of videos. I've already gone ahead and imported the data
into R and attached it. We will examine the assumptions for model
of the relationship between Age and Lung Capacity, with the Lung Capacity being our Y variable,
also known as the outcome or dependent variable. Recall that we can produce a Scatterplot of
Age versus Lung Capacity using the "plot" command.
Also recall in earlier videos we saw we can fit a Linear Regression Model using the "lm"
command. Here, I will fit a Linear Regression Model
predicting Lung Capacity using Age, and we will store it in an object MOD.
We also saw we can ask for the summary of the model
using the "summary" command. Here we can see estimates of the model Slope
and Intercept along with other summaries for the model.
Finally we saw we can add a regression line to this plot using the "abline" command.
Here we would like to add a regression line for our model.
You recall that regression line can be thought of as the predicted or fitted Y value which
is labeled Y hat ( ^). This can also be though of as the mean of
Y given X; this can be expressed as the intercept or
b-not plus the slope b-1 times X. The residuals or errors are labeled with an "e"
and are the difference between observed Y value and the predicted or fitted Y value.
The Standard deviation of these errors or residuals is called the Residual Standard
error in R and is presented in the model's summary.
This gives us an idea of the average or typical sized error or residuals.
The residual or error terms are very useful for checking the model assumptions as well
as other things about the model fit. You recall that a few assumptions are made
when fitting a linear regression model. The first assumption is that the Y values
or the errors are independent. The second assumption is that the Y values
can be expressed as a linear function of the X variables.
The third assumption is that the variation of observations around the regression
line is constant, this is also know as homoscedasticity. The fourth assumption is that for a given
value of X, the Y values or the error terms are normally distributed.
The first assumption requires knowledge of study design or data collection in order to
determine the validity of this assumption. The second, third and fourth assumption can
be checked by examining the model residuals or errors.
R has a set of built in regression diagnostic plots, four of these in total!
These can be produced be asking R to plot the model that we fit.
You should note here that this will produce the diagnostic plots not a plot of model fit
itself. We can see when submitting this command we
are asked to hit return to see the next plot. When we hit return we can see the first plot
here. This plot is known as a Residual plot.
Here the X-axis is the predicted or fitted Y values: the Y hats (^).
On the Y axis, is the residuals or errors. If the linearity assumption is met, we should
see no pattern here: the red line should be fairly flat.
If the variation is constant here, we should see no pattern: these points should look like
a cloud of points. We can take a look at the second plot; this
is known as a QQ plot or quantile quantile plot.
The Y-axis is the ordered, observed, standardized residuals.
On the X-axis is the ordered theoretical residuals. This is what we would expect the residuals
to be if the errors or the residuals are truly normally distributed.
If our Y values or errors or residual terms are normally distributed these points should
fall roughly on a diagonal line. The third plot here as well as the fourth
plot you see here are other plots that can help us to identify non-lineariteis, non constant
variance as well as other troublesome observations. If we would like to have all four of these
plots appear on one screen, we can change this using the "mfrow" command.
We would like to have the plotting screen split into 2 by 2,
allowing us to have 2 rows of plots and 2 columns of plots or 4 plots in total on one screen.
We can see now when we submit the plot model command, all 4 of these diagnostic plots appear
on one screen. Okay, let's go ahead and put this back to
having only one plot appear on the screen, and now let's take a look at how non-constant
variance will show up in a residual plot. I've already loaded the dataset where there
is increasing variance. Let's take a look at a scatterplot of X
versus Y to see this. Let's also go ahead and fit another regression
model, we will call it model2 predicting Y using X.
let's go ahead and add this regression line to our plot.
When we take a look at the plot here, we can see for larger X values the variation in Y
is larger or the errors or residuals become larger. Let's take a look at the regression diagnostic
plots for this model. First, we can take a look at the residual
plot. Here when looking at the residual plot, we
can see this sort of megaphone shape appearing. This suggests that variance is increasing:
larger predicted values are associated with larger errors or residuals.
We can also see the red line in the plot is fairly flat indicating the linearity assumption
is met. In our QQ plot we can see the data looks fairly
normal, and again we can see the increasing variance
showing up in this other diagnostic plot. Now, let's take a look at another set of
data I have loaded where there is a non-linear relationship.
Here we will take a look at a plot of XX versus YY.
We can see this non-linear relationship appearing, let's go ahead and fit a model, we call it
model3: that's a linear model predicting YY using XX.
and let's go ahead and add this line to our scatterplot.
Here we can clearly see there is a non linear relationship.
If we take a look at the diagnostic plots for this model, when looking at the residual
plot we can see the red line has this curved pattern showing up. there is clearly curvature
associate with the residual or error terms. Taking a look at the QQ plot, the data look
fairly normal, and once again we can see this non-linearity showing up in our other diagnostic
plots as well. You maybe wondering what is the use of making
these diagnostic plots if we can see the violations of the assumptions when we look at the scatterplots
themselves. Well, when fitting a multiple linear regression
model we will have multiple X variables. We are going to make the same set of assumptions
in a multiple linear regression model that we have in the simple linear regression models
except we are not able to produce scatterplots of the data when we have more than 3 variables.
These regression diagnostic plots will allow us to check the validity of the assumptions
we make when we have too many variables to be able to visualize the relationship between
them. In the next video in this series we'll talk
a bit more about examining the model fit.
For more Subscribe to marinstatslectures