Hey guys, in this video I'm going to go over how to do a Box-Cox transformation
and in particular, this video is mainly going to be a demo in how to do it in
R because it's pretty simple. But before I show you how to do it in R,
I just wanted to go over some of the background of what is a Box-Cox
transformation, and before I even get into that . . . a funny story about Box-Cox
transformation is that Box and Cox are two statisticians who--they came up with
this idea--the paper was first published in 1964 and literally the only reason
these people collaborated with each other was because their names sound
similar to one another and they thought it would be fun to write a paper because
their names are similar. And they basically came up with some great idea
that is better than anything I'll ever come up with so . . . that's a funny story
I guess (lol). Anyway, so this is basically the idea behind a Box-Cox transformation.
A transformation in general, when you do a simple or multiple regression, you have
certain model assumptions that you need-- that your model needs to hold true for
you to be able to do a regression correctly. You need normality of the
errors, for example, linearity, and homoscedasticity. Those are just some
assumptions, right? So sometimes when you run a model like this, your assumptions
are not met. The Box-Cox transformation is specifically supposed to handle non-
normality of the errors, but actually if you do a Box-Cox transformation a lot
of times, if you have other problems like non-linearity, it will fix that
as well and it will fix heteroscedasticity, but it was mainly--the
focus was to fix the violation of non-normality of errors if you have that
problem. A transformation in general: Maybe the true relationships between X
and Y--and let's say this is a simple linear model--maybe the relationship
between X and Y is not one-to-one, so maybe the true relationship is X affects
Y squared and not Y. If you try to regress a model that's X on Y, your model
assumptions might not look very good and your model might not be very informative.
If you have an a priori, you know, hypothesis or
theory of the relationship between X and Y, then you can go ahead and fill in
whatever, you know, the relationship you think it is. And this would be considered
like a variance-stabilizing transformation, but a lot of times you
don't know the true relationship between X and Y, and also in their paper, they
focus on transforming the Y, so the dependent variable, but it's also--you can
do Box-Cox transformation or just transformations in general on the
independent variables or the covariates as well, so it's accurate either way. But
let's say that you don't know the true relationship between X and Y. Then you
can do a Box-Cox transformation. Let's go over what that is. Basically what they
thought of--it's a very simple but very-- incredibly useful idea--was they just
said take Y to an unknown parameter lambda equal to X*Beta plus epsilon and
then use a maximum likelihood estimation to estimate what the relationship is
between X and Y. And then you can plug in after you estimate your parameter lambda,
then you can re-estimate a model with the correct lambda and your model
assumptions should be improved. So this works--I haven't really read the whole
original paper--but I think it works very well for when lambda is between negative
2 and 2, or maybe negative 3 and 3, it's not supposed to be for some huge
transformation. You should never get like Y to the 1000 or 100 or something.
It's really meant to be for small transformations, which is what you would
expect for a transformation anyway. And also, another thing I just want to
mention before we get into the R demo is that if your--so if your lambda that
you estimate the MLE for ends up being around 1, then your relationship is just
the standard Y equals X*Beta plus epsilon. And as well if it's 0, then that
transformation is going to be log y equals X*Beta plus epsilon.
And that's because you have this relationship here, where lambda can't
equal 0 on the bottom because then you have a zero on a
denominator. But if you take the limit of this, it has been shown that this is
equal to log Y. So if you get that the MLE maximum likelihood estimator for
lambda is zero, then just know that that means that they want you to transform it
using a log transformation. That was just some background information and now
let me switch over to R to do a quick little demo . . . Okay so here's a quick
little code that has a demo for how to do Box-Cox transformation. The dataset
that I'm going to analyze is from the faraway package and the package that
actually has how to do Box-Cox transformation very simply is called
MASS. So I'm going to load in both those packages. I already have them installed.
This demo is basically from some old homework problem, so it's maybe not the
most interesting dataset but it was 'Butterfat content of milk by breed' and
it has two covariates, breed and age. And this is categorical data, so you can do a
Box-Cox transformation on categorical or continuous data. It works either way.
You can use an 'lm' or 'aov' function to use the Box-Cox function in MASS.
I went ahead and I'm gonna fit a model without doing a transformation first,
then I'm going to plot the model diagnostics, then I'm going to do a
Box-Cox transformation, and plot the model diagnostics for the transformed
model. So this is the original full model. It's butter fat is regressed on breed,
age, and the interaction of breed and age. These are categorical variables plus
the interaction. So that's what I did in this line, then 'plot' plots some model
diagnostics. Let me increase the size of them and this will plot 4 model
diagnostic plots. The 'Residuals vs. Fitted' plot will show you linearity.
It's not that bad. The second one will show you the normality of the errors.
This is a QQ plot. Here the normality assumption is not great, especially at
the tails and especially at the upper tail. But the lower tail as
well, it seems to be going off from the straight line, which represents normality
of the errors. So that's not great. If we look at the 'Scale-Location' plot this
shows whether you have homoscedasticity. This is not that bad, but it's
supposed to be--this red trend line should be straight. We see it's not
completely straight, especially in the beginning. That's a little bit different
from those at the end, but it's not too bad really. And this is just showing if
you have any residuals or outliers.
It's not too bad of a model fit,
except for, really, the normality of the errors. Now let's go back to the code and do the
Box-Cox transformation to see what Box-Cox thinks is the relationship
between butterfat and breed and age. I already loaded in the MASS package and
to run the Box-Cox transformation, you just put in 'boxcox' and then the name of
the model. I think that's literally the only
required parameter that you have to input. Lambda, what you see here, is
telling it how to--it's going to output a plot--so actually let me show you what
the plot looks like with the only required parameter. And you see here
it plots basically a maximum likelihood plot for the values of lambda and the
default is, it shows you lambda from negative 2 to 2. So when it plots it
plots it automatically with a 95% confidence interval. And so if you add
the code that I had previously, which was lambda sequence from negative 3 to 3,
that will plot from negative 3 to 3 the lambda. It will show you the full 95%
confidence interval because here it's getting a little bit cutoff.
If I re-plot that now you can see the upper bound--or sorry--the lower bound,
the upper bound, and the actual MLE. So the MLE, if you just look at it
visually, it gives you an estimate between -1 and -2.
It's telling you the relationship is not exactly linear. And if you wanted to, you
can also check the actual value that it's giving you exactly, using this code.
And so that gives me that the best lambda
estimation is -1.42 repeating. Traditionally there are basically two
ways that you can do a transformation: You can plug in the exact value of
lambda that it gives you, but it's also very common in the literature for people
to round. If you see here, it's telling you basically that you can use negative
2 if you wanted to because it's in the 95% confidence interval or you can use
negative 1 if you wanted to as well because that's also very safely within
the confidence interval. Maybe if you are--if you're running this analysis on
your own data, you can think: Okay, it makes sense to me for whatever reason
that the relationship is an inverse and the number itself is closer to negative
1 then to negative 2. So I'm just using the kind of the tradition of
rounding it to the nearest whole number. The next model I did is called 'full
model dot inverse' and instead of using butterfat, now I'm taking butterfat to
the -1. So I'm using it as an inverse and then I'm using the same
covariates breed, age, and interaction and so I'm going to run that model and then
I'm going to plot model diagnostics for the transformed model using butterfat as
an inverse. So let's look again at the four diagnostic plots. Now if you see
here, we have a very straight trend line, which is what we want. So this first
model, or this--yeah this first plot shows linearity. So we have great
linearity. It's even improved from the last time, even though the last time we
didn't have a non-linearity problem. Now the main problem, which was non-normality
of the errors has been improved a lot more. Now we see that the plot points
really do fall on the straight line that represents normality of the errors.
That has improved significantly. We see also that the trend line for
heteroscedasticity has improved. So this used to be kind of like that, which was
not too bad, but now the trend line is even straighter than it was before, which
represents homoscedasticity. So it has improved not only the non-normality
of errors, it's also improved linearity, and constant variance,
and this has pretty much stayed the same. So as you can see, the transformation is
really simple to do and it can very easily fix a lot of the model assumption
violations that you see in your model.
That's it for this video and thank you
guys for watching!!
*chiptune music plays as outro*