Hello, I am Tom Reader, and in this video I am going to explore the difference between
fixed and random effects in statistical analysis.
You need to understand this if you are going
to analyse your data correctly, and avoid the terrible statistical sin of pseudoreplication.
I'm going to assume that you have a working knowledge of the basic statistical tests which
are taught in most introductory stats courses at University, and that you understand basic
stats jargon like "factor" and "dependent variable".
First-up, I should explain that all of the statistical tests you've encountered so
far - t-test, analysis of variance, regression - they all use a mathematical model to describe
how the dependent variable (the response variable) is influenced by the effects in which you
are interested, which are caused by the independent variables, which include both factors and
covariates.
You may not have seen models written down, but you use them every time you run
a test, and they represent your hypothesis about how the world works.
Now it turns out that the way that we define the model for any given situation depends
on what we think the values of the independent variables mean. Much of the time, we are implicitly
assuming that the values of an independent variable are "fixed", and that they represent
the entire population of values in which we are interested.
So if the independent variable is a factor, for example, we study the factor levels which
we consider to be interesting; there are no other levels to study, or we are just not
interested in other levels that there might be.
For example, we might be studying the health effects of smoking. Let's say we measure
the blood pressure of smokers and non-smokers. Since everybody falls into one of those two
categories - you either smoke or you don't - there are no other categories to worry about.
Alternatively, we might be comparing acceleration
among models of supercar, and we choose the two most popular models to assess, because
we want to know specifically which of these is quickest.
Although there are other models
of supercar, we might be happy to ignore them, because they are not the focus of our study.
In these situations, we are dealing with what is called a fixed factor or fixed effect.
Sometimes, however, it is more appropriate to assume that the values of an independent
variable are drawn at random from a larger population of possibilities.
For example, rather than setting out to study two particular models of supercar, I might
be interested in variation in acceleration among models in general.
Perhaps I want to
test the idea that variation among models is relatively small compared with the
variation caused by differences in the smoothness of gear changes, or the reaction speeds of
drivers.
In this case, I might select a sample of car
models to be representative of the wider population of models which exist.
If so, as long as I
have replicate measurements of acceleration from each one, the models represent different
levels of what we call a random factor or a random effect.
Now this might sound like we are dancing on the head of a pin - surely the distinction
between fixed and random doesn't really matter?
Well, it does matter, for two big
reasons.
First, the mathematical models for the two
different types of effect - fixed and random - are different, and the wrong model can
lead to the wrong conclusion.
Secondly, and perhaps more importantly, random
effects are often overlooked altogether - people simply don't realise that they could be
affecting their results. This can lead to the statistical sin of pseudoreplication.
Most basic stats courses and text-books, and most of the simple off-the-shelf tests you
know about, assume that all the effects that you are studying are fixed. But there are
potential random effects all over the place, and you need to know how to spot them, and
how to deal with them.
So, how do you spot a random effect? Well,
most of the effects in which you are interested are going to be fixed. If the main purpose
of your study is to test the effect of a factor or a covariate on your dependent variable,
it's probably a fixed effect that you are looking at. Even when you are sampling factor
levels from a larger population, your choice is probably highly non-random: you select
the factor levels in which you are particularly interested.
For example, in our supercar study, we are probably more interested in the acceleration
of specific models of car - is the Ferrari or the Lamborghini the fastest - rather
than the general pattern of variation among models.
So fixed effects are not too tricky. But random effects are sneaky, and they crop up when
you least expect them. Remember that study looking at the health effects of smoking?
You might decide to take five repeat blood pressure measurements at hourly intervals
from each participant, to help deal with measurement error, and the fact that blood pressure measurements
fluctuate quite a lot over time.
That's fine, but each group of five measurements
comes from the same participant, so they are not statistically independent. You've introduced
the possibility of a random effect of participant ID.
Your participants are probably a random selection from a large study population, and you aren't
really interested in whether one particular patient has higher blood pressure than another.
But you need to account for the fact that the repeated measures from each person will
be correlated, because some patients have naturally lower blood pressure than others,
for reasons which have nothing to do with smoking.
If you don't deal with the random effect, you will appear to have many more independent
replicates than you really have - in other words your study will be pseudoreplicated.
Ok then, so in this case perhaps I abandon the idea of repeated measures altogether because
it complicates things too much.
But what if half of my participants have their blood pressure
measured by one nurse, and half by another? Every nurse will vary slightly in how they
take the measurements, and although I am not interested in comparing the performance of
particular nurses, I have still got a potential random effect of the identity of the nurse.
If one nurse deals with the participants who are smokers, and one deals with the non-smokers,
my study has only got pseudoreplication - there is only one true replicate, one nurse per
participant group, and any difference between the groups could arise purely because of that
random effect of nurse.
This would be a fatal flaw in my study.
From this example, hopefully you can see that random effects are really important, and you
have to deal with them or you risk your analysis being meaningless.
We've already seen that one way to remove random effects is simply not to repeatedly
sample the same level of that effect - if every measurement comes from a different level
of the random effect, then effectively each observation is statistically independent,
and you don't have any pseudoreplication.
But defining what a "level" is can be
tricky. The levels are often recognisable as different individuals (as in our blood
pressure example, individual participants), but that's not always the case.
And anyway, as we saw in the example, there may be very good reasons why we want to take
multiple measurements from the same individual.
The simplest alternative to the presence of
a possible random effect is to average across the data within each level. So in our example,
if you have five blood pressure measurements from each person, you could take the average,
and hey presto, you have a single estimate of that person's blood pressure, and the
random effect has disappeared.
But this is throwing away a lot of data.
A better alternative is to model the random effect properly in your analysis.
To do this,
you are probably going to have to venture beyond the simple statistical tests which
are taught basic stats courses. In particular, you are going to want to find out about "mixed
effects models", which allow you to model a mixture of fixed and random effects. But
that's a topic for another day!