The ANOVA test is analysis of variance.
Variance will exist between groups,
especially when one group is different
than another. But there is also variance
within each group; random variation that
is just noise in the model. ANOVA
tells us if there is more signal than
noise in that variance. Before we go
further with ANOVA, let's learn some
important definitions for the analysis
of variance. An independent variable in
ANOVA is called a "factor". In fact, you
will sometimes hear this called a
"factorial analysis of variance." One
factor can have multiple levels or
conditions. Each condition of the
independent variable is called a "level"
or "treatment". If the factor is exercise,
then the levels might be: control group,
no exercise, 2.5 hours of exercise per
week, and 5 hours of exercise per week. If
the factor was something about sleep,
then the levels might be: low, medium, and
high levels of sleep. The symbol for the
number of levels in a factor is k. The
number of subjects in the factor is n,
just like we've always used for sample
size. Differences produced by the
independent variable, such as the amount
of exercise, or the quantity of sleep, are
called "treatment effects" and are
revealed as the variance between. We do
hypothesis testing with ANOVA in two
steps. First we do the ANOVA, and then
- if we find statistically significant
results - we do a follow-up test. The
reason for these two steps is that ANOVA
is an omnibus test. "Omnibus" is something
that comprises several items. An omnibus
test comprises several comparisons. ANOVA
tests for overall differences among
multiple groups. ANOVA tells
that differences are here somewhere, but
it will not tell us where those
differences are. It tells us THAT group
means differ, but not WHICH means differ.
To know which means differ, we must do a
post hoc test. "Post hoc" is Latin meaning
"after." We will only do a post hoc test if
the ANOVA is significant. (More on
that in a moment.) ANOVA is an F-test and
it uses the F ratio, so called because it
is named after British statistician and
geneticist, Sir Ronald Fisher. This is a
picture of a plaque at his house. The F
ratio is a measure of the variance
between treatments (in the numerator),
divided by the variance within
treatments (in the denominator).
A significant F ratio (or F value) tells
us that the experiment was successful.
The group means differ, but you still
need to follow up with a post-hoc.
When comparing groups, there are two sources
of variance: between and within. Variance
between groups is due to actual
treatment effects, plus differences due
to chance or error. If the high exercise
group ended up being healthier than the
no exercise control group, then there
would be a difference between groups.
Variance within groups is due only to
chance or error. In the low sleep group,
each person in the group would sleep for
different amounts of time, and that is
variability within the low sleep group.
The F ratio is the variance between
treatments divided by the variance within
treatments. Because variance is a
squared value, it is always positive. And
because the F ratio divides two
variances that are always positive, the F
ratio itself will always be positive. The
F ratio will tend to
stack up around a value of 1.00, with
fewer values as you move away from 1.00
to the right. Why do the value stack
up around 1.00? Because if there's no
effect from a treatment, then the only
variability will be due to random chance.
When you divide the F ratio, you would
be dividing random chance between by
random chance within. In other words, any
time you divide something by itself, the
value will be 1.00. So, let's look at the
f-distribution. When all of the
variability in your model is random
noise, then dividing between by within
will cancel out to 1.00. But when you
start getting more variability between
the groups, then the numerator will grow
larger and the ratio will start creeping
out to the right, greater than 1.00. Values
far from 1.00, beyond an F-critical-value,
(that can be determined from the F table),
will be "statistically significantly
different." In the shaded area the area
beyond alpha of .05, you would
reject the null hypothesis that all of
the groups are the same. Here are some
examples of variance between and within.
Shopping for jeans: the price of the
jeans varies between stores, but also
within a single store. Certain stores are
more expensive and others are less
expensive. That's variance between the
stores. But within any given store, the
price of the jeans will not be exactly
the same. That price will differ for
different pairs of jeans. That's variance within.
Crops: the growth of crops varies
between plots (or fields) and also within
the plot, so not all of the corn plants
in a given plot will be exactly the same
height. That's variance within the plot.
The corn in the field that got
fertilizer would grow better than the
corn that got no fertilizer, and that
would be variance between the plots.
How about the temperature in your house?
Are all of the rooms in your house the
same temperature?
The temperature varies between rooms, but
it also varies within a room. The
basement would be cooler than the attic
or the living room. That's variance
between the rooms. But within the living
room, the temperature within the room
will vary from near the sunny picture
window, to over toward the middle of the
room, where it's cooler. So your dog might
find the warmest spot of sunshine to
take a nap in front of the picture
window, because it's just a little warmer
there than elsewhere in the room. There's
variation in the temperature within your
room. Radio station quality: do you
like all of the songs that are played on
your favorite radio station? The quality
of the songs is going to vary between
stations, but also within the station
that you're listening to. So this is why
you have a favorite radio station,
because you prefer the songs on that
station to another type of station.
That's variance between. But even within
your favorite radio station, there are
some songs you like better than others.
That's variance within. If the omnibus
ANOVA test is significant and
differences exist, you follow up with a
post-hoc test. Post-hoc tests compare
each individual mean against the other
means. The post hoc is necessary
whenever you reject the null hypothesis
and there are three or more groups. You
would not do a post hoc test if the
ANOVA was not significant, because there
are no differences to find. Nor would you
need a post-hoc when you're comparing
only two groups. If a t-test is
significant, you can just look at the two
group means and see which one is higher.
But when you have three or more groups,
you need a way to tell which means are
different from which other means.
Post-hoc tests use a stricter criterion
to accept an effect as significant, hence
they control for "family-wise error rate".
They control for the increased likelihood
of Type I errors that occurs when
you do multiple comparisons on the same
data with a t-test. The simplest example
of a post-hoc is the Bonferroni method,
in which you divide alpha by the number
of groups. If you have three groups,
with an alpha of .05, then your
alpha level for the post-hoc is .05
divided by 3, or .0174.
For four comparisons, it is 0.0125.
For five comparisons with an alpha of .05,
then your alpha level for the post-hoc
is 0.01. The Bonferroni method
is simple, but it's also excessively
conservative...and that's bad, too.
It increases the likelihood of making Type II
errors, missing effects that really
exist, because it tightens up the
decision-making criteria so much. Another
alternative, however, is Tukey's Honestly
Significant Difference post hoc test.
Tukey's honestly significant difference
test tells us how far apart the means
must be in order to be truly significant.
This is the post-hoc tests that we will
use in this example. It's a very good
test, but it requires that the sample
sizes be equal. If your sample sizes are
not equal, you still have plenty of
options. SPSS has 18 different types of
post hoc tests. Each one is used when
certain criteria exist, or when certain
assumptions are unmet in the ANOVA.
We will use Tukey's Honestly Significant
Difference post-hoc tests for this
example...and remember it's pronounced
"Tukey" not "turkey." Let's use a simple
comparison to illustrate the elements of
the ANOVA. In this example, we're asking
the question, "Does type of diet affect
weight loss?" We have 20 randomly selected
people who have been randomly assigned
to four groups. Each group uses a
different diet approach. So, here we have
one factor (which is diet); the one factor
has four treatments or
the type of diet: willpower, Paleo diet,
Mediterranean, and vegan. Any differences
in the scores within a group is variance
within. Variance within is due to chance.
People lose different amounts of weight
within each type of diet. The differences
in the overall amount of weight loss for
each group represents variance between.
Variance between the groups is accounted
for by the random variation within, plus
variance due to the treatment. Another
way to think about this would be to
consider that because all of the scores
differ from each other, we have a total
amount of variance within the model.
That total amount of variance is made up of
variance within, which is caused by
chance within each group, and that total
variance also comprises variance between
the scores for each group. This is the
variance between. We will divide the
variance between, by the variance within,
canceling out the element of chance,
leaving us with a ratio that reflects
the size of the treatment effect. That is
our F ratio (or F value). And so with that
introduction as to why we are doing
ANOVA, we are ready to talk about the
assumptions and the hypotheses for the
one-way ANOVA.