We are ready to learn about how to
compare more than two means, or how to
do hypothesis testing when we have more
than two groups. The test we will use is
called ANOVA or a one-way between
subjects analysis of variance. Now that
you've learned about t tests and how
they can be used to compare the means of
samples, you may wonder why would we ever
need anything else? Anytime we need to
compare means, we'll use a t-test! But
hold up, there's something more you need
to know. T tests are fine when you're
comparing two means, but if we have three
or more groups, the t tests have a flaw.
and here is why we need more than just
t-tests. A t-test can be used to compare
two independent means. Remember "Tea for
Two"? But the t-test has some limitations.
The t-test is limited to only one
independent variable with two groups, and
only one dependent variable. So if we
want to compare multiple independent
variables, or multiple dependent
variables, we're out of luck. And the
t-test can only compare two means. But
often we would like to compare means
from three or more groups. But then
again, if we want to compare three means,
why not just use three t tests? And
here's where the problem comes in. We
cannot use multiple t tests because
using multiple tests on the same data
causes the Alpha level to skyrocket and
this means that we are more likely to
make Type I errors, or to find
erroneously statistically significant
results where no result truly exists. So
let me illustrate this problem with an
example about exercise. The Mayo Clinic
recommends exercising 2.5 hours per week
with moderate aerobic activity for your
health. Well, what if we wanted to compare
people who exercise 2.5 hours per week
with a control group that does not
exercise at all?
But while we are at it, why not add
another group that exercises for five
hours per week, doubling the recommended
time. Will that double the health benefit?
Now, this might seem like a good time to
use our T tests. We have three groups: A, B,
and C, so we could run one T test
comparing group A and B, and then we
could run a second T test comparing
groups B and C, and then we would run a
third T test comparing groups A and C.
But with our alpha level set at .05,
that means that each
comparison has a 5% error rate.
So what's the problem? Well, let's examine
this using the analogy of a lottery.
Imagine that you are going to play the
lottery. Now, normally I would not
recommend this because the lottery is a
tax on people who are bad at math and
don't understand probability, but this is
a special lottery: only 20 tickets are
going to be sold for ten dollars each,
and the prize is one thousand dollars.
This is a good lottery to play! So what
would you do? Well, if you were smart, you
would buy all 20 tickets. Why is that?
In this analogy, every lottery ticket has a
1 in 20 chance (a 5% chance) of
winning. The probability of any ticket
winning is p = .05, but even
though any one ticket has a 5%
chance of winning, if you hold all of
them, you have a 100% chance
of winning. If you hold all 20 tickets,
you will spend two hundred dollars but
one of them will certainly be a winner
and you will win one thousand dollars,
although you will not know ahead of time
which one will be the winner. On the
other hand, you won't really care which
one was the winner, but that's not the
point. The point is that this same logic
applies with hypothesis testing. Every T
test has a p = .05
chance of being a Type I error,
of erroneously finding a difference
where none exists. For any single t-test,
you have a five percent chance of being
wrong. But if you keep running t tests on
the same data, one of them is going to be
a Type I error, just like that lottery.
And you don't have to run 20 t-tests to
have this problem. For instance, if you
reject the null in just five groups, your
alpha level is functionally about a .40
One of your tests will identify
an effect that does not exist. One of
them will be a Type I error, but you will
not know which one was in error. Only now,
you will care, because one of your
published findings is going to be a lie
and you won't know which one. So the
solution is analysis of variance.
Analysis of variance is abbreviated as
ANOVA. Analysis of variance analyzes the
variances of the groups to assess
differences in means between the groups.
ANOVA is actually an extension of
regression, called the general linear
model, or GLM. In SPSS will be one of five
models called GLM1 through GLM5.
We will be talking about the GLM1
model. The GLM1 ANOVA model can be
used with one independent variable with
any number of conditions or levels. But
you have options! The general linear
model can be used with both independent
groups or related groups, called the
repeated measures ANOVA. General linear
model can be used with multiple
independent variables or multiple
dependent variables, called a MANOVA. The
GLM can include covariance as a control
called an ANCOVA or a MANCOVA with
multiple variables. The general linear
model is actually a special form of
regression, so in reality, everything that
we are doing with hypothesis testing
from Z tests, to
t-tests, to correlation, to ANOVA, is
actually all regression. It's all
regression under the hood. So how would
the ANOVA work with our now familiar
hypnosis example of hypothesis testing?
Using our research question of "Is recall
better under hypnosis or without?", we
would begin with an existing memory test.
We would randomly select participants
and then we would randomly assign
participants to one of three groups.
What might those groups be? The first
group of our independent variable would
be those who are hypnotized. We would
compare their memory scores to a second
group that was not hypnotized, and
perhaps a third group, who was drunk. The
dependent variable for all three groups
would be number of words recalled. In the
end, we could answer the question: "Do
hypnotized participants remember more
words than the unhypnotized or the drunk
participants?" If the ANOVA was
significant, then one of the groups was
different from at least one other group,
but we would need a post-hoc tests to
figure out which group differed from
which other groups.