When an ANOVA is significant, you need a follow-up test to tell which group is different from which other groups.
For our statistically significant Wizard of Oz ANOVA
we will use Tukey's Honestly Significant Difference post hoc test
to figure out which regions of Oz have more wicked witches than which other regions.
Even if you are using SPSS or some other statistical software,
this video will help you understand what the Tukey's HSD is and how it works, and
remember, it's Tukey not turkey.
Tukey's Honestly Significant Difference,
or HSD, is the least amount that means must vary from each other to be truly different.
The honestly significant difference can be calculated with the information contained in the ANOVA summary table,
the same information that we would use for calculating effect size, by the way.
If the means differ by more than the HSD value, then they are honestly significantly different.
That word, honestly, implies that the means may look different and yet, not actually be different. For example,
I have assisted researchers with projects in which two groups might differ when using a t-test,
yet in the context of the full ANOVA, the means were not honestly different. This is not a mistake.
This is actually avoiding a mistake.
When small t tests are significant, but the ANOVA is not, the mistake was a Type I error in the t-test.
We avoided that mistake by using a post-hoc that showed that the means were not honestly significantly different.
Calculating the HSD
requires three steps. Step one: Find q
To calculate an honestly significant difference, we will first find the constant "q", and
then we will plug that into the formula that you see here. Where
HSD equals honestly significant difference,
q is a constant
found in the studentized ranged q table, coming up next,
MSW is the mean square within,
which you can find in the ANOVA summary table.
n sub k is the sample size for one treatment group and
the sample sizes are assumed to be equal when using Tukey's Honestly Significant Difference.
Q is the constant found in the studentized range q table. Let's go to that table now.
To find q, you need the degrees of freedom within and the number of treatments
Notice that this is the actual number of treatments, or in our case, number of regions.
This is not the degrees of freedom between or k minus one.
Use the same alpha equals .05 significance level as the overall F test,
look up q in the studentized range table using K equals 4treatments and
degrees of freedom within of 16
q = 4.05
As with other critical value degrees of freedom, if your degrees of freedom within is between two numbers,
use the next lower number. This will correspond to a larger q value, or in this case, a larger F max value.
Step Two: Find Tukey's Honestly Significant Difference
For this, we will return to our ANOVA summary table.
We need the mean square within,
which we can find right there.
The mean Square within equals 1.20,
so now we know q and the mean square within.
The last thing that we need is n sub k,
which is the number of participants in each group.
Again, assuming that each group has the same number of participants, so therefore q equals 4.05.
Mean square within is 1.20 and
n sub k is 5. There were five munchkins in each group.
So now we will plug in each of those numbers and solve.
The honestly significant difference is
1.98
That means that the minimum difference between means must be more than 1.98
for them to be honestly significant different.
Which brings us to
Step Three: Compare the means
We must compare each mean to the others.
Means that differ by more than 1.98 are statistically
significantly different, and remember, that 1.98 is specific to this example. Other ANOVAs will have a different
HSD value.
Means that differ by less than 1.98 are not honestly significantly different.
So here are the means for each group.
We could do some subtracting, for instance, the difference between group 1 and group 2 is
1.4 - 1.0, or 0.4.
Not honestly different.
But the difference between group 1, mean of 1, and group 3, mean of 3.6
Is 2.6, that is honestly different.
This table shows how all of the means differ. Mean differences greater than 1.98 in bold are
statistically significant.
There are four differences out of six.
So we would conclude that means one and two are different than three and four.
but one and two are the same as each other and three and four are the same as each other.
So essentially, we have two larger groups: North and South
vs. East and West
That means that there are more wicked witch complaints in the east and the west.
Perhaps all of the good witches live in the north and the south or
perhaps, if you've seen the musical Wicked,
perhaps we're judging too hastily.
Regardless, we need to summarize our ANOVA.
Step four was to calculate the statistics.
We had an F ratio with 3 and 16 degrees of freedom equal to 10.49,
P < .05. It was statistically significant, so
Step 5: Make the Decision
there was a statistically significant difference
among the 4 regions on wicked witch complaints F(3 and 16 degrees of freedom) = 10.49, p < .05.
Post hoc tests revealed north and south were lower than east and west.