You should report a measure of effect size whenever you report a statistical
test. Let's learn how to create an effect size for a one-way ANOVA, both by hand,
and in SPSS. Why do we need to report an effect size, when we already have a
statistically significant hypothesis test? And the answer is the difference
between statistical significance and practical significance. Because
statistical significance is a function of sample size, as n increases you will
find statistical significance even for small and relatively meaningless
differences. Big enough sample, everything will be statistically significantly
different, and many of those differences won't really matter in the real world.
Statistical significance means that the differences that you observed in the
means in your test was unlikely to occur if the null hypothesis was true. If the
groups were not different as you surmised, you would not get these
findings, but you did get these findings, so it is highly likely that the groups
were different after all. And that is a statement of probability, statistical
significance tells us that differences we found were unlikely to have occurred
by chance. Practical significance is a standardized measure of the difference
between your sample and the null. It removes the influence of sample size.
Practical significance means that the difference has clinical real-world
significance. It is a way of determining if statistically significant differences
really matter in the real world. Now there are two kinds of practical
significance. Strength of association and effect size. Strength of association is a
measure of the percentage of variance in the dependent variable accounted
for by the independent variable. In an ANOVA, that measurement is Eta squared.
Eta squared is equivalent to R squared and is interpreted in the same way. It
explains the percentage of variance in the dependent variable accounted for by
the independent variable. In our Munchkins example, 66% of the complaints
about wicked witches were explained by the region in which the munchkins lived.
That's a lot, but Eta squared tends to be biased in that it overestimates the true
effect size. Therefore, the strength of association most commonly reported for
in ANOVA is a partial Eta squared. Partial Eta squared is interpreted the
same as R squared. It is still the percentage of variance in the dependent
variable accounted for by the independent variable. However, partial Eta
squared is an unbiased correction to Eta squared.
When working by hand, partial eta squared can be calculated with the information
in the ANOVA summary table. It is especially useful for repeated measures
ANOVA, and multiple factor designs, and it can be requested using the GLM 1 model
in SPSS. More about that in a moment. To calculate the partial Eta squared
strength of association, we need a formula. All of the elements that are
going to go into this formula can be gleaned from the ANOVA summary table
that we created when we finished our analysis. We would begin with the sum of
squares between which plugs in in two places in the formula. We would then get
the sum of squares within, and plug it in. For the example with The Wizard of Oz,
this would be 37.75 divided by the sum of 37.75
plus 19.20, which equals 0.663. That means
that 66.3% of the variability in complaints about wicked witches is
explained by the region in which the Munchkins live. Remember that .663
value because we will see it again in SPSS. Now there is one more measure of
effect size that you may encounter with an ANOVA, Omega squared. Omega squared is
the standard effect size measurement for an ANOVA and it can be calculated from
the data available in the ANOVA summary table. It is also interpreted the same
way as R squared, however, Omega squared is an unbiased measure of effect size
and as such it will always be smaller than partial eta squared, which will
always be smaller than eta squared. Of the three, partial eta squared is the
most widely reported. Omega squared is calculated using this formula. You can
see that it is a little more complex than the partial eta squared formula, but
everything that we need to calculate it can be found in the ANOVA summary table.
We would begin with the sum of squares between, which plugs into the formula,
minus the degrees of freedom between, times the mean square within, which also
plugs in two places in the formula. Finally we plug in the sum of squares
total and we have all of the numbers. This example uses the data from the
previous video about the Wizard of Oz but you can use the same procedure with
any one-way ANOVA. In this case, Omega squared equals 0.587. That means that 58.7%
of the variability in complaints about wicked witches is
explained by the region in which the munchkins live. If you want to compute an
effect size for ANOVA using SPSS, you will need to use a different set of
drop-down menus to run the ANOVA than we used with the compare means command. This
is the way that you would run an ANOVA typically if you were doing it for
real-world research. The first method I showed you is easier for teaching
because it contains only the information for the ANOVA and leaves out the rest of
the general linear model making the output easier to understand.
However the compare means approach does not have an option for partial Eta
squared. To conduct this analysis using the wicked witch .sav dataset, go to
analyze > general linear model > univariate. move complaints to the dependent
variable box, and then move region to the fixed factors box. Now you can click on
options. Under options click on "estimates of effect size." You may notice others of
these options are the same as the one-way ANOVA that we did with the
explore means command. You could do your ANOVA with this set of commands instead.
Plus it gives you more options. So now click on continue, and ok. This is the
same output as we got using the compare means command but with a lot more detail.
In the output, you will find the partial Eta squared value, 0.663 in the last
column for the row region. Same value that we got calculating by hand. That
means that 66.3% of the variability and complaints about wicked witches was
explained by the region in which the Munchkins lived.