As the name implies a Two Factor Analysis of Variance allows you to decide whether or
not two factors make a significant difference in your experiment. As an example let's look
at these specimens known as dog bones, and the way this works there's dog bones made
of materials of copper, steel, and plastic, and at both ends of the dog bone you would clamp these
into a machine, so you apply a clamp on the left and the right side, the machine would
slowly pull apart the dog bone, and it would measure the force needed to break that material.
These regions of the dog bones are known as fillets, and you have choice for either small
fillets or large fillets, and we're curious whether or not the material, copper, steel,
or plastic, or the fillet radius makes a difference in the breaking strength of each of these
samples. This means we have two different factors, the first factor is the material
that we're using, and the second factor is the fillet radius. So in all we've got six
different types of dog bones. So as an engineer you ask somebody to make these dog bones for
you and run the experiments. And that technician returns a day later and presents you with
a table, and in that table it tells us the material, steel, plastic, or copper, and then
the fillet radius, small or large, and for each of these samples the technician measured
a strength required to break it. So if I receive this table the first thing I would ask the
technician is whether or not he or she ran the experiments in the order shown, so for
example is this the first experiment, the second, the third, etc. on down, and if that's
true it's a really good thing because that means the technician had randomized the order
in which he or she had tested all of these samples, and it's really nice to randomize
your sample, because you eliminate things like instrument drift for example. Perhaps
the instrument was accurate for the first couple of samples, but something caused it
to drift over time, or it caused it to become slightly inaccurate over time. So by randomizing
these things we can do what's called randomizing out this instrument drift, and there is a
lot of different unknowns that you won't be aware of, so it's always, always a good thing
to randomize the order in which you run your experiments. So in this case we've got a number
of different experiments that were run, so a whole slew of these. We want to figure out
based on all of these data points does it matter what material the dog bones were made
out of, and does it matter whether or the not the fillets were small or large, and it's
really difficult to tell at a glance looking at this table whether or not either of those
things are true. Before you do any kind of statistics at all confirm with the technician,
or if you are running the experiments yourself, confirm that all of the tests were run correctly,
in other words ask yourself, did the dog bone slip in the instrument, were there any chips or any other
abrasions in the dog bones, and if this is true for any of the dog bones immediately
throw out those data, those particular data points are worthless. Don't arbitrarily throw
out any data points, if they look particularly high or low don't arbitrarily thrown them
away unless you can justify it. Once you confirm that all the experiments were run correctly
and none of the samples were damaged before measuring it always at the data graphically.
So here I've plotted the force needed to break each sample, and I've plotted the material,
and I've plotted whether or not it was a small or a large fillet on each of the samples.
In each case here are the actual data points that allows me to see the actual measurements, and in
each case I've plotted the mean value for each of the bars, plus or minus one standard deviation
with the error bars. And that allows me to see at a glance that the steel samples are
likely stronger than the plastic samples, and likely stronger again than the copper
samples, it also lets me see that there's a chance that the large fillet makes the sample
stronger than the small fillet, but that's not entirely obvious because we see these overlapping
data points. So we wonder, "well, is it by chance that the large fillets are stronger,
or is this a real phenomenon, could we justify that they are indeed stronger. That's tough
to tell without doing a two factor analysis of variance, and that's the real utility of
doing statistical analysis, when it's tough to tell whether or not a difference is significant.Probably
the most useful thing that you'll get out of a two factor analysis of variance are three
different p-values. In this case I've called them P material, P fillet, and P interaction.
Typically a small p-value, a p-value of less than typically 0.05, we can say that material
we could justify, we could say "yeah, it makes a significant difference." So when we look
at these data I would expect a relatively small value for the p material. It looks
like the steel is indeed a lot stronger than the copper, I don't see much overlap, or any
overlap at all between the data points, between those two, and it looks like the plastic samples
are going to be the weakest, so I would expect in this one we run the analysis, I would expect
a small value for p material. It's a little bit harder, like I said, to figure out whether or not the
large fillet makes a significant difference, whether it's significantly stronger than the
small fillet. So I would expect a p-value for the fillet to be small but not necessarily
less than 0.05. In this third p-value, p interaction, it tells us whether or not the
large fillet made a more significant difference when we ran one factor versus another. It
looks here on the graph that the difference between the large and the small fillet for
the copper sample is larger than the difference for the steel sample. If that's indeed that
case then I would expect a small value for p interaction. When I run the two factor analysis
of variance I get a very, very small value for the p material, and that indicates
it's highly unlikely that we just randomly chose steel samples that were stronger than
copper, and it's highly unlikely that we just randomly chose samples from plastic that are
weaker than both copper and steel. We could say that the odds of this occurring are about
one in two billion, so highly unlikely. The p-value for the fillet is not as small as
p-material, but it does look like it made a significant difference, so this value of
0.025 is smaller than our selection criteria of 0.05, so in this case I would say that "yeah,
the fillets actually made a difference, it actually significantly improved the strength
of both copper, steel, and plastic, and this p interaction of 0.6 suggests that is doesn't
make a more impactful difference for plastic, copper, or steel, it improves it about the
same, by the same amount in all three cases. Here's a second simulated data set in which
I've tightened up the standard deviations for all of my samples, and I did that to show
the difference between the large fillet and the small fillet are more significant, we
don't see any overlap, or very little overlap what so ever between all of the data sets,
and consequently that decreases our p-value from, it was 0.025 before and now it's 3x10^-6,
so much more significant difference for the fillet. And here's a simulated data set in
which the material made a significant difference, but the fillet radius itself didn't make any
difference what so ever, we see a great deal of overlap between the data sets for the small
and large fillets, and consequently we get a relatively large p-value for that factor.
And here's a simulated data set in which neither the material nor the fillet made a significant
difference. We see it across the board the strength, the breaking strength is about the
same and there is significant overlap between all six of our samples. Here's a simulated
data set in which the strength of the material didn't make a difference, a relatively large
value, a p-value for material, copper, steel, and plastic, break about the same, however
the fillet actually made a difference. So in this simulated data set the large fillets,
for whatever reason, decreased the strength of the material, and we see the p-value for
the fillet is relatively small. Another thing is that there is no interaction between the
three, and what that means again is that the difference between the mean values for copper,
steel, and plastic, the large fillet decreased the strength of the material, but it did so
by the same amount in all three cases, so there's no interaction in this simulated data
set. So you might be asking, "what does it take to have an interaction, what would that
look like? How could we generate a simulated data set in which p interaction is a small
value?" This is what that data set might look like, I see a very small value for both p
material and p fillet, so there's again the material made a difference, and the presence of a fillet
made a difference, however now I see a very small value for p interaction, and when I
look at the difference between the small and large fillet for copper, and the difference
between that for steel I see about the same value, however when I look at the plastic
it looks like the presence of a fillet made a more significant difference, or it was more
beneficial for the plastic sample than it was for the copper or steel, and this is where
we see that interaction come into play, we see a relatively small value for p interaction.
And if you see something like that going on there might be something that you don't understand
experimentally, or there might be some other reason that the large fillet improved the
strength of the plastic more than the copper or steel. So to conclude I'll leave these
six simulated data sets here with their associated p-values just so you could see qualitatively
the differences in the data sets and how the affect the p-values.