So in this screencast we will be talking about how to implement an F-test in MATLAB and in
Excel, and what I am showing on the left are two samples, both of which have a sample size
of five, so five different measurements from each of the two samples. And here is a graph
of those two samples so we can visualize it graphically. When you perform an F-test what
you are really asking is how likely is it that the data points from the two samples
came from distributions with different variances. So these values I pulled in my computer program,
I pulled them both from a normal distribution. The question is did I use a distribution with
a larger variance for sample one than I did with sample two? And when I look at these
I say yeah, probably, I look at the spread of the data in sample one and note that that
is larger than the spread of the data in sample two, and before I even do the F-test I am
app to say I do think they came from two populations with different variances, and indeed that
is the case. For sample two, my computer program, I sampled from a normal distribution that
has a smaller variance than the distribution I used for sample one. But remember in real
life you never get to know this, you never get to know the two distributions, all you
ever see are these data points. So in MATLAB I've already entered the arrays for x1 and
x2, my two samples, and when I run it remember that the spread, or the distribution for x2,
the variance for x2, is going to be smaller than the variance for x1, and it looks, that
gives us an argument that the distributions are indeed different, the variances are different
for it. So lets test that. The function we use is vartest2, and I am going to use two
arrays, x1 and x2. When I implement that it says the answer equals one. What does that
mean, answer one? What it means is that at alpha equal 0.05 we can reject the null hypothesis
that the variances are equal, and it provides evidence that the variances are indeed different.
So lets do this, we can calculate that value one, we'll store it as h, but really want
the p-value from the F-test, so I'll do vartest2 again with x1 and x2, and this will return
h again equal to one, but this returns the p-value, and that p-value of 0.04 is indeed
less than our selected value of alpha of 0.05. vartest will default to alpha equal 0.05.
If I change it I can also say, lets say alpha, lets set alpha equal to 0.01 for example.
Now when I run it the p-value is greater than 0.01, and this means that with a value of
alpha 0.01 we cannot reject the null hypothesis, and we'll say the evidence now is not strong
enough to indicate a difference in variance, so that value of alpha remember is entirely
up to you, but typically people will use alpha of 0.05. Here's how I set it up in Excel,
I've got my two samples, sample size of five in both cases. We want to run an F-test to
get the p-value to figure out if the variances are different, so I use f.test, and f test
accepts two arrays. The first one will be sample one, and the second array will be sample
two. I'll close off the paranethesis and push enter, and I do get the same p-value, which
suggested significant below alpha of 0.05, but insignificant if we chose alpha of 0.01.
Here is a different situation. I again pulled random numbers from a normal distribution for
sample one and sample two, and we want to ask ourselves "is the underlying variance
from the two populations, are they different or are they the same?" so let's run an F-test
on these two sample. So MATLAB will do the same thing, vartest2 for x1, and x2, and I
return a p-value of 0.61, h is zero which means we cannot reject the null hypothesis.
It looks like measurements for both samples come form two populations with the same variance.
I could do the same thing in Excel. Let's paste sample one and sample two. I've entered
the numbers for the measurements for both samples, we'll put the p-value here, and again
it's just f.test, I'll push tab to autocomplete it. Select the first array, and select the
second array, and that returns the same p-values we saw at the end of MATLAB, this 0.61307.
That large p-value of 0.6 suggests that the measurements from the two samples came from
populations that did have the same variance. And as it turns out that is the case, I chose
two normal distributions with the same standard deviation for both. Those variances are indeed
the same and that agrees with what we came up with for the F-test. Here is our third
situation. Two samples with five measurements for each sample, and we want to figure out
is the variance for this sample different than the variance for this sample? Did the
measurements come from populations with different variances, and when I look at this it looks
to me like they probably do but we don't know for sure. When I run these in MATLAB I come
up with a p-value of 0.034, or 0.035, and that suggests that we can reject the null
hypothesis, it says the variances are likely different between the two populations. And
the p-value from MATLAB agrees with the p-value that we come up with for Excel, about 0.034,
0.035. Just remember though that statistics is a matter of probability, it only gives
us the likelihood that they came from underlying populations with different variances. So here
a graph of the data again, we look at sample two it looks like we came from a population
with a higher variance than sample one. As it turns out though they came from populations
with the same variance, even though we've rejected the null hypothesis based on these
data, we suggested the variances were different, what we did was incorrectly reject the null
hypothesis, and in doing so we've made a type one error. So just keep that in mind when
you run an F-test, it provides evidence, but it never provides conclusive proof, and
if you remember we don't ever get to see these actual underlying distributions, we can only
make some inference, or we can only report the likelihood that they are different. And
the take home message is the smaller the p-value, the higher the likelihood that the two variances
are indeed different.