This screencast is split into two parts. In this first part we'll look a bit into the
math associated with confidence intervals and in the second part I'll demonstrate what
confidence intervals actually mean using a series of graphical demonstrations. So in
this screencast I want to talk about the meaning of a confidence interval. By now your're familiar
with the mechanics, the mathematics behind how you calculate one, but it's a little bit
tough to interpret what it actually means, what your're actually doing. So for example,
here we have an engineer that measures and records the temperature of boiling water,
and he or she collects 5 different temperature measurements, and these 5 measurements constitute
1 sample. Don't confuse the number of measurements with the number of samples collected. So we
collect those data and you had 5 of them. The engineer reports while the 95 percent
confidence interval for the mean water temperature is 97 plus or minus 6 degrees C. What does
this plus or minus 6 degrees C mean? We might see it in a different form. Could be reported
the 95 percent confidence interval for the mean water temperature ranges from 91 to 103
degrees C, or you might see it, the 95 percent UCL, stands for the upper confidence limit,
for the water is 103, or the 95 percent LCL is 91 degrees C. What we really want to get
at is the true average, the true mean water temperature, use the Greek letter mu to symbolize
that, and if we knew that we wouldn't need to do any of this confidence interval because
we already know the true mean water temperature. And unfortunately we never get to know that
value. One reason we don't know it exactly is because all measurement devices have some
degree of uncertainty. In this case it could be that the thermalcouple, if you put it
in cold water, and then put it in hot water, and then put it back in cold water, it might
be that if it were a lousy thermalcouple it could be off by a few degrees, and that
difference between measurements prohibits us from making a distinct or an exact measurement
of the population mean. Another reason could be if we looked at the tank of water, let's
say that water was heated from below, we might expect a warm region of water at the bottom,
and then everything above that is a little bit cooler, so it's important when your're
taking temperature measurements of this vessel that you take your thermometer and measure
temperatures over here, measure one up in here, another one at the bottom, kind of randomly
all over the place. So if we took 5 different measurements your're going to be trying to
take measurements at 5 random locations, and the true population mean for the water temperature
is an average of an infinite number of measurements scattered randomly throughout the vessel.
Obviously we can't collect an infinite number of measurements, and it would also assume
that our thermometer is perfect, that there is no variation what so ever in the precision
of our instrument. Neither of these things will be true, we can only capture, let's say
you only had time to make 5 measurements and you don't have a very expensive thermocouple,
and that thermocouple has an inherent uncertainty in its measurements. So these are all the
reasons why you see these plus or minus values on the different measurements, or if you see
a range of measurements for the 95 percent confidence interval. But what does it mean?
I mean, 95 percent of what? What is the significance of 95 even? And I'll tell you what it doesn't
mean. It's a common misconception that it means that 95 percent of the measurements
will fall between the range 91 degrees C to 103 degrees C, and that's not true at all.
The best way I can come up with how to describe it is through a number of different simulations,
but before we do that for the technique that I am using, and the technique that you probably
started using assumes that the population of all the measurements are normally distributed,
so we'll be making that assumption throughout this screencast, and a normal distribution
is this bell shaped curve which your're familiar with, and if we're taking water temperature
measurements this side could be relatively hot, this side could be relatively cold. If
we were doing something else, like measuring the strength of an object, we pull on it until
it breaks, and this object over here might be strong, and on the left side these are
weaker objects, and if they're normally distributed the population mean would fall right here.
We don't ever get to know the population mean, all we get to know is if, in this case we
were breaking things, we might break something that is weak, something that is strong, we
might grab ideally, if it's normally distributed we'll capture some more measurements over
here, there's a possibility there might be one that's particularly strong off to the
right. So here I've made a sample size of N=6. If you took these 6 measurements and
averaged them we'd come up with the sample mean, and typically it's x over bar represents
the sample mean. Note that the sample mean does not match the population mean. Ultimately
this is what we want, we can't ever get at it, but we can get close and make some estimates
as to how close we are to the true population mean. The simulation now show, we'll present
a graph, there'll be a bell shaped curve, and we'll find a number of different data
points that were gathered randomly across this normal distribution, so the computer
will choose the numbers that are more concentrated towards the center and we might see a few
of them at the outlier. So in this case I've got a sample size of 1, 2 , 3, 4, 5, and N
is equal to 6, and if we wanted to we could collect a few more samples, and presumably
we'd have a higher sample density out here. And in this case N is equal to 11, but note
that I collected a sample of 11 measurements, I did not take 11 samples. What my simulation
will also show is a horizontal line that represents the true population mean. And remember in
reality we don't get to know the population mean, we don't get to know the distribution,
all we're left with when you do an experiment are these values, these 11 values, and from
these we have to figure out what the true population mean might be. Here's the results
of one run of my simulation, and what I've got in the horizontal line is the population
mean mu, what we're after. I am showing the bell shaped curve, which in reality you don't
get to know, and in my simulation it picks random numbers that are more heavily concentrated
towards the center of this curve. It could well pick points towards the top of it or
at the bottom of it, but they're highly unlikely the further you move away from the population
mean, and then what I've done is calculated the sample mean, x bar, and that's just the
average of these three measurements. I've also plotted the 95 percent upper confidence
limit, and the 95 percent lower confidence limit, and this whole range from top to bottom
is known as the 95 percent confidence interval, and the reason these are all 95 percent instead
of, let's say 90 percent or 99 percent, is because I chose a value of alpha of 0.05,
and I've also opted to collect three measurements for each sample. So if I rerun this I would
collect three more samples, but they wouldn't be in the same location, they'll be randomly
distributed over this normal distribution, so things are going to look different every
time I run the simulation, and what I want you to pay attention to is the number of times
that the confidence interval does not contain the population mean. What this means is when I
run the simulation over and over again 95 percent of the time the population mean will
be within these error bars. One example might be let's say we had a class of 100 students
and each student took three measurements, one sample of materials that would pull apart,
and you measure the force needed to pull the material apart. So strong material requires
a large force, a weak material doesn't require much force at all. And in a pile of parts
if we were able to measure them, let's say we had 100,00 parts, and let's say we spent
weeks measuring the breaking strength of all 100,000 parts then we'd know the population
mean, but unfortunately it took us weeks to do that and now we no longer have any parts
to use because we broke all of them. Let's stick with this simulation. We can't break
all the parts, we can't measure them all anyways. So we've got, it's a classroom environment,
each student collects a sample of three measurements. The 95 percent confidence interval is calculated
by each student. So you go through the math you learned in class, calculate the 95 percent
confidence interval. What this means is that on average 5 students out of 100 will calculate
a confidence interval that does not contain the true population mean, however we'll never
know who those students are, we'll never know who encompassed, truly encompassed the population
mean or not because nobody knows the true population mean. This is the end of part I.
See part II for a graphical demonstration of what confidence intervals actually mean.