[MUSIC] So you've done your experiment: you've randomly assigned some people to have the
option of enrolling in Medicaid and some people to be in the control group, and you've waited
a year, you've gathered your data, and now it's time to look at the outcome and see what
the results are.
So here we've got all our outcome data of our experiment.
Over here, we've got a whole bunch of different outcome variables that we're interested in:
we've got blood pressure, cholesterol, hemoglobin, depression measures and cardiovascular risk.
And if you're a medical doctor, then all of these numbers, these variables, have lots
of meaning to you and why they matter for health related outcomes.
So what we want to know is: what is the effect of having Medicaid on things like positive
depression screen - so this is a measure of whether you are depressed or not.
So does having Medicaid lower depression, make you less depressed?
Well, we're going to find out very soon.
So remember that experiments are relatively easy to analyze.
We just want to compute the average treatment effect, which is the difference between the
average outcomes in the treatment group and the average outcomes in the control group.
So over here in Column 1, they've given us the mean, or average value, in the control
group.
So for example, all of these are means, and in the control group 30% of people had a positive
depression screen, for example.
Now they haven't given us the average values in the treatment group, which is an imaginary
column right here, but what they have given us is the average treatment effect, which
is taking the average value in the treatment group minus the average value in the control
group and then that gives us this column here, Column 2.
So these numbers here are all differences between the average value in the treatment
group and the control group.
So for example, the positive depression screen, we have an average value of 30 in the control
group, 30, and then the difference is -2.21, so the value in the treatment group is about
28.
So only about 28% of people in the treatment group are depressed whereas its 30% in the
control group.
So from this number, it looks like being in the treatment group lowered whether you got
depressed or not.
So, because this is an experiment, we know that this number here, this average treatment
effect -2.21, is a true causal effect.
We don't have to worry about confounders or selection problems or anything like that because
of random assignment.
But we only have a finite number of people.
We have to deal with the fact that we just gathered some data, we didn't actually do
the experiment on everybody in the state of Oregon or everybody in the whole country.
So we have to worry about finite sample problems, that means statistical inference in addition
to causal inference.
That's exactly what these numbers here in parenthesis do.
These are 95% Confidence Intervals.
So let's talk about how we interpret these.
So here we got the 95% Confidence Interval for the average treatment effect of having
Medicaid on positive depression screen.
So here is the interval: -4.02 to -0.40.
The first thing you ask with confidence intervals is: is zero in the interval?
No, it's not.
If zero is not in the interval, you conclude that your estimate of the average treatment
effect,-2.21, is a statistically significant effect.
That means that it's not due to the particular people who happen to be put into your sample
in the experiment, it's really a true causal effect.
So just because we found that this confidence interval doesn't have zero, we conclude that
it's a statistically significant effect, but that doesn't mean that it's a clinically significant
effect.
So is a -2.2% decrease in depression something that matters as far as the world is concerned?
Well, that is something that you have to go to a medical doctor to think about.
But as far as we are concerned, there is no problem with small sample size.
We've been able to detect an effect.
Being in the treatment group here lowered depression by 2%.
Now in the next module, we'll look at another of these variables where this confidence interval
actually has zero, then we have to start thinking about few different things.
[MUSIC].