[MUSIC] So we talked about how Instrumental Variables can be used as our fourth and final
approach to solving the noncompliance problem in experiments.
So let’s look at that in action with the Oregon Healthcare Experiment.
So here's our table of results that we looked at earlier.
And before we examined this column here, the Effect of Lottery Selection, so this was our
intention-to-treat analysis where we redefined the treatment variable from receiving Medicaid
to just being give the option of receiving Medicaid.
And remember over here, each of these are different rows are different outcome variables
that we might be interested in, like what's the effect of giving the option of having
Medicaid on whether you have a positive depression screen or not?
Now we don't necessarily care about what the treatment effect of having the option of getting
Medicaid is, instead we really care about what the treatment effect of having Medicaid
is on whether you have a positive depression screen or not.
That's what column three does.
But we have a noncompliance.
So how do we do this?
Where do these numbers come from?
Well, if we look down here in the footnotes, we can see somewhere over here, “Column
3 reports the local average treatment effect for insurance coverage as estimated by instrumental
variable regression.”
I’ll say two points.
First, “estimated by instrumental variable regression.”
That's a little vague, but what they mean is they're doing two stage least squares,
and they make it clear in the body text of the paper that's what they're doing.
Second, they say “local average treatment effect.”
So they're getting some kind of average treatment effect.
We'll talk about what exactly this “local” word means here in a later module.
So all of these numbers in Column 3 are two stage least squares estimates of the treatment
effect of having Medicaid on different outcome variables.
And they are computed using two-stage least squares.
So the first thing you should notice is - let's compare them to the intention to treat numbers.
So let's just look at the depression measures.
If you compare the estimates, we see that all of the two-stage least estimates are larger
in magnitude than the Intention-to-Treat estimates.
So -9.15, that's a lot larger than -2.21.
So why might the magnitude be larger?
Well, what's Intention-to-Treat doing?
Intention-to-Treat is saying that there's a lot of people who didn't comply with their
treatment assignment.
We said, “here you can have Medicaid” and they said “no thanks,” but what we
did was we kept them in the treatment group, even though they didn't actually get Medicaid.
So if having Medicaid actually has a positive treatment effect, then these people who didn't
comply with the treatment assignment, they're included in our treatment group, but they're
not actually receiving any of the positive benefits of treatment.
So that's going to bias our overall effect estimate downwards, closer to zero, and once
we solve for this problem by using the instrumental variable analysis, we'll get a larger effect.
So that's exactly what we see happening here, the Intention-to-Treat numbers are all smaller
than the two-stage least squares estimates.
Now, another way to think about this different in magnitude is to remember how the Indirect
Inference Estimator works, and that's not how we got these numbers, but the intuition
is going to be basically the same.
So the indirect inference estimator starts by looking at the correlation between the
outcome variable and the instrument.
That's actually exactly what this column here shows us, the Intention-to-Treat analysis.
Why is that?
Well, because when we were analyzing noncompliance in our experiments, our Instrument is the
treatment assignment variable.
If you were assigned to get Medicaid or not, that's the Instrument we are using, and then
our treatment is whether you actually received Medicaid or not, and our outcome is any one
of these.
So, this column here is the regression of the outcome on whether you were assigned to
treatment or not.
That's exactly the reduced form regression used in the indirect influence estimators.
So, getting this number is the very first step.
Then, in the second step, we look at the relationship between the Instrument and the treatment,
so whether you actually got treatment or not, and how does that compare to your outcome
variable.
That's called the first-stage estimator, and that's shown in this Table over here, Table
S9, the first-stage.
So this is our second step.
So here on the left-hand side, we have different measures of treatment, so, for example, let's
just focus on the first line.
Did you ever get Medicaid during the study period?
So, did you receive Medicaid?
That's our treatment, so this is now our outcome variable, because we want to know what's the
effect of the instrument - being assigned to receive Medicaid - on the outcome - actually
receiving Medicaid.
So here, we see that in the control group some people still manage to receive Medicaid
even though they weren't assigned to it.
Then if we look here, this second column shows the difference in the percentage of people
who received Medicaid between the control groups and treatment groups - the lottery
winners, the people who were assigned to get Medicaid.
So there is a very large number, 24%.
So winning the lottery greatly increased your chance of actually getting Medicaid.
It didn't push it up to 100% because we had noncompliance, but it has largely increased
it.
So this number here shows us that the relevant assumption in the instrumental variables analysis
holds.
The Instrument actually did cause people to get treated, to actually receive Medicaid.
So, the indirect influence estimator is going to take this number here, or similar numbers,
and then adjust this number here to get this number here.
So we take our reduced form numbers, the effect of the outcome on the Instrument, adjust it
using the effect of the treatment - sorry, of the Instrument on the treatment, to get
the actual effect of treatment on outcomes.
That's how indirect inference estimator works.
So that's another way to interpret the increase in magnitude between this one and this one.
So, now we've gone through basically all the analysis in the Oregon Health Experiment,
and you can pick up the new England Journal of Medicine that has that paper published
and you can read every single piece of it and perfectly understand it.
[MUSIC]