so we talked about how instrumental variables can be used
as our fourth and final approach to solving the non compliance problem
in experiments.
So let's look at that in action with The Oregon Healthcare experiment.
So here's our table of results that we looked at earlier.
And before we examine this column here, the effect of lottery selection.
So this was our intention to treat analysis, where we redefined the
treatment variable from receiving medicate to just being given the
option of receiving medicate.
And remember over here each of this different rows are different outcome
variables that we might be interested in.
Like what's the effect of giving the option of having Medicaid on
whether you have a positive depression screen or not.
Now, we don't necessarily care about what the treatment effect of
having the option of giving
Medicaid is, instead we really care about what
the treatment effect of having Medicaid is on whether you have a
positive depression screen or not, that's what column three does.
But we have non compliance, so how do we do this?
Where do these numbers come from?
Well if we look down here in the foot note we can see somewhere
over here, column three reports the Local Average Treatment Effect
for insurance coverage as estimated by Instrumental Variable Regression.
It two points.
First, estimated by Instrumental Variable Regression.
That's a little vague, but what they mean is they are doing two
stage least squares, and they make it clear in the body text of
the paper that that's what they are doing.
Second, they say Local Average Treatment Effect.
So they're getting some kind of average treatment effect we'll talk
about exactly what this local word means here in the later module.
So all of
these numbers in column three are two staged lease squares
estimates of the treatment effect of having medicaid on different
outcome variables and they are computed using two stage least squares.
So the first thing you should notice is let us compare them to the
intention to treat numbers, so let's just look at the depression
measures, if you compare the estimates we see that all of the two
stage squares estimates are larger in magnitude than the intention
to treat estimates, so minus 9.15 that's a lot larger than minus
2.21, so why might the the magnitude be larger? Well what's the
intention to treat doing?
Intention to treat is saying that there's a lot of people who didn't
comply with their treatment assignment, we said here you can have
medicaid and they said no thanks, but what we did is we kept them
in the treatment group, even though they didn't actually get medicated
medicaid.
So if having medicaid actually has a positive treatment effect then
these people who didn't comply with the treatment assignment they
are included in our treatment group but they are not actually receiving
any of the positive benefits of treatment, so that's going to bias
our overall effect estimate downward closer to zero and once
we solve for this problem by using the instrumental variable analysis
we''ll get a larger effect so that's exactly what we see happening
here.
The intention to treat numbers are all smaller then the two stage least square
estimates.
Now another way to think about this difference in magnitude, is
to remember how the indirect inference estimator works, and that's
not how we got these numbers, but the intuition is gonna be basically
the same.
So the indirect inference estimator starts by looking at the correlation
between the outcome variable and the instrument.
That's actually exactly what this column here shows us.
The intention to treat analysis.
Why is that?
Well because when we're analyzing noncompliance in the experiments,
our instrument is the treatment assignment variable.
If you are assigned to get Medicaid or not.
That's the instrument we're using and then our treatment is whether
you actually received Medicaid or not, and our outcome is any one
of these.
So this column here is the regression of the outcome on whether
you were assigned treatment or not.
That's exactly the reduced form regression used in indirect inference
estimator.
So getting this number is the very first step.
Then in the second step we look at the relationship between the
instrument and the treatment.
So whether you actually got treatment or not
and how does that compare to your outcome variable?
That's called the first stage estimator and that's shown in this
table over here.
Table S9.
The first stage, so this is our second step, so here on the left
hand side we have different measures of treatment, so for example
let's just focus on the first line, did you ever get Medicaid during
the study period?
So did you receive Medicaid?
That's our treatment so this is now our outcome variable because
we want to know what's the effect of the instrument being assigned
to receive Medicaid on the outcome actually receiving Medicaid.
So here we see that in the control group some people still manage
to receive Medicaid even though they weren't assigned to it then
if we look here, this second column shows the difference in the
percentage of people who received medicaid between the control groups
and
treatment groups, the lottery winners, the people who were assigned
to get Medicaid.
So there is a very large number 24%, so winning the lottery greatly
increased your chance of actually getting Medicaid.
It didn't push it up to a 100% because we had noncompliance.
But it largely increased it.
So this number here shows us that the relevance assumption in the
instrumental variables analysis holds, the instrument actually did
cause people to get treated, to actually receive Medicaid so the
indirect inference estimator is going to take this number, here
or similar numbers and then adjust this number here, to get this
number here.
So we take our reduced form numbers, the effect of the outcome
on the instrument, adjust it using the effect of
the instrument on the treatment to get the actual
effect of treatment on outcomes, that's how instrumental the interactive
inference estimator works.
So that's another way to interpret the increase in magnitude between
this one and this one.
Now we've gone through basically all the analysis in the Oregon
healthcare experiment paper, and you can pick up the New England
Journal of Medicine that has that paper published and you can read
every single piece of it and perfectly understand it.
[MUSIC]