[MUSIC] Most of the time when you do a randomized experiment, say a drug trial, you're going
to assign some people to get the drug and some people to not get the drug.
Well, the people who are in the treatment group, who you give them the drug, they might
not take it.
That's called Noncompliance, which happens when people don't actually get treatment when
they're supposed to, or conversely, when somehow people in the control group obtain the real
drug and start taking it.
That's noncompliance.
Now, there're four ways of dealing with noncompliance.
The first is called Intention-To-Treat analysis.
And the idea here is to redefine the treatment variable that we care about, and then just
pretend that's the variable we cared about all along.
What I mean is we initially wanted to know the effect of taking a drug on some outcome.
But the problem is people in our treatment group weren't taking the drug, or some of
them weren't.
So what we're going to do is say: actually our treatment of interest is whether we gave
people the option of taking a drug.
And with this redefinition, noncompliance is impossible by definition, because we choose
whether to give somebody the option or not, not them.
So with this new definition, we can analyze the experiment just by comparing the outcomes
- the average outcomes in the treatment group with the average outcomes in the control group,
forgetting about the fact that some people on the treatment group might not actually
be getting treatment.
And that comparison is going to give us the causal effect of being given the option of
receiving treatment, and more specifically it will give us the average treatment effect.
So for example, in Oregon Health Experiment, some people were given the option of enrolling
in Medicaid.
Not everybody did, but some people were given the option.
What we're going to do is just compare everyone in the treatment group, those who were given
the option, with everyone in the control group, those who weren't given the option.
And that's going to give us the average treatment effect for being given the option to enroll
in Medicaid on whatever outcome of interest, cholesterol, blood pressure, etc.
The second approach for dealing with noncompliance is Instrumental Variables Analysis.
And the benefit of this is that it's going to let us get the true effect of having Medicaid
versus not, as opposed to just the option of enrolling into Medicaid.
The downside is it's not going to let us learn the average treatment effect.
It's not going to let us learn the average unit causal effects over the whole population.
Instead, we're going to learn the average unit causal effects over a subset of the population.
So there's a trade off that by using this different analysis, we're going to be able
to get effects of actually getting Medicaid, but only for some people.
We're going to talk about this method a lot more in detail later on.
The third approach is to assume random compliance.
That is, assume that people just randomly decide to take the drug or not when they're
given the option, or assume that people randomly decide to enroll in Medicaid or not.
If this is the case, then you can just drop noncompliers from your data, and compare the
average outcomes of people in your treatment group who actually enrolled in Medicaid with
the average outcomes of people in your control group, who didn't enroll in Medicaid because
they weren't given the option.
And that difference is going to give you the average treatment effect of having Medicaid
on cholesterol or whatever.
So unlike the instrumental variables analysis and intention to treat, we actually get the
average treatment effect for the whole population.
The downside here is that this result really depends on this very sketchy assumption of
random compliance.
It just probably isn't true in most cases.
People just make a decision of whether or not to enroll in Medicaid, or whether or not
to take some drug that they're in some clinical trial for.
And that decision probably isn't random.
It probably depends on things that are going to affect their outcomes which we all observe.
And if that's the case, if random compliance is actually not true, and people are making
purposeful decisions whether to enroll or not, then this comparison where we just drop
noncompliers is not going to give us a causal effect.
It's not going to give us the average treatment effect.
The fourth and final approach for dealing with noncompliance is called Bounds Analysis.
Now the nice thing about bounds analysis is that it works with very weak assumptions.
We don't have to assume random compliance.
We don't have to redefine the treatment variable like in intention-to-treat analysis.
And, unlike our instrumental variables analysis, we're actually going to learn something about
average treatment effects - the treatment effects over the whole population.
So it fixes those downsides of all those other approaches.
The problem with it is that there's a cost of being so general.
And the cost is we're not going to be able to learn exactly what the average treatment
effect is.
We're just going to get a bound for it.
What that means is we're going to get a number where we say where the average treatment effect
isn't larger than this, then we give it a lower bound which says, well, the average
treatment effect is definitely larger than this.
So in some cases this is really helpful.
If both the lower bound and the upper bound are positive numbers, then we know that the
average treatment effect is positive, for example.
Or if both are negative numbers, we know that the average treatment effect is negative.
But if the bounds include zero, and have negative numbers and positive numbers, then we're not
going to be able to learn the sign of the average treatment effect.
Is having Medicaid good or bad for this outcome?
We don't know.
And that's because we're not making that many assumptions.
If you wanted to make random compliance then you'd be able to know average treatment effect
exactly.
So those are the trade-offs we face in doing this kind of analysis.
And in the Oregon Healthcare Experiment, we'll go through each and every one of these methods
to see what they give us.
[MUSIC].