[MUSIC]
most of the time when you do a randomized experiment, say
a drug trial, you're gonna assign some people to get the drug and
some people to not get the drug.
Well, the people who are in the treatment group, who you give them
the drug, they might not take it.
That's called non-compliance which happens when people don't
actually get treatment when they're supposed to.
Or conversely, when somehow people in the control group obtain the
real drug and start taking it.
That's not compliance.
Now there're four ways of dealing with non-compliance.
The first is called intention to treat analysis.
And the idea here is to redefine the treatment variable that we
care about.
And then just pretend that's the variable, we cared about all along,
okay.
What I mean is, we initially wanted to
know the effect of taking a drug on some outcome.
But the problem is people in our treatment group weren't taking
the drug or some of them weren't.
So what we're gonna do is say, actually our treatment of interest
is whether we gave people the option of taking a drug.
And with this redefinition non-compliance is impossible by definition
because we choose whether to give somebody the option or not, not
them.
So with this new definition, we can analyse the experiment just
by comparing the average outcomes in the treatment group with the
average outcomes in the control group forgetting about the fact
that some people on the treatment group might not actually be getting
treatment.
And that comparison is going to give us the causal effect of being
given the option of receiving treatment, and more specifically it
will give us the average treatment effect.
So for example,
Oregon Health Experiment, some people were given the option of enrolling
in Medicaid.
Not everybody did, but some people were given the option.
What we're gonna do is just compare everyone in the treatment group,
those who were given the option with everyone in the control group,
those who weren't given the option.
And that's gonna give us the average treatment effect for being
given the option to enroll in Medicaid on whatever outcome of interest,
cholesterol, blood pressure etc.
Second approach for dealing with non-compliance is instrumental
variables analysis.
And the benefit of this is that it's gonna let us get the true effect
of having Medicaid versus not, as opposed to just the option of
enrolling into Medicaid.
The downside is it's not gonna let us learn the average treatment
effect, okay.
It's not gonna let us learn the average unit causal effects over
the whole population.
Instead, we're gonna learn the average unit causal effects over
a subset of the population.
So there's a trade
off that by using this different analysis, we're gonna be
able to get effects of actually getting Medicaid, but only for some
people.
We're gonna talk about this method a lot more in detail later on.
Third approach is to assume random compliance.
That is assume that people just randomly decide to take
the drug or not when they're given the option.
Or assume that people randomly decide to enroll in Medicaid or not.
If this is the case, then you can just drop non-compliers from your
data, and compare the average outcomes of people in your treatment
group, who actually enrolled in Medicaid with the average outcomes
of people in your control group, who didn't enroll in Medicaid
because they weren't given the option.
And that difference is gonna give you the average treatment effect
of having Medicaid on cholesterol or whatever.
So unlike the instrumental variables analysis
and intention to treat, we actually get the average treatment
effect for the whole population.
The downside here is that this result really depends on this very
sketchy assumption of random compliance.
It's just probably isn't true in most cases.
People just make a decision of whether or not to enroll in Medicaid
or whether or not to take some drug that they're in some clinical
trial for.
And that decision probably isn't random.
It probably depends on things that are going to affect their outcomes
which we all observe.
And if that's the case, if random compliance is actually not true,
and people are making purposeful decisions whether to enroll or not,
then this comparison where we just drop non-compliers is not going
to give us a causal fact.
It's not going to give us the average treatment effect.
The fourth and final approach for dealing with non-compliance is
called bounce analysis.
Now the nice thing about bounce analysis
is that it works with very weak assumptions.
We don't have to assume random compliance.
We don't have to redefine the treatment variable
like in intention to treat analysis.
And unlike our instrumental variables analysis, we're actually gonna
learn something about average treatment effects.
The treatment effects over the whole population.
So fixes those downsides of all those other approaches.
The problem with it is that there's a cost of being so general.
And the cost is we're not gonna be able to learn exactly what the
average treatment effect is.
We're just going to get a bound for it.
What that means is we're gonna get a number where we say where the
average treatment effect isn't larger than this, then we give it
a lower bound which says, well, the average treatment effect is
definitely larger than this.
So in some cases this is really helpful.
If both the lower bound and the upper bound are positive numbers,
then we know that the average treatment effect is positive, for
example.
Or if both are negative numbers, we know that the average treatment
effect is negative.
But if the bounds
include zero, and have negative numbers and positive numbers, then
we're not gonna be able to learn the sign of the average treatment
effect.
Is having Medicaid good or bad for this outcome?
We don't know.
And that's because we're not making that many assumptions.
If you wanted to make random compliance then you'd be able to know
average treatment effect exactly.
So those are the trade offs we face in doing this kind of analysis.
And in the Oregon healthcare experiment, we'll go through each
and every one of these methods to see what they give us.
[MUSIC]