(low bass techno music)
>>Hi again students.
In this video,
we will study another quasi-experimental technique
that can be used to obtain causal estimates of a treatment
from observational data.
This is a pretty fun one called regression discontinuity
or RD for short.
RD is covered in chapter six of our textbook.
Suppose we want to measure
the effect of being a National Merit Scholar
on college GPA or college completion rate
or even on salary in the first job after college.
Well for sure, we can't just compare the GPAs
of National Merit Scholars and other students
because selection into treatment isn't random.
You have to score very high on a standardized test
to be a National Merit Scholar.
In other words, those are smart kids
who would very likely do well in school and in life
without getting the scholarship.
So we can't simply attribute their superior performance
to the effect of the scholarship.
Regression discontinuity gives us an unbiased estimate
of the treatment effect by comparing outcomes with students
who just barely qualified for the scholarship
with outcomes of students who just barely missed
qualifying for the award.
Let's consider an example.
Suppose that in Oklahoma the cutoff PSAT score
for winning the National Merit is 1,400.
Well, it's a bit more complicated than that in real life.
Let's say that students scoring 1,400 or above
get a National Merit Scholarship while those below do not.
While we know we can't just compare
the outcomes of the two groups,
regression discontinuity proposes
comparing the outcomes of people just above 1,400
to the outcomes of people just below 1,400.
Formally, we can consider
that right around that cutoff score,
the awarding of the scholarship is essentially random.
One student fell ill and did a bit worse.
Another student dreamt about cosines
and did a little bit better.
The gist of the matter is those who just missed the cutoff
are great counterfactuals
for those who just made the cutoff.
Let's look at two graphs that explain the technique.
First consider a graph with the probability
of getting the scholarship on the vertical axis
and the PSAT score on the horizontal.
In our example, for PSATs below 1,400,
the probability of treatment is zero.
And for scores above 1,400, the probability jumps to one.
This is the discontinuity that we are using
to identify the treatment effect of the scholarship.
Now consider a graph with say salary
at first job after college on the vertical axis
and PSAT score again on the horizontal.
To test whether the scholarship increases earnings,
we basically look for a discontinuous jump in earnings
at the cutoff score of 1,400.
In other words, does the continuous jump
and the probability of treatment
produce a discontinuous jump in post-college earnings?
If so, we attribute the jump
as being caused by the scholarship.
The estimated treatment effect
is simply the size of the jump.
Graphs like these are at the heart of RD analysis.
As we will discuss later,
we could estimate the size and significance of the jump
using local linear regression or a polynomial regression.
But if the graph doesn't show a jump,
there's no real effect to be found
by further statistical analysis.
I also want to mention that what is required
is not that the probability of treatment
goes from zero to one at the cutoff.
That's the cleanest case called a sharp discontinuity.
All that's required is the probability of treatment
jumps discontinuously at the cutoff.
Cases where it is less than zero to one jump
are called fuzzy discontinuities.
This graph gives an example of a fuzzy discontinuity.
As always, there are factors that can confound our analysis
and we need to check for them.
Fortunately, it is fairly straightforward to do so.
The first thing is to make sure
people can't precisely manipulate the treatment.
For example, if a treatment was conditioned
on having an income below a certain threshold,
we want to make sure that people with too high income
were not strategically under-reporting
to qualify for the treatment.
This is important because we are assuming that people
on either side of the threshold are essentially the same.
If people who are truly away from the threshold
can misleadingly appear to be at the threshold,
it messes up the validity of our identification.
We can test this simply by plotting the distribution
of the variable that determines treatment
against the test score and make sure that it is smooth
around the cutoff value.
In a similar way,
we want to make sure that other potentially relevant factors
do not take a discontinuous jump at the threshold.
For example, if we graph parental income against PSAT score
and it also jumps discontinuously at 1,400,
then we would be worried that the effect on earnings
could be partly due to parental income
and not just to getting the scholarship.
Again, we can test this by plotting the distribution
of the potential confounding variable against the test score
and make sure it is smooth around the cutoff value.
For estimating the treatment effect in an RD model,
we can either use all the data in a parametric model
that is usually a polynomial
or use limited data around the cutoff point
in a non-parametric model
that is usually a local linear regression.
In the first case,
we are worried about getting the functional form correct.
For example, if we estimate a linear model
and there is a smooth non-linear relationship
between the outcome variable, in our example earnings,
and the variable determining treatment, test scores,
we may find a jump that truly is not there.
Consider the following graph from your text
that illustrates the issue.
As you can see,
given the true non-linear relationship in the data,
the linear model estimate a treatment effect
that is not really there.
Now, here's an example of fitting a polynomial
to estimate the treatment effect.
The model in the graph regresses the outcome on the level,
square and cube of X along with a dummy variable
that shifts the intercept at the cutoff point.
In this example, a significant treatment effect is found.
Another option is to estimate separate polynomials
on each side of the cutoff along with a dummy variable.
Most RD papers will present several different
estimations of the treatment effect.
The non-parametric approach only uses data near the cutoff
and runs a linear regression on the subset of the data.
Analogous to the functional form issue
in the parametric approach,
here we have the issue of how much data to discard.
Or more formally, of choosing the bandwidth
for the nonparametric model.
Again, in most cases, papers will present results
from several different bandwidth choices.
Given the choice of bandwidth,
the regression is just the outcome
regressed on the variable determining treatment,
plus a dummy variable to shift the intercept at the cutoff.
Alternatively, the regression line could be allowed
to have a different slope on either side of the cutoff.
This graph illustrates the idea.
We have data on either side of the cutoff
and our choice of bandwidth
determines how much of the data we use
to estimate the treatment effect
with a local linear regression.
Because we are actually getting
local randomization of treatment,
most RD papers do not add other co-variants
to the estimated models.
The regressions are just the outcome
regressed on a polynomial
of the variable determining treatment,
plus a dummy for the possible shift at the cutoff
on all the data
or the outcome variable linearly regressed
on the treatment determining variable
along with the cutoff dummy
on a subset of the data near the cutoff.
The big advantage of RD
is it comes closer to actual randomization of treatment
than any of the other quasi-experimental methods.
The big disadvantage is the RD requires a treatment rule
that has a discontinuity in the probability of treatment
at some known, measurable point.
Despite this complication,
researchers in health sciences, labor economics,
development economics, and political science,
have all found productive uses for the technique.
Researchers have even used RD
where the cutoff is geographical
rather than a test score or an income level.
When conditions allow,
RD is the cleanest and most convincing way of estimating
a treatment effect in observational data.
The major issues are making sure you have an adequate
functional form in parametric estimation
and showing that your results do not depend heavily
in your choice of bandwidth in non-parametric estimation.
(inspiring theatrical music)