Often it is the case that a unit's treatment is determined
by whether another variable of theirs is above or below a threshold,
and this leads to the idea of regression discontinuity.
The basic idea is that people just above the threshold get treated and people
just below the threshold don't get treated.
And yet these people are basically the same, there's really no systematic
difference between the people who got a score just above the cut
threshold and people who got something just below it.
So we have something that look like a randomized experience the
people who're just above get treated, the people who're
just below don't get treated so if there's no systematic difference
between those groups, then we can just compare the average outcomes
in the people on the right hand side of the cut off with people
on the left hand side of the cut off.
And that's it, so
the best way to understand this is to go straight to the example
and we're gonna go to the very first study that ever used this in
fact that invented regression discontinuity a paper by Thistlethwaite
and Campbell in 1960.
And they were interested in the following question.
How does receiving social recognition for outstanding academic
achievement affect various outcomes like whether you win scholarships
or whether you want to got on to college or not.
Does it affect your career plans?
So here's how they try to answer that they use data from the National Merits
Scholarship Program which started in 1955 and and continues till
today.
Now I'm going to give you a simplified overview of exactly how this
program works.
l'm gonna leave out a few details just so that I can focus on the
main essence of their identification idea.
So today here is how it works.
There's about 1.5 million high school students who take
the PSAT exam.
Now, allof those students 50000 or so, of the highest students are
taken, and among them, 16000 of the highest students, are given
a certificate of merit.
And the rest are not.
Now why is a certificate Why is a certificate of merit important?
Because they showed the authors, that in 1960, the people who got
the certificate of merit were more likely, 2.5 times more likely
to be precise, to be mentioned in newspaper articles than the students
who didn't, who had gotten just a cut-off for getting the certificate
of merit, test score cut-off.
In addition those students who got the certificate or merit, their
names were all put in books and distributed to colleges and universities,
and to scholarship granting agencies.
So the people who got certificate of merit, they got more public
More public recognition than students who didn't so what we can
do is look at the effect of getting a certificate of merit or not
on various outcomes and that's basically gonna tell us something
about the effect of public recognition that's there main idea so
let's go look at the data and see what they found.
So here is the first main result from the [UNKNOWN] and Campbell
paper, over here on our horizontal access we've got abstitute/g
test scores of students and arbitrary units.
So, this is their PSAT test score basically, and by arbitrary units
they mean that they've normalized the scale of test scores.
For these population of students.
So right here in this dotted line, that's the cut off test score
for getting a certificate of merit so everyone who got test scores
up here, the high test scores winners, these people all got certificate
of merit, these people down here, they did not get the certificate
of merit.
So, what's on the vertical axis?
That's the percentage of students winning scholarships.
So we got two lines which appear in almost eligible worlds.
The first line appear labelled G is just the percentage of students
winning any scholarship whatsoever.
This line is the percentage Percentage of students winning scholarships
of $150 or more which made it sound like a very big scholarship
these days but if you put this in, this is $19.60, if you put it
in today's dollars you get something around $1200.
So that's a lot more substantial.
So these are the big scholarship.
So the few things to notice to begin with, if we just look over
here we see that the percentage of students winning scholarships
is increasing in your test score so that seems to make sense people
will get higher test scores, win more scholarships on average.
In both overall and high paying scholarships
now over here, we actually see the kind of opposite, forget about
this, just look the high scoring students.
Over here we see the opposite trend, getting a high test score means
you'll get less scholarships, now why is that?
That's because the people who are over here did really well on the
PSAT test and it made them more likely to actually go all the way
to the end of this process and get a national merit scholarship.
So if you receive the national merit scholarship, that it's less
likely that you'll receive an outside scholarship from some other
agency or scholarship granting group.
So that's why we are actually seeing it going down because here
we're looking at overall scholarships, now including national merit.
So that's the basic layout of this table, now let's look at the
regression discontinuity idea.
So here, by the cut-off, the basic idea is that the people who are
just below the cut off
essentially saying the people above, that if they had taken the
test on a different day, maybe the person who scored a ten would
have scored an 11 and maybe the person who scored an 11 would have
gotten a ten, just because it's something completely random, not
because there is any systematic difference between these two groups,
so that means these two groups, so that means that we compare the
students who just got about a ten with the students who just got
an 11 and any difference in average outcome between these two groups
is solely attributed to the fact that people on the right hand side
of the cut off got a certificate of merit and the people on the
left didn't, so if you do that comparison you can see that there
is a jump.
There's a discontinuity in the regression line, right here.
It seems that people who got the certificate of merit had about,
let's see, about 51% compared to about
Percent compared to about 46% so there is a five percent gap or
a five percent treatment effect of getting the certificate of merit.
So that's one resolve and if you see the same thing is true for
overall scholarships not just for the high paying ones that there
is this discontinuity of To six percent of students and that difference
there at the boundary at the cut-off is our estimated treatment
effect.
Now this was 1960 and we've come a long way in understanding how
do we compute confidence levels for this differences right at the
boundary which is not an easy thing to do so So you can see these
lines are really wiggly so if you actually were to take this data
and computer confidence interval, even though a point estimate just
the difference this regression lines we see is positive so I suggest
there is a treatment effect which is what the offers conclude we 127 00:08:00,188 --> 00:08:00,000 actually wouldn't have
Significant effect because of all the excess variation that's going
on over here.
So that's the basic idea of regression discontinuity.
Now there is one more thing I want to mention, now on the horizontal
axis this is test score, so it's very important that we are looking
close to the cut-off because if we were just to compare everybody
over here in this group so take the average percentage of students
winning scholarships over here the average outcome, and compare
it to the average outcome over here, we would also see that overall
the average is higher, overall here the average is higher maybe
it's about the same but it's a little bit higher it looks like,
but that will be confounded because the students on average over
here have better test scores that's why they are here in the first
place.
The people over here on average have worse test scores than this,
so there is a confounding variable with our treatment.
It's only once we look close to the cut off, that that confounding
variable is no longer an issue because close to the cut off these
two all have roughly the same test score.
So that is very important we have to look just next to the cut off
and only compare averages in the right with averages on the left,
that's the main idea.
[MUSIC]