(melodic electrical humming)
>>Now let's talk about reality checks.
How do you know that your diff-in-diff model
satisfies the parallel trend assumption
and is giving you an unbiased estimate
of the effect of the policy or treatment
that you're studying?
If you have enough data, pre-treatment,
you can graph the outcome variable
for the treatment group and control group
and see if they track each other.
That is, we can use pre-treatment data
to see whether the parallel trends assumption holds.
This test is usually done informally
by presenting the graph and discussing
how well the two groups track each other.
Another way to test the validity of the model
is to look for an effect where
there has not been any treatment.
This type of test is usually referred to as a placebo test.
The idea is this.
If the trends are not parallel,
then we may find a spurious treatment effect
when in fact there's been no treatment given.
If we run a fake diff and diff with the same data,
and we find a spurious effect,
then we are not very confident that our actual diff-in-diff
is capturing a real treatment effect.
Another way to say this, is if the situation
is suitable for our technique to work properly,
we should not be finding a significant effect
from non-existent treatment.
We've explained the basic logic of diff-in-diff,
described how to implement the technique
both in a "raw" form and via regression,
we discussed the importance of parallel trends
and how to test for them.
Let's conclude this section by talking about
a few more examples of research
using the diff-in-diff technique.
As our textbook mentions,
the first difference in difference project
was done in the 1850s by Dr. John Snow in England.
He wanted to show that the disease cholera
came from drinking contaminated water.
In London, water was supplied by two companies,
both of which initially drew it right out of the Thames,
right as the river passed through the middle of London.
Needless to say, that was some nasty polluted water.
A lot of raw sewage was being dumped
into the river in this highly populated area.
However, in 1852, one of the companies
started sourcing their water
from a spot upstream from London.
The doctor compared death rates in districts
served by the two water companies
before and after the one company
started using cleaner water and found
that cleaner water greatly lowered mortality from cholera.
A more up-to-date example can be found
in a 2016 National Bureau of Economic Research Working Paper
by two economists, Justin Pearce and Peter Schott.
They compared counties with manufacturing jobs
vulnerable to competition with China
to counties whose manufacturing is less vulnerable,
before and after the US significantly opened up
trade with China in October 2000.
Using diff-in-diff, they found that opening trade
led to a significant increase in suicides
among white males in the counties
more vulnerable to Chinese competition.
Finally, I also want to mention
a slightly different type of diff-in-diff analysis
that comes from my own research.
In a 2016 paper published in Economic Inquiry,
me, Daniel Hicks and Weici Yuan studied
whether gender imbalance in China created incentives
for unmarried men to undertake
conspicuous consumption to attract mates.
You know, like peacocks.
We compared the difference in spending
on automobiles between married and unmarried men
in both districts, where the gender ratio
was relatively equal, and districts where
the gender ratio was highly skewed towards more men.
We found that unmarried men spent significantly more
on cars relative to married men
in the highly unbalanced districts
compared to the balanced ones.
Here, even though there was no before and after,
we were able to use a classic diff-in-diff technique.
Here, the treated control group is unmarried men
and the control group is married men.
Instead of happening across time though,
the treatment happens across geographic space.
We don't observe the treated group
before and after treatment, instead we observe them
at the same time, but in places where the treatment
is and isn't happening.
Some were in gender balanced districts,
and some in districts with too many men,
where competition for females is extreme.
Let's wrap up.
We know it is generally going to be wrong
to estimate a treatment effect by comparing outcomes
between the treated and control groups,
unless treatment was randomly assigned.
But comparing the differences
between the treated and controls
before and after treatment eliminates
the influence of any time-invariant confounding variables.
If nothing else is causing the difference
between the groups to change,
this technique, which we call diff-in-diff,
can give us an unbiased estimate of the effect
of the policy or treatment we are studying.
As I look around the room, I see no one has any questions.
I guess that means I knocked this lecture out of the park.
I'll see you soon to talk about
our next technique, propensity score matching.
(violin music)