♪ [music] ♪
- [Josh] Welcome back.
Today we continue our pursuit of causal knowledge.
Recall that private university alumni earn wages 14% higher
on average than the wages earned by public university grads.
Does that mean private university education
causes your wages to go up?
As with most of the questions we ponder,
the facts are not in dispute,
rather it's the interpretation that's contentious.
- [Narrator] Let's compare private school graduates
with those who attended public schools.
Private university grads differ in a number of ways.
For example, they have higher SAT scores.
Attendees of private universities score 120 points higher on average.
These SAT stars sport orange sweatshirts.
Private university grads also come from wealthier families --
13% higher than public university students.
The rich kids have green pants.
It would seem that public/private comparisons
are not apples to apples.
Perhaps their 14% wage gain is caused
by pre-existing differences in earnings potential
rather than by private university attendance.
- Like many who have walked before us
in search of causal knowledge,
we are waylaid here by selection bias.
- [Narrator] Selection bias misleads us
into interpreting naive comparisons as causal effects.
- [whisper] Come with me.
- [Narrator] Here we see selection bias tricking us
by directing traffic.
Those with higher test scores go to the left
towards private universities.
Those with lower family incomes go to the right
towards public universities.
Public/private comparisons have causal force
only when the groups compared are otherwise identical on average.
For then, we can happily say ceteris paribus.
But private schools are typically more selective and more expensive
than their public university counterparts.
So those going left are not comparable to those going right.
This is how selection bias bewitches us.
- Although colleges really do select their applicants,
the term selection bias refers to any comparison plagued
by systematic differences between groups
other than the difference we're focused on.
When the groups being compared differ in many ways,
we've lost ceteris paribus.
Selection bias is the principal enemy facing metrics students
and metrics masters alike.
Our five most important weapons in the fight against it
are the Furious Five of Econometrics.
- [whisper] The Furious Five.
- [Josh] Selection bias is insidious and pervasive,
but our weapons are powerful,
for we do not need to ensure that the individuals
to be compared are identical.
We don't need virtual clones.
Rather we need only ensure that the groups compared
be the same on average.
Our most powerful weapon, strong and dependable,
is random assignment of group membership.
Imagine a secret experiment in which applicants
to public and private colleges
are randomly assigned to attend one or the other.
Seems only fair.
And maybe we'll learn something from this too.
In the interest of science, I have proposed such an experiment
at MIT where I teach econometrics.
I'd like to replace our skilled but well-paid admissions officers
with a coin toss.
Random assignment of college admission ensures
that when it comes to cross school comparisons,
ceteris is paribus on average.
Unfortunately, for science, I have not yet convinced MIT admissions
to replace its staff with a stack of pennies.
- As we'll discuss later, random assignment is often impossible
or impractical so we must look for practical
and inexpensive strategies that have the same
ceteris paribus inducing power as random assignment.
Kamal, where should we look?
- [Kamal] I don't know.
If we could somehow control for...
- Correct.
- [whisper] What?
- Metrics masters are control freaks.
We implement statistical strategies
that make the groups choosing different paths as similar as possible.
Rather than simply comparing wages of public and private alums,
we look within sets of alumni
that have similar ability and background.
Within these sets, we make public/private comparisons
but not across them.
This strategy moves us one giant step closer to ceteris paribus
and apples to apples comparisons.
Let's look again at the Furious Five.
- [whisper] Furious Five.
- [Josh] Our principal tool in the struggle
for control is regression.
Regression is a neat way to compare two groups
while simultaneously holding many differences
between those in the groups fixed.
- Do regression estimates show that private university education
is worth paying for?
- [Man] Good question.
- [Narrator] Using regression to adjust for applicant ability
and family background and a few demographic characteristics
like race and sex reduces the private college premium
from 14% to 9%.
- Nine percent still seems pretty good.
- [Man] Nine is legit.
- [Narrator] But do we have true ceteris paribus?
Camilla?
- [Camilla] Um... I'm not sure we controlled for everything,
Maybe private school students are more ambitious or smarter
in ways not fully captured by test scores.
If so, comparisons are not apples to apples
even after the adjustments you speak of.
- Worrying indeed.
The possibility that the variables we've adjusted for
using regression do not fully account for group differences
is called omitted variables bias or OVB.
- [man] That's bad.
- [Narrator] OVB is selection bias in a regression context.
- We suffer the effects of OVB when the regression we've got
is not the one we want.
The regression we want, the regression of our dreams,
has more and better controls than the one we've got.
How can we control for something like ambition?
Is there an ambition index?
It's not easy to produce ceteris paribus comparisons.
Regression is a tool.
It's not magic.
Yet sometimes the results unearthed by this tool are striking.
Masters of Econometrics professor Stacy Dale
and Alan Krueger faced the challenge of selection bias and OVB.
In a famous academic study, Dale and Krueger controlled
for the many possible differences between students who've attended
different types of schools.
They had the insight that selection bias in this context originates
in two forces: student ambition and college opportunity.
Most students have a pretty good sense of their own aptitude,
inclination, and motivation for school work.
These forces are summarized by the type of schools they apply to.
At the same time, college admissions offices invest
massive hours and energy into ascertaining
who will succeed on campus.
They evaluate and select for academic ability and college commitment.
What if we compare the outcomes
of those who had the exact same acceptances and rejections?
Compare two high school students, Maya and Mariana.
Both admitted to UNC and Duke but not to Yale.
Similarly ambitious and judged similarly capable
by these three schools' admissions offices,
Maya opts for Duke because a friend is going there,
while Mariana heads to UNC in Chapel Hill.
- [woman] Come on, Heels!
- [Josh] Maya and Mariana are not clones of course,
and they've chosen different types of schools for personal reasons.
But otherwise, they have a lot in common.
The personal factors that drive them to choose between the schools
on their menu might not be closely related
to their future earning power.
Pooling many such comparisons moves us one giant step closer
to ceteris paribus.
Remarkably, a regression model that controls for the sets of schools
to which applicants have applied and been admitted
shows almost no difference in earnings between public
and private graduates.
In other words, averaging over a large number
of Maya to Mariana type comparisons,
the private school premium falls to nothing.
Maya might have enjoyed her expensive Duke education,
but on average at least students like her will do no better
in the labor market than comparable public school peers.
That's quite a change from our initial 14% earnings gap
favoring elite school alumni.
Regression has the power to turn a clouded statistical night
into a clear causal day.
But you'll need to know a little more before you can regress
with skill and confidence.
- [Narrator] You're on your way to mastering econometrics.
Make sure this video sticks by taking a few quick practice questions.
Or, if you're ready, click for the next video.
You can also check out MRU's website
for more courses, teacher resources, and more.
♪ [music] ♪