My name is Betsy Levy Paluck and I'm a
social psychologist at Princeton University.
My question is, what works to
reduce prejudice in our worlds?
So, how can we reduce expressions of
prejudice, derogatory and hate speech,
discrimination and violence, and for
many around the nation and the world
this past summer's Black Lives Matter
protest brought this question into focus.
We've also been asking this
question in response to treatment
of migrants and refugees, and of
Asian-Americans.
What can we do?
How can we reduce prejudice?
So, I'm a social psychologist.
I came to this question in my
third year of graduate school.
When I was the teaching assistant
for my mentor, Donald Green,
who was a political scientist.
And he was teaching a class called
Political Intolerance and Hate Speech.
And Don asked me to help him find papers
for the last meeting of the class,
which would be devoted to solutions.
He asked me to look at the psychology
research literature, where prejudice
reduction is a major topic of study, to
find research on solutions that had been
demonstrated to change behavior in the world,
using really rigorous research methods.
And I thought this would be an easy task,
but with these two conditions,
change behavior in the world one and
examining the solution with research
methods that could show that the
intervention had causal impacts.
I came up short.
This failure changed my career.
I spent the rest of my time in graduate
school trying to find every single study of
prejudice reduction that had ever been done,
in any field of study or in the policy or
non-profit world, published or unpublished.
And Don and I wrote about my
collected findings in 2009.
Our bottom line was that we didn't
know what were the most effective
tools to reduce prejudice because
the research was insufficient.
To understand what kinds of interventions
could reduce prejudice in the world,
we need randomized controlled experiments,
the kind that we've been relying on
the past year to know whether our
vaccines are safe and effective.
Experiments can tell us if a
program, an encounter, a training,
a campaign, an exercise, whatever
tools we're using to reduce prejudice
are causing prejudice reduction.
We wouldn't take vaccines based on studies
that just ask people for their impressions
of how they had changed after taking the
vaccine, or their thoughts on the experience.
So we need evidence of how it caused change,
and we need to take prejudice, racism,
religious hatred, ethnic dehumanization,
prejudice against those were not able-bodied;
we need to take it as seriously
as we do the pandemic.
So, we wrote this review in 2009 as a
challenge to the field and the huge
private and non-profit industries of
bias reduction, sensitivity training,
diversity training, educational
and media programming and so forth.
And we called for serious
research and experimentation.
And from that moment on I set about
doing that work myself, as well.
So two years ago we realized that
soon it would be a dozen years since I had
searched the research literature for evidence
of what we know about reducing prejudice.
And in the intervening years, we
had conducted our own experiments
on prejudice and violence
reduction, observed our colleagues,
and I had been able to research some
of these projects with my own students
after I had become a professor at Princeton.
So our question was, what had
happened since that time?
And as the clock tick toward 2020,
we searched the literature again.
And we wrote a paper that
asked three questions.
So first of all, what had
happened in research on prejudice
reduction in the last dozen years?
Second, what did the research find?
How powerful were these interventions?
What was their average effects in the world?
And then, third, can social science
answer the public's call to
reduce prejudice in the world?
So I'm going to tell you
the answers right now.
First, there's been a sharp uptick
in prejudice reduction research.
This is great news for the research endeavor.
Some of these studies are extremely exciting.
They're very strong research and they're
looking at innovative programming that
we should know about and learn from.
And then second, the typical
type of prejudice reduction
intervention takes a particular form.
It's brief.
It's an individualized experience.
Third, some of our most popular
ideas for reducing prejudice have
very little evidence behind them.
So, things like implicit bias training and
diversity training overall. And then fourth
and finally, the group of studies that are
the strongest and the most careful, they show
only very modest reductions in prejudice.
So what does this mean for our efforts
to reduce prejudice in the world?
I want to describe these
findings in a bit more detail.
But what I want to say first is that
they lead me to a different conclusion
compared to the one that I reached as a
graduate student. As a student looking
at our studies of prejudice reduction, I
thought we just needed better research.
But now with some time passing, I think that
we actually need to rethink our approach
to how we try to reduce prejudice in the
first place, not just better studies to
understand the impact of what we've done.
And specifically, most prejudice reduction
programming that we've been able to evaluate
up to this point is aimed at changing our
individual ideas, experiences, and goals.
And the new approach,
I think we need to take,
I think we need to focus on
and put our energies behind, is designing
and evaluating prejudice reduction
programming that aims at our collective
ideas and our collective experiences.
So let me explain a little
bit more how I reached this
conclusion and then what it means.
So first the uptick in research.
Over the past 12 years we found about
481 randomized controlled experiments
testing various approaches to reducing
prejudice, reaching in the last year of
our review, about 60 experiments per year.
This is great news, especially
because some of these experiments
are enormously enlightening.
And I want to tell you about some of them.
These top experiments are the ones
that were testing programming out in
the world that was aimed at people's
social and political realities in
addition to their psychological biases.
And they used excellent research methods
to test their impact, really large samples
of people who were involved, cutting
edge statistics, and they made their
process and their data public so that other
researchers can see how they'd done it.
So for example, two separate studies
tested the idea that bringing people
together to cooperate with one another
for a common goal could reduce prejudice.
Both were conducted by ambitious PhD
students who looked to sports as
an arena where cooperation is
particularly compelling and powerful.
So in Northern Iraq, Salma Mousa
tested whether Christians and Muslims
brought together in a local soccer league
would be more likely to interact, form
friendships, patronize one anothers
businesses, and even shift some of their
policy attitudes after playing on the
same team or an opposite teams.
In India,
Matt Lowe tested a similar
question, bringing low and high
caste men together to play cricket.
So whether sports can bring us together in
our daily lives as an age old question, and
yet they were making new progress on it.
This was one of the first times it
had been tested with scientific rigor.
And they found modest and mixed effects.
So, people in their studies were more
likely to interact with one another and
even collaborate outside of the league.
But these effects had limits and
there was no impact, for example,
on their policy attitudes.
Another study, that was excellent, tested
whether confronting people online when they
use racist language, worked to reduce the
expression of racist slurs, specifically
using the N word against other Twitter users.
So in this study, another doctoral student,
Kevin Munger tested whether people from
one's own racial group and whether people
with many followers versus few followers,
on the social media platform, were more
effective at discouraging racist language.
And he tracked whether this
impact endured over time or not.
The answer was that white men with lots of
followers were most effective at discouraging
other white men from using racist language
and that the effect lasted for weeks.
And then some studies were heroic because
they studied programs that many of us
find ourselves in, but programs about
which we surprisingly know very little.
So in one experiment, testing an online
diversity training, Edward Chang and
his colleagues studied an international
corporation that used an online diversity
training and they tracked employee
attitudes, but also their actual behaviors,
like whether they supported colleagues
who were underrepresented at the company.
They found almost no effects of the training,
except for effects on women and people of
color at the company, who became even more
supportive of others following the training.
So these are some examples of the
outstanding studies that made real
progress on our questions about what
works to reduce prejudice. The typical
types of studies we found, however,
used very different techniques.
For example, the most popular approach
that we found over the past dozen years
is one that didn't even exist when I was
doing this review as a graduate student.
This approach is about contact with members
of other groups, but not actual contact
like in the soccer or cricket leagues.
In these studies, investigators ask
whether thinking about your friends who
are connected to members of other groups,
or just imagining having contact with
other groups could reduce your prejudice.
The next category of training modules
used emotional and thought regulation
strategies, such as empathy, perspective
taking, or rethinking stereotypes
as a way of reducing prejudice.
And finally, the third most popular
category of intervention was
also a bit of a thought exercise.
It was one in which people are
asked to rethink the boundaries
of groups to be more inclusive,
like recategorizing immigrants as
neighbors in your mind, thinking about
them with different categorical schemes.
This figure demonstrates how
the average prejudice reduction
strategy works at the moment,
at least the kind that are allowed
to be evaluated by researchers.
The typical strategy is to reduce
prejudice reduction through
an individual's mental life.
To try to change expressions of prejudice
by changing the way an individual
thinks or feels about the group.
Additionally, we noted that 76% of all
interventions and programming could be
characterized as "light touch" programming.
This is a term that means the
intervention was easy to implement, brief,
and cheap. And this might be really
great for spreading the programming
across large amounts of the population.
But then the question is, is it working?
What is the average effect
of all of these programs,
according to this research?
Looking at the best of them,
the experiments of the highest
quality, the impact is very modest.
It's a standardized effect of 0.18,
and here's how that translates.
Say I measured your prejudice on a 100 point
scale, where zero means you're very prejudiced,
you feel very coldly toward another group.
50 means you're neutral and 100
means you have no prejudice.
You feel very warmly,
you love the other group.
So, let's take a person who
isn't super prejudiced,
according to this scale. They're at a
40. On average from all the work that
we found that was evaluated by the very
best research, the interventions would
move that person from a 40 to a 44.
So, we might argue that this is not
the change that we need to see in
the world, but maybe it's okay.
Maybe it's encouraging if we, especially,
if we repeat the programming and
these small changes add up over time.
The other problem is, is that
we have no evidence that this
additive process is happening.
So very few researchers are doing
this kind of long-term research to see
whether repeating these kinds of
interventions helps the 40 to become
a 44 to become a 48 and so forth.
There are other really important
holes in this research.
For example, there are only two high-quality
studies of diversity training and very
few studies of whether implicit bias
training works, or if implicit bias
changes following any kind of training.
And what this means is that two of
our most popular responses to racist,
sexist, or other kinds of prejudice
events in the world, to give people this
kind of training, has no support in the
research as an effective tool for action.
Perhaps some of it is effective, but
for several reasons, including that
many of these trainings are proprietary
businesses, many companies refuse to
open their doors to researchers when
they are engaging in these trainings,
we don't know if they're working
and even if they're
having a negative effect.
As I mentioned, I used to think that
better research would help us to answer
this question of what reduces prejudice,
because there are many good ideas out there.
But this present research that we just
completed has left me facing the possibility
that better and better research on our
current popular solutions will only show
us that the programming is not doing much.
And that's what this figure shows.
So, as the errors in the research
gets smaller, the size of the impact
of the program gets smaller, too.
So these data suggests that if we keep
using the same ideas, our improved
research is going to show us that
we're not changing anything at all.
That the impacts of the average
intervention is close to zero.
So what should we do?
I think that we need to change the
way we think about our programming.
So currently many of these programs,
the ones designed by researchers and
the ones designed by practitioners,
they think about prejudice as a
problem inside of our minds, and
people design programs to address
individuals and their individual
thoughts and feelings and experiences.
And of course, we all know that prejudice is
something that pervades society, is a social
phenomenon, is present in our structures.
And so, I think that we need to start
designing our interventions in a
way that reflects this knowledge.
I think we need to start designing
programs that change structures in society,
as a way of changing our individual
thoughts and feelings and experiences.
It is still really important to
change individuals, as a psychologist
I feel that, and I see evidence of
that. But we also have evidence that
individuals are changed by their contexts.
Maybe less
so the other way around. It's very hard for
individuals alone to change their contexts.
So, what do we mean then about changing
structures or changing our collective
experiences as a way of reducing
expressions of prejudice in the world?
Of course, by structures
I mean things that we all recognize,
like institutions, rules, leadership.
An example I use sometimes, is that
traditionally our approach can be likened to
studying a single training at a company,
a corporation say, a white fragility training.
And then the research question
becomes, a few years down the line,
do we see more black board
members at that company?
The reverse, what I'm
suggesting here is that,
we change the representation
and increase the number of black
board members on that company.
And then a few years down the
line, see whether white fragility
has shifted at that corporation.
But also, I think that we should
think about the kinds of ways that
we can change collective experiences
with less formal structures.
I think this also matters with
pop culture, with social media
movements, with collective action.
These tools of collective change also
affect how larger groups of people think
about things together and affects how they
perceive the standards of their community.
Because here's something I've learned
about prejudice along the way, from my
own research, we know that the expression
of prejudice is often held back.
If people think that those around them,
their community don't support prejudice.
On the flip side, hate crime research
shows that people
are more likely to commit hate crimes
when they think that their community
supports them and their views.
So we should design prejudice reduction
interventions according to these insights
about prejudice, rather than aiming for brief
individualized treatments, we should use
tools that change our collective experiences,
our perceptions of community standards.
We need leaders, public statements,
rules, and collective attention and
movements. Our data from this review,
I think it provides a mandate to think
big, to think ambitiously. The data say
that we can't just train away prejudice.
It is up to us to think big, and this
insight will define the next stage of my
career, but I hope also the next stage
of innovation, practice, and research
around reducing prejudice in the world.