In this short talk I'm going to share our recent work
on appeals of content moderation decisions.
When users experience content moderation,
platforms increasingly allow them to appeal the decision.
However, recent work on users who self-reported experiences
with content moderation, found that they describe it
as speaking into a void.
In our work we go beyond self-report data
to experimentally test whether different forms of appeals
can improve users' experiences.
Our between subjects design uses a scenario
where participants' Facebook accounts have been suspended
for spreading misinformation.
The experimental conditions vary whether and how
they can appeal that decision.
The intervention tested four versions of appeals.
A no appeal baseline, a written appeal evaluated by a person,
very similar to systems that exist now,
a written appeal evaluated by an algorithm,
and finally a behavioral appeal, inspired
by prior work, that tries to let users show rather than tell
that they're a good user through their behavior.
After seeing the scenario, participants
who are allowed to appeal write their appeal.
But then all participants are told that their appeal failed
and their account was incorrectly suspended.
We did this to ensure that it was not just
outcome favorability that improved perceptions,
but actually accessing procedural control or voice.
All participants then answered survey questions
about fairness, accountability, trustworthiness,
and feelings of control.
As well as open-ended questions about their perceptions
of the decision making.
Participants for the online experiment
were recruited from Qualtrics to be representative
of the US population in terms of age, gender, education, race,
and household income.
After cleaning, we analyzed data from 449 participants,
182 for the quantitative and an additional 267
for the qualitative results.
So we'll start with the quantitative results
for our first research question.
How does adding appeals change users' perceptions
of the fairness, accountability, trustworthiness,
and their feelings of control in an algorithmic decision making
system?
We find that despite the existence of this mechanism
for control, users do not find the existing appeal
process fair, trustworthy, or accountable.
Nor the variations that we developed to try to improve it.
And while none of the differences
are statistically significant, it
was in fact the no appeal condition
that users thought to be the most trustworthy, accountable,
and so on.
So what this indicates is that the current appeal
process falls short of addressing users' needs.
To answer our second research question, what
do users choose to contest, we analyzed the text
that users write in their appeals
and the open-ended questions.
We find that users disagree with many parts of automated content
moderation, from the fundamental idea, to how it's performed,
to the automation of the process.
For example, one participant argued
that automation can't adapt to different circumstances,
saying, Facebook banned me for this very thing.
Because they said the money in the board game
I was trying to sell was counterfeit.
It was Monopoly money, it's not supposed to be real money.
Other common themes that emerged argued
with the opacity and the inconsistency of the process.
Asking for explanations and examples,
and comparisons for how others are treated.
Saying, it is inconsistent, arbitrary and never,
ever explained properly or understandably to the user.
They do what they want when they want.
And people took these accusations very seriously.
For many, 45 participants or just over 10%,
they wanted to provide context, not just about the post itself,
but also about themselves.
This often led participants to make virtue ethics
style arguments that they're not the kind of person who
shares misinformation.
Saying, I've never knowingly misled anyone in any way,
or I'm honest and not the type of person
who would do this at all.
And unfortunately for some users,
they say this would lead them to leave the system.
In part because they don't think they
have control over the decisions.
Saying, I would get off Facebook because it's harder
to prove it's not true than it is to not try to fight it.
And at the end of the day, Facebook does what it wants.
We offer a number of suggestions for improving
the design of appeal systems for content moderation,
like scaffolding the appeal and unburdening the user.
But argue that fundamentally, a framing focused on punishment,
eliminates the potential for content moderation
to serve educational functions.
And in the tradition of human computer interaction,
to allow users and the system to recover from error.