Hello, everyone, my name is Dr.
Miriah Steiger and I'm a director of
Global Research for Wellness and
Resilience in the Content Moderation
industry.
In this research article, we
surveyed the psychological risks
that content moderators face in the
course of their profession and
describe a range of strategies for
reducing the emotional labor
involved in this work.
Content moderators play a vital role
in the functioning of social media
platforms today by safeguarding
users from exposure to inappropriate
content.
While one could ideally rely on
machine learning to automatically
detect and filter out inappropriate
content, human interpretation
is often required due to
high accuracy requirements and
cost of errors and the subjective
nature of the task and complex ever
changing moderation, policies
and forms of offending content.
Prior literature has estimated that
over 100,000 paid human
moderators are staffed globally,
spanning internal reviewers,
contractors from third parties
and outsourcing to online labor.
Former research has revealed that
prolonged exposure to some forms of
offending content may
impair moderator psychological
functioning, which can be
further exacerbated by metric
demands.
Potential adverse health effects can
include alterations to beliefs or
worldview, interpersonal
conflict and stress related
disorders just to list a few.
And given the necessity of human
workers in performing this task,
we adopt a social justice
perspective to motivate
the need to support such operations.
Therefore, we make three primary
contributions in this publication.
To begin, we discuss
moderators psychological
symptomology, drawing
on the occupations involving
secondary trauma exposure to
further guide understanding of those
symptoms and support mechanisms.
Subsequently, we review a
combination of both programmatic and
technological support for
confronting these challenges,
adopting the lens of the
preventative care model and
discussing treatment options,
and examining ways to measure the
effectiveness of such applications.
Our final contribution includes
recommendation of best practices
as well as directions for future
research.
By applying the model, we begin
with discussing primary
interventions which apply
strategies that limit symptom onset
or prepare an individual to manage
potential adverse situations.
For example, while the first
line of defense includes
technological filters to limit human
exposure, we noted
how challenging this detection is
to automate with sufficient
accuracy.
Therefore, programmatic
interventions are necessary form
of additional support to prepare
moderators by use of pre-employment
risk disclosure and resilience
training prior to
any and immediately following
exposure to content.
Second level interventions are
essential as moderators will
inevitably encounter triggering
content.
The fundamental purpose of secondary
care is to reduce symptom severity
following onset, thereby
returning to a state of stability.
Technological approaches such as
alterations to image size and
blurring or muting videos
represent secondary support
and in terms of programmatic aid,
ongoing resiliency, skill based
training for workers is recommended.
And finally, tertiary interventions
provide treatment for following the
manifestation of a psychological
disorder, which requires
an evaluation by a licensed
professional,
those meeting severe criteria
should be referred to community
resources as they
fall outside the workplace, setting
scope of practice.
If appropriate, tertiary
technological interventions such as
virtual reality can be beneficial.
They've been studied to treat stress
related disorders by increasing
mindfulness and producing lower
levels of anxiety, sadness and
anger.
Across technological and
programmatic services, we propose
a variety of directions for future
work with the potential to better
protect content moderators
as they safeguard social media
platforms.
Our authorship in this work spans
academia and industry, bridging
expertize and technology, social
science and clinical mental health.
We come together and form an
alignment to support the well-being
of content moderators.
We advocate for continued growth of
academic industry partnerships,
engaging with these issues,
advancing both scholarship and
practice.
We thank you for your attention and
encourage you to reach out to us
with any questions.