- Well, hello, everyone.
My name is Jordan Huffaker,
and today I'm gonna tell you
about how we can use crowdsourcing
to detect emotionally manipulative language.
Let's get right into it.
A long time ago, in an ancient America,
it was once true that people subscribed
to a number of name brand newspapers,
such as the Los Angeles Times or The New York Times
or something similar and then it would then
have it physically delivered to their home.
Okay, maybe this wasn't that long ago.
But many still think of this era
of the information ecosystem as sort of the glory days.
The days where we didn't have any of the problems
that we experience today.
But in fact, perhaps those people
who think that are warped by nostalgia.
Because the reality was that this information
ecosystem was still rife with problems
including a sensationalist tabloid media
that resembles a lot of the click-bait
fake news we see today.
Yellow journalism, which was quite
literally misinformation that started wars.
And even coordinated disinformation campaigns
like the Nazi's, the ring.
And I encourage you to see Bytwerk
for full history of that.
And so I just wanna start by contextualizing
us in the reality that this information ecosystem
has always had problems.
And those problems continued in 2020.
And while a lot of the way that we get our
news has shifted online.
We still see a lot of misinformation, hate speech,
and harassment throughout our information ecosystem.
Marwick and Lewis cataloged the different actors
that participate in spreading this kind of information
and that include things like political actors,
such as Russian trolls in Twitter,
financial ones like Macedonian teenagers
trying to generate click-bait fake news for ad revenue,
and even ideological actors who are trying
to push extremist ideas into our discourse in culture.
So while it would be the topic of research
for many years to come from many researchers,
to address all of the problems
in this information ecosystem,
I address in this work one particular
rhetorical strategy that many of these actors
use to amplify their problematic content.
And that is emotionally manipulative language.
Let's take a look at an example of what I mean.
This is a snippet from Tucker Carlson
where he calls Congresswoman Ilhan Omar
"A living fire alarm.
"A warning to the rest of us that we better change
"our immigration system immediately or else."
That's very sinister.
Using these dramatic words and phrases
trying to induce xenophobia within his audience.
And the problem with this kind of content
is that it causes reader, or viewers
who would otherwise gonna be processing
information logically and through slow
and methodical methods, instead it process it
very quickly because they have an emotional reaction.
And they make impulsive decisions as a result
that they may not, otherwise, have made,
such as sharing the content or leaving an angry comment.
And so it would be really useful if we can detect
this kind of language because we can build
interventions that could potentially stop
the spread of this information throughout the internet.
But there's no way to do that yet.
And so the focus of my work was to build
a system that can consistently detect
emotionally manipulative language.
But this is really challenging
because a lot of the content
that we look at looks more like this content.
This is an example of a post by Camila.
She describes the challenges
that she's gone through while immigrating
through the Southern border, the US border.
And it doesn't matter how you explain this information.
It's just emotional and it's inherently emotional.
And so for that reason we call this
intrinsically emotional content.
And the main challenge will be,
how can we disentangle this intrinsically emotional content
from an emotionally manipulative language.
So to get started on that challenge,
we developed a dataset.
We started with news articles
and then we would take them and create a shorter version.
Usually a small text snippet out of that
and then we would then take those text snippets
and create a version that has a little
to no emotionally manipulative language.
And then we would take that snippet
and create another one that has a lot
of emotionally manipulative language.
All while maintaining the same
information content across both snippets.
And so now this gives us a clean dataset
where we have labels with and without
emotionally manipulative language
while maintaining the same informational content
between different paragraph pairs.
And in order to evaluate the validity of our dataset,
we handed these text snippets
to a variety or a two journalists
who then rated them according to a variety
of different aspects including,
which of them has more emotionally manipulative language,
which we found they tend to be able to agree,
which has more intrinsically emotional content,
and which is more likely to be publishable
and reputable news source.
And so now we have a clean classification
of problem that we can try a variety of approaches on.
And so the next step was to try
some off-the-shelf tools.
And that includes things like emotional detection
and sentiment analysis.
We generally found that these approaches
were not quite yet capable of detecting
emotionally manipulative language.
And this should come at no surprise
as these problems, emotion detection
and sentiment analysis, are not the same thing
as detecting emotionally manipulative language.
Also, they were trained on a different
kind of dataset, typically Twitter.
So it's possible if we had a dataset
that was more related to our problem
with better labels, then we could potentially
make these approaches do better.
And one way that we might create that dataset
is to use crowdsourcing.
And so we tried a really simple crowdsourcing approach
where we simply just asked workers
how much does the paragraph
intentionally stir emotion in the reader?
So we would have them rate on a Likert scale
from one to five and then we would average
the responses and use a decision threshold
to decide if that average is above the threshold,
then it had then, we say that the paragraph
has emotionally manipulative language.
And if it's below it, then we say that it does not
have emotionally manipulative language.
This approach is nice
because it allows us to be more aggressive
or more conservative if we want to,
or more aggressive depending on our use case.
But we found that no matter what we do,
the approach did not perform well.
While we could get high recall
if we use more aggressive threshold,
we couldn't get better precision
and that was due to workers conflating
intrinsically emotional content
with emotionally manipulative language.
So let's take a look at an example
to further understand what's happening.
This was an example that I showed you earlier
that workers would have messed up.
They would've conflated
with emotionally manipulative language.
And if you look at this example,
it sort of make sense why this is happening.
If you asked them why does the paragraph
intentionally stir emotion in the reader,
you can see that this is actually really challenging.
It's really hard to disentangle
where that source of emotion is coming from.
Is it from the informational content of the paragraph?
Or is it due to the way that it's presented?
And we know from psychology
that when people are presented
with challenging questions such as this,
they typically substitute them with easier ones,
such as how much emotion does this
paragraph make me feel?
It's much easier and it would produce the wrong output.
And to further explain what is happening here
was look at an analogy.
I use this analogy of an optical illusion
and if you take a look at that bar in the middle
and I asked you is the right side of the bar
darker than the left side
of the bar you may say yes.
But if I remove all that additional context,
you can see that the bar is actually the same
brightness all the way across it.
And so one thing we could do is pull from this analogy
to create an approach that works in our scenario.
So let's look at how I did that.
We start with, we're going back to that
original paragraph that I showed you
initially from Tucker Carlson.
We introduced an approach that we call anchor comparison.
Where instead of asking workers to rate
the amount of emotionally manipulative language
exist in this original paragraph,
we can instead ask them to try to remove it,
to rewrite the paragraph into a version
that has a very little or no
emotionally manipulative language.
And so with this new version
that we call the anchor version of the paragraph,
we can use it as a point of comparison
to judge the original content
just by asking them how much more dramatic
is the original compared to the anchor.
And this approach allows us
to make that comparison and see
that the original has much more,
but in the case that we showed
did not work previously,
workers would only be able to make
minor edits to the original,
such as changing living in fear to just being afraid.
And so when we asked them how much
more dramatic is the original compared to the anchor,
the difference would be very small.
And this would come back as negative.
And so now we have really useful approach
that we call anchor comparison.
But how are we actually implement this in the system?
Well, we could do that through two steps.
If we introduce a text snippet
and we insert it into a component
we call anchor transformation
meant to create an anchor version of the paragraph,
we could then feed that anchor version of the paragraph
into the comparison step which then compares
against the original content into a classification output.
A binary of zero or one.
But how might we actually build
that anchor transformation step?
And so to do that we pull from the state-of-the-art
copy editing and this is in particular
Bernstein Soylent system that find, fix, verify workflow.
And then we add one additional step, the filter step
intended to reduce the amount distortion
that happens at the original paragraph.
And so now we have a useful system
where if we feed in a text snippet,
we can find the portions that have
the emotionally manipulative language.
We can have workers suggest potential changes
to remove that emotionally manipulative language
while maintaining the same informational content.
And then we can have a different set of workers
remove those edits that actually do change
the informational content or distorting it
and then we can have a final step of workers
picking the best edits that remove
the emotionally manipulative language
while maintaining the same information.
And so now the output would be an anchor paragraph.
And we can use that as a point of comparison
to judge the original content.
Again, rating on a Likert scale from one to five
and using that threshold to determine if it does
or doesn't have emotionally manipulative language.
And so while I don't have a time to tell you
about exactly how we evaluated the whole system.
I'll give you the high level findings.
And that is that if you use iterations
you can improve the recall.
Or you can use the agreement threshold
to reduce the conflation within the filter step.
So now I've showed you how we can detect
emotionally manipulative language.
But let me take a step back
and explain the general class of problems
that we manage to solve by using anchor comparison.
And to do that, let's look back
at that Tucker Carlson example one final time.
So if you look at this example
and take a look at just the first phrase,
"A living fire alarm."
This is one of the phrases that I've said
that could be rewritten to remove
the emotionally manipulative language
while maintaining the same information.
And the reason that is possible
is because it's overlaying multiple
messages on top of each other including one
where he's saying that she, being Ilhan Omar,
"Is an example of how immigrants
"are ungrateful and quick to criticize."
This is potentially a problematic stereotype,
but it's made much worse when we add
an additional message of the blasting fire alarm
that's warning us that if we don't act immediately
that the whole nation will burn,
or our culture will be set on fire.
It's much more dramatic and it induces
a lot more urgency to do something.
And so we call this kind of phrase
that has multiple messages overlaid
on top of each other a social reference.
And this is the broad class of problems
that anchor comparison is made to solve.
Other examples would be terms
such as dirty crime-ridden cities,
which has the literal meaning
of dirty streets and alley ways
as well as the racial connotations.
Also, migrant caravans which has the meaning
of a line of cars as well as Middle-Eastern connotations.
But in general, we say that social references
refer to language that invokes connotations
by overlaying parts of social and cultural contexts.
And this is what anchor comparison is meant to solve.
So today, I've told you about a new crowdsourcing
approach that we call anchor comparison.
It is the first crowdsourcing approach
that can detect social references
while mitigating conflation error.
I thank you so much for you attention.
I thank our sponsors for the support
and I encourage you to see
the full paper for more details.