Hi, everyone.
I'm Silvia, welcome to my talk.
Our work studies the effect of sociocultural variables
on sarcasm communication on social media.
Sarcasm, a form of non-literal language,
is a contextual phenomenon.
And qualitative research in psycholinguistics
points out significant influences
that speaker and listener's sociocultural backgrounds
can have on whether sarcastic communication is effective,
that is whether the message intended by the speaker
is accurately perceived by the listener.
These studies suggests that sociocultural information
should be considered by automatic sarcasm detection
systems.
However, there is a shortage of quantitative evidence,
especially in the context of OSNs regarding
the specific variables that should be considered
and the degree of influence that each variable has.
Our work aims to fill in this gap.
Specifically, we formulate three research questions.
First, is the effectiveness of sarcastic communication
influenced by whether the interlocutors have
similar backgrounds?
Second, which sociocultural variables
have the most influence?
And third, earlier research suggests
that sociocultural variables may only
have an influence when other contextual cues to speak
our intent are missing, does this also
apply to online sarcastic communication?
To answer our questions, we first chose a set of variables
to investigate by looking at both psycholinguistic studies
of sarcastic communication and linguistic theories of sarcasm.
We chose age, gender, location, and English language
nativeness.
Choice motivated in section 3 of the paper.
Next, on a crowdsourcing platform,
we asked Twitter users to provide
sarcastic and non-sarcastic tweets that they themselves
had posted in the past.
Tweet labels were thus implicitly specified
by their authors.
We refer to these labels as intended sarcasm labels.
The output was a data set of tweets,
each tweet with its intended sarcasm label.
Next, we formed several treatment groups
of third party annotators, each group containing
the representatives of one sociocultural background.
And we tasked each group with relabeling our tweets,
referring to any such label as a perceived sarcasm label.
Finally, we compared intended and perceived labels
from across treatment groups.
And with this methodology in mind,
let us look at how we answered each research question.
Our first one was, is the effectiveness
of sarcastic communication influenced
by whether the interlocutors have
similar sociocultural backgrounds?
To answer these questions, we considered two treatment groups
of third party annotators.
The first one had annotators with the same sociocultural
background as the speakers.
The second one had annotators of opposing backgrounds.
We then compared perceived labels
from each treatment group with intended labels using F-score.
And we noticed that the first list in the group
was significantly more able to recognize sarcasm
as intended by the speakers compared to the second.
So this suggests a positive answer
to our first research question.
Let's now look at the second question,
which social cultural variables have the most influence?
To answer this, we collected perceived labels
from four further treatment groups
of third party annotators.
Each group had annotators of the same background as the speaker,
except for flipping one variable of the investigation.
So for instance, list equals speak minus age,
denotes a treatment group where the listeners have
the same background as the speakers,
except for a different age.
We found age to be the most influential,
followed by English language nativeness and gender.
Country was not statistically influential.
Our final research question was, can contextual cues
alleviate social cultural differences?
This time, we asked listeners to consider
not only the text of the tweet that they were annotating,
but also surrounding tweets and any contextual information
that they might find on the Twitter profile
of the author of the tweet.
And we found age to still be a significant barrier.
The impact of all the other sociocultural variables
investigated was made insignificant.
So how are these results useful for future work?
First, we provided quantitative empirical evidence
that sociocultural information should
be considered in the design of future social analysis tools
that either study sarcasm or look into related phenomena
where sarcasm may have an influence,
such as the expression of sentiment, emotion, or hate
speech.
We also made public the data set of tweets annotated
for intended sarcasm, as well as the code
used for surveying and statistical analysis.
In future work, we could study the potential interactions
between sociocultural variables.
Thank you very much for listening.
I hope you are now motivated to read our paper.
And I look forward to the Q&A session.