- So thanks.
Hi, I'm Max Curran.
I'm a Ph.D candidate at UC Berkeley School of Information.
I'm here today to present a project titled
Understanding Digitally-Mediated Empathy,
An Exploration of Visual, Narrative,
and Biosensory Informational Cues.
And I worked on this project with my co-authors
Jeremy Gordon, Lily Lin, Priyashria Sridhar
and my advisor John Chuang.
I'm just gonna start with some motivation
and background for this project.
So there are clear reasons why
we communicate through technology.
Chiefly among them, it eliminates the need
to be co-located with the person you're talking to.
It uses one to many communication
and also you have increased control
over self-presentation when you are
communicating through mediated channels.
In particular, you can be anonymous online
which is really difficult to do when you're in person.
This communication takes many forms.
One to many posts on things like social media platforms,
one to one messaging and messaging applications.
Often one of the downsides of mediated communication
is you have a reduced set of non-verbal and verbal cues
to understand how someone is feeling.
So often times we'll substitute things like
text, emojis, photos, links, video
to kinda get across our point.
An interesting practice that's
kinda surfaced in recent years
is this idea of live streaming, either streaming,
in one example in this screenshot is a gamer
streaming kinda their physical appearance
while they are playing a game.
Another one is things like reaction videos on YouTube
where someone will watch something like a movie trailer
and you can see their face react
and you kinda maybe understand how they're feeling that way.
These are kinda, I think, ways to kinda reclaim
some of these cues that are
typically lost in mediated communication.
But another interesting direction that's kind of,
another cue that's been explored recently in HCI research
is social biosensing.
What is biosensing?
Biosensing is the capture and interpretation
of physiological data generated
from our bodies and behaviors.
It includes things like heart rate
and also something called electrodermal activity.
It's changes in the moisture level of the skin,
indicates kind of like how excited you might be
or a degree of arousal.
And there's examples of things like this that already exist.
So like the Apple Watch for example.
You can share your heart rate.
There's other really interesting research projects
that are exploring kind of like
what is the meaning that's interpreted
in these when you share signals like this?
So work by Howell in 2016,
she explored these color changing fabrics on t-shirts
that depended on user electrodermal activity
during these intimate conversations.
And Fanny Lou has done some really interesting work
with x persivity with biosignals,
or biosensing data, like EEG patterns,
or creating these kinds of smart watch apps
that you can share a little animated figure
that's based in part on your biosignal data.
So there's, the findings from a lot of this work
is that these signals can increase perceived closeness,
they're kind of lightweight forms of social interaction,
increased sense of intimacy,
and they can enhance expressivity depending on
how they're presented.
There's a clear relation I think to empathy,
in this work, and I don't think it's been explored
with that lens yet so that's kinda one of the
impodices behind this project.
And there's room for experimental exploration of these,
of this work to compliment the existing work
that's already out there.
So what do I mean by empathy,
empathy is a term that's used a lot in
day to day discourse and it's also used a lot
in research, it has a long history of research,
of research and social and psychological,
or social psychology and psychology.
So I'm gonna put a definition up here that I'm gonna use
in this talk which is the understanding and responding
to the mental and emotional states of others.
In an empathic situation there's typically a target
of empathy, someone who is sharing an experience
or experiencing something in the moment,
and an observer who is watching that person
and trying to empathize with them,
or trying to understand how they're feeling
or feel the same things.
There are a couple different components of empathy
that have been talked about in psychological research,
one of them is cognitive empathy,
that's the perception of the mental states
of another person, which is kind of the focus
of this study going forward.
There's also affective empathy which is the feeling
of someone else's feeling, not just knowing what
they're feeling but actually feeling that,
those emotions yourself.
And recent research has suggested that empathy has
a motivated nature, empathy's not something that
we're either born with or, either you have it or you don't,
but it can actually change and in a situation
you can be motivated to empathize more or less.
And one interesting kind of component of empathy
as well that's come up in recent work is
mind perception, which is kind of a precursor
to empathy in the first place.
So you have to kind of ascribe, mind perception
is the process of ascribing mental capacities to someone,
kind of perceiving them as another person
who experiences emotions, has their own perspective,
which is necessary before having
any sort of empathic experience.
A 2011 med analysis of trait empathy
concluded that there's been a decrease of 40%
in trait level empathy in U.S. college students
since the 1970s, this was self reported empathic concern
in respect of taking components of empathy.
And at a time when there are narratives and social
political division, there's narratives that
social and political division is growing
that makes someone feel that,
and we're kind of questioning technologies role
in this process, it's an interesting time to think about
how technology might afford or discourage empathy.
And we're also re imagining current forms of social media,
things like social VR, it's kind of an exciting time
to think about what new forms
these communications could take.
So talking about what we did for this study,
we had two broad research questions.
The first, not broad, they're pretty specific actually.
(laughing)
The first one is how do different types of
informational cues about an anonymous target
affect observers accuracy rating the targets
state and the observers general feelings of empathy
toward the target.
So this first research question we addressed
experimentally, we had three different types
of informational cues we were testing.
One was a narration, kind of a description of how
the target's feeling, one was EDA, the electrodermal
activity I mentioned earlier, versus a control condition
that was just the content itself which,
I'll kind of show what this means later on.
We hypothesized that narration would have kind of the,
the highest accuracy, and followed by EDA because
it's some information but not very clear what it is,
followed by the control condition where you don't
get any extra information.
And then the second research question was more qualitative,
what are the observers qualitative impressions
of different informational cues.
How to describe they're strategies for empathizing
with the target when given different informational cues.
And we address that one with some semi searching interviews.
So what did the methods of the study look like?
This is kind of a scary slide
but I'll try to go through it piece by piece.
So there were two phases to this study, one was a target
phase where we kind of collected information
to produce stimuli for the observers.
So in this target phase the target experienced
a short film in virtual reality called Pearl.
It's part of the Google's spotlight series,
and it's an animated short film describing,
it's described as a girl and her dad criss cross
the country in their hatchback.
We used VR because it was a way to have, for the target
to have an immersive emotional experience,
and capture things like their first person perspective
recording what they're looking at, their EDA,
in kind of a lab setting, so it was a nice medium for that.
So as I mentioned we have collected their field of view
recording we were able to record what they were looking at
during the VR experience.
We also collected their electrodermal activity
while they were in the VR experience.
And after they did the VR portion they watched the video
that we recorded, the field of view video
and they narrated over it and described how
they were feeling when they were in the VR experience.
And then they watched that video again and provided
dial ratings on a dial continuously while they
watched it on a valance scale , so from very negative,
very positive, how were they feeling
at each given moment of the video.
And then we did some overall emotion ratings at the end.
How they were feeling generally.
And so we used that information to generate
stimuli for phase two which is the observer phase.
So in this phase we had three different conditions,
so the observers either watched the,
they only watched the 2D version of the recording.
So they either watched just the field of view
by itself, the field of view with narration subtitles,
which I'll show you a picture of in a moment,
or the field of view with the eograph.
And while they watched it they were asked to do
a similar rating test continuously on a dial,
how do you think this person is feeling in the moment,
in the given moment that they're in.
And then we also have them answer some questions
afterward about how the kind of general feelings
of empathy for the target,
have them rate how difficult the task was
and how confident they were in their ratings.
And finally we did a 20 to 30 minute semi searcher
interview with each of the 60 observers
kind of covering things like their strategy
for using the information, how did they do their ratings,
what did they think of the information that was provided,
things like that.
And that's what the rating looks like they were
on a computer and had this dial in front of them.
But the stimuli, oh no...
Okay,
so this is what the stimuli looks like,
the center one here is the field of view condition.
So this is just kind of the field of view,
what they were looking at.
The left one here is the narration condition,
so this is the subtitles generator from how the target
described how they were feeling that we transcribed
and put on the video.
And then this one on the right is the EDA condition
where the observer can see this animated EDA graph,
and try to do the ratings wile they watched that.
Sorry.
Okay, so what did we find.
First we found the full screen button...
(laughing)
So the panel on the left here are the ratings,
the average ratings of the observers in each condition
so the top, the middle condition here
is the field of view one.
So in each of these panels there's a jagged gray line
in the back, those are the targets self ratings
of how they were feeling from negative to positive.
And then overlaid on each of them is the average,
or the mean ratings of the observers in that condition.
So something you can see here is that in general,
the observers did pretty well, and that's because
they were watching kind of the video, they could see
the context in all the cases.
So the empathic accuracy, which is the correlation between
the targets self ratings and the observers rating them.
It's pretty high, overall.
The narration condition was clearly the highest,
that red line follows the jagged line pretty well.
Interestingly the EDA graph at the bottom there
you can kind of see it avoids the extremes more.
We heard in the interviews that people,
that the EDA information kind of made people second guess
what they thought the target was feeling,
and I think one of the ways we might see that manifest
is that they kind of avoid labeling something
as really negative or really positive
by kind of staying around neutral.
And an interesting case in particular is the very
ending part of the video, it's kind of this last scene,
spoiler warning sorry, that the daughter and her father
in a car and they're getting their picture taken
cause the daughter's kind of become famous,
and in both the field of view and EDA conditions
the observers kind of assume that that was a happy ending.
But in the narration condition the narrator
described it as being, she was confused and she didn't quite
she kind of felt claustrophobic and so you could see that
near the end here the narration one follows
the target's ratings more.
The speaker on the right is the mean across
these different, across the whole time series.
And we find a significant differences in empathic
accuracy that correlation between all the conditions.
But not in the way that we had hypothesized.
So it turns out the narration was indeed the highest,
it was the most, people were the most accurate
when given narration information.
The EDA actually preformed worse than with field of view.
So that information actually reduced accuracy overall
compared to not having that information at all.
And then we also did some post task measures
so things like the state empathy, these kind of general
empathy questions were all really high
so we didn't see significant differences there.
But and there weren't differences between,
among difficulty of confidence, or the overall
labeling of the emotions.
Interestingly it wasn't a significant difference
but the confidence in the field of view condition
was a little bit higher than the other two, perhaps
cause there was no other information to kind of check,
check their assumptions of how the person was feeling.
And so, in our interviews we found a couple interesting
themes that are in the paper but I just wanted to get into
one of them in particular, so even through EDA
kind of had this negative effect on accuracy,
there's other components of empathy
that might be arguably more important.
So one of them like I mentioned earlier was mind perception.
This ability to, or detection of the targets
internal states and kind of the ascribing of human like
qualities and mental capacities to the target.
So both the narration information and the EDA
seemed to function to kind of prompt this mind perception
in the observers, so in the narration condition one,
one observer said "I couldn't let my feelings get in the way
I had to look at it from what I was reading about
how the narrator felt".
So it was very clear that they couldn't just assume
how the narrator was feeling because
they're reading about it.
And the EDA condition interestingly is somewhat
of a similar response, one person has summed it up nicely
saying "Sometimes I feel like there were some spikes
in the graph, where I thought there shouldn't be.
But maybe it was because the viewer was a parent
and maybe they would know some things that kids don't.
Maybe that's why."
So in the same kind of way the EDA graph didn't,
when it didn't behave as the observer expected it to,
it kind of made them think oh this person has
their own perspective and experiences
in an interesting way.
So there's some key limitations and room
for future work here.
Some limitations, there's definitely some scope
limitations here it was a controlled laboratory setting.
With young adult U.S. undergraduate and graduate
student population, we had a single empathy target
in this study, we didn't do multiple ones
which can vary, can change the results for sure.
One dimension in particular is the expressivity
of the target, how much they're able to express themselves
in things like narration is gonna affect how
accurate the observers would be.
And we also, we had a very specific operationalization
of empathy, so this single dimension valance rating
and on being accuracy was a highly simplistic
form of measuring empathy.
Some future work that I'm interested in exploring next,
is kind of honing in on this mind perception humanization
effect so we saw from the interviews and maybe
evaluating those experimentally.
Looking at things like arousal rating accuracy
rather than just valance.
In particular because EDA is associated more
with the arousal dimension of emotion than valance.
Explore non-anonymous contexts which many
of our experiences online and with each other
are non-anonymous.
Thinking about the affective component of empathy and
thinking about using more personally significant content
to the target.
So in conclusion, investigating the effects of different
cues on the cognitive component of empathy,
we found that while narrative information resulted
in higher accuracy inferring the valance of
anonymous others experience.
The addition of biosensory information significantly
lowered accuracy,
which is experimental evidence echoing a lot of
the related work in social biosensing, but this information
is tentatively perceived as ambiguous
and it's difficult to precisely interpret.
Qualitatively after we found that both narration
and biosensory information served as reminders
of another's presence, and that their experience
may be different from one's own.
Suggesting that biosensory information
which can be collected and embedded automatically
could aid in mind perception which is a precursor
to other empathic processes.
And in some cases could be more important
than accuracy alone.
Future work will continue to explore
empathy and related phenomena with more diverse scenarios
such as non-anonymous ones, and then some other
aspects of empathy.
So thank you, and I'll take now questions.
(clapping)
- We have time for one or two quick questions.
- [Audience Member] I was wondering what you think
about related emotions such as compassion and also
the negative version of empathy such as
empathic distress and what your viewings are about
trying to look at these things all together.
- Yeah, yeah so like empathic concern and things like pity
are often included in discussions of empathy,
nothing that's not, not feeling like the other person,
but it's like feeling bad for them
or feeling sorry for them.
Which are interesting components.
There's actually one, there's some work I think
[Inaudible] Reiss, R E I S S, at Harvard Medical School
and they did a study where they looked at
showing patients EDA information to doctors
to try to improve like the ability to understand
how they're feeling, and for it to just indicate
the intensity of emotions, or the pain, or the stress
that patients were experiencing.
So that was an interesting application.
But so far I haven't focused on those components
of empathy in my work, but in future work
I'd like to incorporate those.
- We can take one more quick question.
[Inaudible]
Anyone?
Yes.
- Hi Thane.
(laughing)
- [Audience Member] [Inaudible]
- Yeah.
- [Audience Member] [Inaudible] ...interesting that
[Inaudible] More or less motivated to empathize,
and the whole effects like that requires a lot of effort
to kind of narrate your experience... [Inaudible]
- Yeah, so the motivated empathy theories
by psychologist at Stanford, his name is Jamil Zaki,
and one of, he actually outlines these strategies
that people can take to control their empathy.
His paper's really interesting I can't summarize
it all right now, but one of them is to increase
attention to emotional cues, so that could
typically include things like looking at
people's facial expressions or paying attention
to their tone of voice.
And biosignals have this interesting quality
of being an additional cue and drawing attention
to it might increase the motivation to empathize.
Or you can choose to pay more attention to it
and might increase the active empathy in the first place.
But yeah I think it's really exciting to think about
the ways we can encourage or discourage empathy
and his work is really great.