When designing a conversational agent
that assists users in the tools like Microsoft Office
it is a design choice if an agent designer
decide to use the metaphor of a clip,
which is an office supply.
Since it talks in human language,
anthropomorphizing it with attaching eyes
and expressive eyebrows on it can
also be considered a design choice.
Just like this, conversational agent designers
attach a certain metaphor to communicate its system.
Then how would such metaphor like Clippy,
perceived differently from other metaphors
like Duo from Duolingo
or different animals from Nintendo’s Animal Crossing?
Or most commonly, a human forms
like Microsoft’s Tay, or a Pandorabot’s Kuki?
Hello, my name is Ji-Youn Jung, presenting our work
entitled Great Chain of Agents: The Role of
Metaphorical Representation of Agents
in Conversational Crowdsourcing.
With my co-authors Sihang Qiu, Alessandro Bozzon,
and Ujwal Gadiraju from Delft University of technology,
we were curious about metaphorical representation
of conversational agents.
According to a conceptual metaphor theory by Lakoff and Johnson,
people’s understanding of a complex and abstract system
is often conducted metaphorically.
Famous example in HCI domain can be
the ”Computer is a Desktop” metaphor,
which helped people, back in the past, to understand
what this computer thing was used for.
By adopting the metaphor of an office desktop,
people understood that they could make a folder,
create files, send mails, et cetera.
Lakoff and Turner identified a “Great Chain of Being"
metaphor in his following book, “More than Cool Reason”,
as one of the fundamental conceptual metaphor framework.
Tracing back to the Greek philosophers
like Aristotle and Plato, Great Chain of Being
is argued as unconscious cultural model
indispensable to our understanding of ourselves,
our world, and our language.
Consisting of mainly five entities,
God, Human, Animal, Plant, and Inorganic Objects,
its hierarchical structure implies that
entities corresponding to a higher level
of the Great Chain of Being have all the
qualities that the lower level possess,
in addition to their distinct qualities.
Great Chain of Being allows people
to understand someone, in this case Achilles,
with their schematic knowledge of a lion,
within the Animal domain.
Here, the metaphor is emphasizing
the bestial power of Achilles
through our understanding of a lion.
Then, taking each levels of Great Chain of Being,
what difference would it make when agent designers
convey the system ability, which depicted
as a black box here, either a woman,
a bear, or a tree?
To find that out, we adopted the lens of
conversational microtask crowdsourcing.
By making traditional web interface into
a conversational interface, researchers have
found that the conversational interface improved
the pereceived worker engagement, and
significantly higher worker retention
while maintaining output quality.
So we formulated our research question as follows.
How do different metaphorical representations
of a conversational agent
impact worker engagement, perceived cognitive load,
intrinsic motivation, and trust in
conversational microtask crowdsourcing?
In this study, participants from a crowdsourcing
platform Prolific are introduced to
one of six metaphors for a conversational agent
they interact with.
In addition, they were also assigned to
either an Information Finding task or Image Classification task,
which led to in total of twelve conditions.
After they finished conducting the microtask,
they evaluated the interaction with the chatbot.
Image Classification task asked participants
to analyze images of birds and to classify
the shape of their beak.
The control condition did not show any
agent avatar, while other conditions with
metaphorical representations displayed their own avatars.
In Information Finding task, workers were asked
to find taxonomic classification information
about the bird on Wikipedia.
We controlled the metaphor sampling
with the Stereotype Content Model,
where two dimensions of warmth and competence
has proven to impact impression formation.
We sampled metaphors that belongs
to the high-warmth and high-competence.
And there we decided to have
god as just God, Human as Trained Professional, Animal as Dog,
Plant as Avocado, and Inorganic Object as a Book.
Each metaphor condition were manifested
not only with their avatar, but also with their
narrative during the interaction.
While they spoke most of the messages in same manner,
some messages differed in order to clearly
communicate the metaphor of an agent with the workers.
Our result indicated that different metaphors
from Great Chain of Being impacted three factors.
That are: Worker Engagement, Intrinsic Motivation,
and Perceived Cognitive Task Load.
On the top of the screen, you can see
the overall result that was shown to be significantly different.
First, the Object metaphor received significantly
lower engagement score and Aesthetic Appeal score
compared to the Animal, which is a Dog, metaphor.
Also, perhaps in a similar vein,
Object metaphor also received lower Interest-Enjoyment score,
which is a sub-score from Intrinsic Motivation scale,
compared to the Human and Animal metaphor,
and a control condition.
However, interestingly, it resulted in
lowering the perceived cognitive load compared to
God, Human, and Plant metaphors.
God metaphor resulted in lower perceived usability
than the Control condition without any representation,
and God metaphor also resulted in higher task load than
the Object metaphor and a Control condition
without any representation,
In conclusion, our result show that
there’s a trade-off of choosing one level
of metaphor for representing the conversational agent.
Interestingly, unlike previous findings where
providing higher anthropomorphism and human-likeness
has shown to be beneficial,
in our study, Human metaphor did not
overpower compared to other metaphors.
It did increase interest-enjoyment than the
Object metaphor, however it also resulted in
giving higher cognitive load to the crowd workers.
For a Design implication, it seems like
in the context of conversational crowdsourcing,
adopting Animal metaphor or not giving any metaphorical
representation seems like the most beneficial,
as it improves worker engagement and intrinsic motivation
without giving workers higher cognitive task load.
When it came to the Object metaphor,
while it defected worker engagement and intrinsic motivation,
it seemed to lower cognitive task load significantly
than any other metaphors that designers could choose.
Therefore, if agent designers consider
lowering the task load as their upmost priority,
they could adopt metaphors from the Inorganic Object domain.
Finally, our study shows that there
were no benefits in adopting a metaphor
of God or a plant.
In summary, our contribution can be threefold.
We found evidence that there are trade-offs
in using a metaphor from different levels
of Great Chain of Being.
We propose a method to systematically
analyze such metaphor space
by adopting Great Chain of Being framework.
Finally, we enrich the current discourse
on users’ metaphorical understanding
of conversational agents.
If you are also interested in metaphorical representation
of conversational agent, please get in touch!
I’m on Twitter as stopyunn with two "n".
and thanks for watching.