Hello everyone. My name is Yihe Liu, I am honored today to present to you our work with
a Anushk, Dr. Yang and Dr. Bruckman. The paper is named "Will AI Console Me when I Lose my Pet?
Understanding Perceptions of AI Mediated Email Writing." This work is conducted
in the Social and Language Technology lab in Georgia Institute of Technology.
In this work, we ask the question, "how does the involvement of AI in writing emails
affect users' perceived trust in the communication?"
Since the launch of GPT3, AIMC tools have been rapidly deployed by many applications.
For example, Shorty AI can write complete paragraphs based on the title of the paper
and the short abstract. This makes analyzing AIMC important because it is important to ensure that
trust is not lost due to the environment of a AI system.
We replicate previous work in the context of emails instead of short form writing
like smart replies or text messages. We investigate in the interaction
between interpersonal emphasis and the effect of AI mediated communication, communication on trust.
Finally, prior work suggests perceived AI involvement decreases trust. So we complement
prior work by a qualitative analysis of in depth interviews.
The related work focuses on two aspects: AIMC and trustworthiness.
The replicant effect states that the receivers find a center less cooperative
if they suspect the uses of AI. The Hyperpersonal model suggests that the structure and increases
cues of emails might be a relevant factor to judge trustworthiness. We also follow previous
work on trust to define it based on its three dimensions, that is ability, the benevolence and
integrity. Ability reflects the competency of the sender to have influence on the receiver.
Benevolence is the extent to which the sender wants to do good to the receiver. Finally,
integrity is defined as the receivers perception that the sender adheres to set up principles.
For each of these three dimensions. We design a survey question to quantify it.
We follow a Wizard of Oz approach. That means we will actually write the questions, but the
participants will be told that they are written by AI. We ask questions that measures trustworthiness
based on the three dimensions in the previous slide. At the end of the survey, we measured the
disposition to trust, computer attitude score, AI attitude score, and subject expertise score.
The independent variables are interpersonal emphasis. We have three scenarios, low,
medium and high. For low the scenario is inquiry about a product. For medium. The person
is writing email, for party invitation. And for high interpersonal emphasis scenario,
the sender is writing an email that consoles the receiver of the loss of a pet.
We also have three AI priming condition. First, the email is written by a human,
or the email can be written by a human with the aid of an AI, or
it can be written completely by AI. Our dependent variable is the perceived trustworthiness.
For our quantitative study, we want to find how AI condition affect trustworthiness.
In the leftmost graph, we see that if the participant is told,
the email is written by an AI, the participant perceive the email to be less trustworthy.
We also find that as the interpersonal emphasis of an email increase, the perceived trust was in this
also increased. However, in the graph on the right, we find that regardless of AI prompt,
the email is perceived to be more trustworthy as the interpersonal level increases.
This contradicts our hypotheses that human trust AI for factual knowledge
and distrust AI for writing emotional texts. So why does this happen?
In our qualitative results, we find that first, participant is not too happy
about the fact that a person is using AI to write email, which is in accordance to the findings of
previous studies. More importantly, participants evaluate email more on paralinguistic cues,
such as names, location, or other social context that the email may have. Overall,
we find that participants start the email should sound more formal for Scenario product inquiries.
And more conversational for scenario, party invitation, and constellation of pet loss.
Participants saw typo as either a good or bad sign of trust was in this longer email were preferred,
because they contain more details. And finally, social cues by the sender's financial status
can be used to evaluate the ability, which is one dimension of trust.
You find that participants valued inquisitive cues accuse more than the eye prompts. When
judging the trustworthiness of an email. Participant five said "I just forgot, I have
the impression in my mind that those messages were written with the help of an AI system
while I was reading the messages, because it felt quite natural. And they use this word like 'haha',
I thought that was quite funny. Yeah, so I totally forgot. That was a help of the system.
It's quite amazing." We find the distinction between what people say they feel about AI,
and how they actually react to it. And that might be because participants value linguistic cues
more than the AI prompts. This has serious implications for designers of AIMC tools, because
this suggests that AI writing assistant tools will be accepted over time, if they sound like human.
Thank you so much for your attention.