Hi there and welcome to the presentation of our paper
“Capable but Amoral?
Comparing AI and human expert collaboration in ethical decision making”
My name is Suzanne Tolmeijer
I'm a recent PhD graduate from the University of Zurich
and on behalf of my co-authors, I welcome you to our presentation.
The motivation for this research revolves around the fact
that we see increased autonomy in AI as its capabilities are increasing.
As a result, both users and society are giving different responses.
On the one hand, we see algorithmic appreciation,
where people actually prefer an AI to do certain tasks over a human,
thinking it is more precise and less biased.
On the other hand, we see people showing algorithmic aversion,
Where they actually prefer a human to do certain tasks over AI,
responding more strongly to AI mistakes.
While for some tasks it's very clear whether the AI is doing a good job,
for others is less clear-cut.
One example is ethical decision making
The reason that is relevant to look at ethical decision-making for AI is twofold.
On the one hand, with this increased autonomy it can be relevant
to have an ethical component in the AI’s decision-making.
On the other hand, it can actually inform us of our preferences
when we have to formalize what we want the AI to do
in this use case where there is no consensus on what the right thing is.
We look at the following research questions.
First, we focus on trust, asking ourselves: How does reported trust
in a human and AI expert compare for ethical decision-making support?
Note that for all these questions we're comparing human vs AI.
Next, we look at responsibility, asking: How is responsibility attributed
when interacting with a human or AI expert with different levels of autonomy?
The abbreviations here mean human-in-the-loop or human-on-the-loop,
where human-in-the-loop means that the human is still part of
the decision-making process and the AI is supporting,
while in human-on-the-loop, the AI is making the decisions 35 00:01:58,000: --> 00:02:02,000 and the human supervises but can intervene if something is wrong.
Finally, we look at reliance, asking: how does reliance on human vs AI advice compare?
In some studies, trust and reliance are deemed the same thing.
So just to clarify, we use the following definitions:
We define trust as a “belief that an agent will help achieve an individual's goal
in a situation characterized by uncertainty and vulnerability.”
We measured this with a validated scale.
Reliance, on the other hand, is a “discrete process of engaging or disengaging with the AI”,
Which we measure by looking at the behavior of the participants.
To research this, we used a two by two mixed between-within experimental design,
looking at the topic of drone support for human lives lost or saved.
We looked at the level of autonomy, which is either human-in-the-loop or human-on-the-loop,
used as a between-subjects variable, and scenario framing,
so either we maximize the number of lives saved or we minimize the number of lives lost.
In the scenarios, people had to choose in a multiple-choice question
where for each option, they got a certain number of lives lost or saved
and a certain change that this would happen.
For this 2 x 2, we asked people to do this with both human and AI experts,
so this is used as a with in variable, to compare how people respond to human vs AI support.
To test this with participants, we had a simulation game made, that looked something like this:
People were welcomed at the training center by Captain Smith,
where there was an instruction on what they would do.
Participants have to fill in demographic questions and do a first training mission
so they get acquainted with the interface, after which they get to their actual
work environment, where any moment a new mission can occur.
[Alarm sounds]
As you see, Captain Smith is telling us there is an emergency situation.
When we go there, he explains what went wrong and what type of expert will support us.
In this case, it's a human expert but it can also be an AI one.
Once they get to the mission part, they see the following:
On the top, they see an avatar with the type of expert that's helping them.
In this case, it's an AI. These avatars were pre-tested to make sure
they didn't influence our results.
Next, on the right, you see the advice that the expert is giving.
In this case, they advise Target number one.
On the bottom right, you see the different options the participants had.
In this case, it's an advice and save setting where we try to maximize the number of lives saved.
On the left, everything is playing out in real-time.
So when participants are in the mission, it looks like this.
[drone sounds]
As you can see, the drone is moving closer as participants have to make their decision
and confirm the option they selected. For each setting, so AI and human,
they have to go through four different missions. Now, on to the results.
With regards to trust, we find that there's an overall reported higher trust in AI
and higher-capacity trust, meaning that participants have higher trust in the capabilities
of an AI over a human expert. On the other hand, the human expert gets higher moral trust,
so participants have higher trust in the moral capabilities of the human expert over the AI.
In terms of responsibility, we find that there's a higher responsibility ascription
to the human expert over the AI expert, but in the AI case, we do find that
the programmer or the seller of the AI are both deemed somewhat responsible.
Finally, in terms of reliance, we see that for the final two missions there is
slightly higher reliance on AI advice than human advice, but we need to dive further
into these results to see whether this was actually triggered by the type of scenarios
that we presented or the ethical preferences that the different participants had.
In terms of expert autonomy, we did not find any effect between human-in-the-loop
and human-on-the-loop. This was quite surprising given that this is a big focus of research
and many different results are being found. One possible reason is that we had the framing
in a way that the participants were always deemed responsible in the end.
Potentially, this could partially be the cause. However, for design implications,
this lack of expert autonomy effect mixed with the fact that human experts are considered
more responsible and more morally trustworthy while the AI is more capable,
means that for ethical decision-making, the focus should be on meaningful human control,
Where the human is in charge and the AI supports the human in ethical decision-making.
And with that, I want to close. Thank you very much for your attention,
I look forward to your questions, and please feel free to reach out for any further feedback.
Thank you.