Hi I'm Sandhya Saisubramanian from the ...
University of Massachusetts Amherst.
Today, I will be presenting our work ...
on understanding user attitudes to as negative side ...
effects of AI systems.
AI systems are used in a wide range of applications and ...
have broad societal impacts. However,
these systems are rarely perfect and their behavior ...
may sometimes be considered as undesirable ...
because they produce negative side effects.
Negative side effects are the undesired effects of an ...
autonomous system's behavior that occur addition ...
to its intended effects when it's operating in the environment.
The severity of the side effects can vary from mild ...
events to safety critical failures.
As more and more AI systems are deployed in the real ...
world, negative side effects are ...
becoming increasingly common.
For example, an autonomous car that is ...
optimizing the travel time to destination may not slow ...
down when it's navigating through a pothole.
As a result it can cause bumpy ride to the ...
passengers as negative side effects.
The primary reason for the occurrence of negative side ...
effects is because most systems operate based on ...
incomplete information about the environment in which ...
they're situated.
This could be because of unavailability of information ...
during system design,
or unanticipated domain characteristics that could be ...
discovered after the system is deployed, or
cultural differences between the target uses and the ...
development team.
Since negative side effects are more likely to be ...
discovered after the system is deployed,
it is important to understand how uses react and respond ...
to negative side effects of deployed AI systems,
which is the objective of our work.
Our study focuses on answering four research questions.
To what extent are users willing to tolerate side effects ...
that are not safety critical.
How does the occurrence of negative side effects affect ...
the user's trust in the system's capabilities.
Are uses willing to assist the system in mitigating the ...
impacts of side effects.
This assistance can be in the form of providing feedback to ...
the system every time they observe negative side effects, or
redesigning the environment so that the system's actions ...
no longer produces negative side effects, or
specifying regions for safe operation of the system.
Finally we are interested in understanding whether users ...
are willing to tolerate some level of suboptimal behavior ...
with respect to the system's assigned task in order to ...
mitigate the impacts of negative side effects.
Our study focuses on negative side effects in two ...
different settings. First,
a Roomba that sprays water on the wall as a negative side ...
effect when it cleans the floor.
Second, an autonomous vehicle that ...
does not slow down when it's navigating through a pothole,
resulting in a bumpy ride for the passengers.
The surveys were conducted on the Amazon Mechanical Turk platform.
We recruited 500 participants to complete a pre survey ...
questionnaire that had 6 questions.
The purpose of the pre survey questionnaire was to ...
assess participants familiarity with a AI systems and their ...
fluency in English.
Based on these responses, we invited 300 participants to ...
complete two different surveys:
one that focussed on side effects in the Roomba setting ...
and another that focussed on side effects in the AV domain.
Each of these survey questionnaires had 10 questions.
Finally we had a hundred and eighty-three responses ...
for final analysis in each setting.
In our study we describe scenarios of occurrence of ...
negative side effects and we required the participants to ...
select an option that best describes their attitude.
To understand user tolerance of negative side effects,
we required the participants to select a tolerance level.
Our results show that the tolerance varies and depends ...
on the severity of impact.
We were also interested in understanding whether users'
trust in the system's capabilities may be affected ...
by the occurrence of side effects.
We required the participants to self-report a trust level ...
between low, medium, and high.
The results show that even when the side effects are not ...
safety-critical, they can affect the user trust ...
in the system capabilities.
Between providing feedback, redesigning the environment,
and specifying safe regions of operation,
feedback seems to be the most preferred tool that users ...
are willing to use to assist the system.
This is possibly because of the simplicity of interaction ...
with the system.
Uses are also generally willing to allow for a slack on ...
the system's assign task, meaning that they're willing ...
to tolerate some level of sub-optimal behavior so that ...
the system can learn to mitigate the impacts of ...
negative side effects.
Overall results provide insights into how users may ...
respond to negative side effects of AI systems.
Our results also support the design of customizable ...
systems to mitigate negative side effects and improve user satisfaction.
Thank you.