The prevalence of AI in our daily lives and professional domains
makes it crucial for us to understand human AI interaction because it can be
alarming when users adopt the applications without knowing how the AI mediated
results are derived, especially when AI presents biases.
A top down guideline may not be feasible because the AI mediated applications
and their potential biases may be perceived differently
depending on their stakes and influences on users lives.
Transparency and explainability
should not only come from AI modeling and computational efforts, but also social
and interactive perspectives from users' viewpoints for trust development.
To inform designs for explainable AI (XAI),
our study investigates user perceptions of AI biases in scenarios
of different stakes through a bottom up and user centered approach.
The characteristic of AI
mediated results or decisions is that it's not directly interpretable
and understandable by users because how the models are
processed or trained involves much complexity.
Some technical biases can be amplified
through AI modeling and result in sociotechnical biases.
Therefore, researchers call for the need of XAI, where expressive
and interpretable explanations should be implemented to improve
users' understanding and confidence in making use of the AI generated outcomes.
But the operationalization of important
dimensions in XAI, such as fairness or transparency, is actually multi
dimensional and complex for practitioners to implement.
General users may feel a dark about
algorithmic explanations based on technical approaches,
which brings forward the importance of a user centered perspective.
The solution to XAI should not be just
more AI, but needs to incorporate an HCI angle to understand and empower
users and their interactions with the systems.
Understanding users' mental models or how
users interact with and make sense of AI systems is crucial in XAI.
HCI researchers should incorporate stakeholders' perspectives,
not just experts, but people who are influenced by AI,
to provide suitable and useful explanations for them.
How do stakeholders, both end users and engineers,
make sense of AI and some potential associated sociotechnical biases?
How do end users seek and engineers
provide explanations of AI models, especially when biases are present.
How do stakeholders, both engineers
and end users, perceive biases and associated
level of explainability in contexts of high or low stakes?
The first three authors conducted one on one,
in depth interviews with 24 general end users recruited from online.
The recruiting criteria included
1) having experience using technologies that they thought used AI techniques.
2) not being part of the AI modeling or
implementing processes, and 3) aged 18 and above.
Our engineer participants took care of different stages
and tasks of AI product development and were in different pay grades,
ranging from interns in large corporates to CEO of startups.
The interviews ranged between 40 to 90
minutes in length and were all recorded for transcription and analysis.
There are two parts of interview questions
for both end user and engineer participants.
First, we included their general
experiences and attitudes towards using and or developing
AI mediated services or systems they identified and if they understand
how they work and perceive any potential biases in those applications.
Then we use scenarios
in high and low stake to probe their perceptions and desired explainability
about the sociotechnical biases that we drew from the public press.
The high stake scenarios
include how AI algorithms yielded biased results for approving
higher credit limit for males, assigning higher possibility of risks
for colored people and making favorable decisions for male applicants.
For low stake situations, we presented
issues like how search engines results yielded gender biased perceptions
for keywords like CEO or how text autocomplete or machine translation
results provide gender stereotypical associations.
Our participants had noticed that AI may generate biased results,
and they considered that such biases reflect the social reality.
An interesting thing we found was that if
the participant's own perceptions were aligned with AI's biases,
they were less likely to spot the biased issue, and they attributed external
and internal sources when they spotted AI biases.
For the internal sources, end users most likely blamed themselves
for not having advanced literacy to know how AI works.
On the other hand, the external attribution involves 1) perceived
erroneous algorithms, 2) engineers' lack of domain knowledge,
and 3) companies that launch AI enabled services.
While it is ideal that the AI models
and the AI applications are made transparent to the users,
our results suggest that users may feel overwhelmed by such glaring transparency.
Sometimes, such disclosure may not be desired if users don't want to know.
When dealing with biases and enhancing AI explainability, researchers
and practitioners need to consider the fine line both cognitively and socially.
In cases where users wanted to know more, there are three ways to offer
explanations, including 1) using an approachable
narrative to display how the AI models work.
2) revealing key metrics that determine the results or what are left
of from the models, and 3) being clear about the conflicts of interest.
Our participants
care about when their public or work related searches crept into their personal
space, especially when the results were biased.
This can actually reduce users' trust.
In general, they cared about high stake
issues regarding people's life or civil rights, but more
importantly, how and where the techniques are applied matter.
The inference making process in AI models adds uncertainty.
AI system designs should have affordances that allow the general users to be able
to understand their functions and reflect upon potential biases
of the system, even if it is the first time they use it.
A mediated role in an explainable
artificial intelligence system can be a service chatbot or an animated agent.
And to our participants, they have lower tolerance about AI
applications that may encroach on their personal space.
They were sensitive about the applications
that muddle the boundaries of public and private contexts.
So XAI designs should include information
about how, where, and what types of data are collected for model optimization.
Also, context awareness designs
should be implemented to avoid disrupting users' boundaries.
Then, our results suggest that high stake scenarios should be bias free.
AI systems should provide nuanced explainability in high stake applications,
but also give users options about how much they want to be informed.