Hello everyone
I’m Zhuoshu from international design institute of Zhejiang University
and today I’ll be talking to you about
capturing the trends, applications, issues and potential strategies of transparent AI agents
Let’s begin with a brief introduction.
Nowadays, AI agents are becoming pervasive in all walks of life
They include virtual agents like the voice assistant Siri
and physical agents like autonomous vehicles
However, it’s challenging to understand
How AI agents work internally
and how to predict AI agents' action
causing HCI issues like distrust and confusion about the performance of AI agents
So how to address these human-AI interaction issues?
Many researches notice the importance of agent transparency.
The word transparency refers to a mechanism that
can report the behavior and intention of agents
the process of decision making
the reasons behind unexpected errors
and any other information helping users to understand the agents
However, while some people found that transparency is effective in building trust
others argued that increasing transparency is dispensable
or might even lead to information overload
That’s why?
We believe it’s because the desired transparency of AI agents could be various
according to the application categories and automation level.
To study this, we applied bibliometric analysis to gain insights from existing literature
With the keywords of transparency, explainability and interpretability
we collected 1240 relevant papers from ACM digital library.
Then we compute the descriptive statistics and
apply the document clustering and semi-automated topic modelling.
Now let’s see the findings.
This figure shows the general trends about the number of HCI publications about transparency
We can see that research focus gradually shifted to transparent AI agents from 2004
and in 2010, papers in this field saw an increasing trend
Emerging events like the explainable AI program keep this field active.
Then we found several application topics and transparency issues from each cluster as shown in this table
I’ll concisely introduce the transparency issues one by one.
According to the first cluster
trust will affect users’ perception of whether agents will perform effectively
and situation awareness will increase users’ perception of dynamic environmental elements.
In cluster 2
the explanation interface focuses on what to explain and how to explain.
As for cluster three
the user-centric explanation pays attention to why, when, and how different users need transparency information
and the interactive explanation supports users’ proactive inquiry about technicalities of agents.
In cluster four, papers mainly discuss about breakdown repair
which can impact how users perceive and interact with the agents.
At last, the fifth cluster is about ethical considerations include privacy and fairness
More details about the issues can refer to our paper.
Despite the above topics
you might be wondering, so how to design the transparent AI agents?
We categorized the applications into a two-dimension framework
according to their risk and timeliness
and proposed some design strategies
For example, the A2 applications such as the clinic diagnosis
will focus more on avoiding risk rather than ensuring timeliness
So such agents would pursue the interpretability and trust of the prediction
and help users understand the inference process or results as much as possible
to reduce the potential risk
Additionally, it’s noteworthy that
the required transparency of agents varies with different automation levels
The transparency level will not continuously increase with the automation level
So we extend the descriptions of SAE level and
produced a universal description for the various AI agents
and suggest the possible transparency level for each SAE level.
This brings us to the end of our presentation
For more details, please refer to our paper.
Thank you very much for listening.