Hello everyone! My name is Maxwell, and today I'm here to talk about visual, textual and hybrid
explanations, and more specifically how user expertise has an influence on perceiving them.
My research was done together with Martijn and Katrien, both
from the Augment Research Group at the KU Leuven.
So we find ourselves in a field of explainable artificial intelligence
or XAI for short. The main goal of XAI is to unravel the black-box of AI algorithms,
and make certain parts of such algorithms transparent to end users using explanations.
These explanations can address various parts of such algorithms, such as the output,
what input parameters there are, and how the input parameters have an influence
on the output. For example here we see a simple textual explanation
that shows us why a certain post was recommended to us on social media. Now realistically for me
it would be more pictures of food, because I like food, but that's besides the point.
Next we see an example of a visual explanation, where this importance map shows us which parts of
the image contribute to the outcome of the underlying algorithm. So clearly there are
many use cases, both theoretical and in practice. However literature often discusses the positives
of explainable AI. While there certainly are many benefits and even needs for XAI, there are also
some drawbacks to keep in mind. One such drawback is that users build mental models of a system with
the way they perceive and interact with it. So the type of explanation chosen can strongly influence
the way users perceive the same underlying algorithm. And besides that, research also
points out that user type also plays a big factor in the way users perceive the same algorithm.
So because this is a scarcely research topic, we decided to explore the effects the different type
of explanations have on perceiving the same underlying algorithm, and more specifically,
how this is influenced by user expertise. So to do this we looked at differences in user
interpretation and their personal preference. We also measured other parameters, but for that i
would refer you to our paper. Now the question is, did we find a big gap between expertise groups?
Well we certainly did! And that's why we also decided to explore a different option on how we
can minimize this gap between different expertise groups. So that's for our third research question.
So for the first iteration we developed two common types of explanations,
the first one being the textual one and the second one being a more visual one using
partial dependence plots which are used quite often both in literature and in real use cases.
Both these explanations convey the same type of information
regarding which input parameters are present, how these parameters currently impact the prediction
model, and how changing these parameters would influence the outcome of the model.
To test the differences between two types of explanations, we set up a between subject study
with 10 experts and 10 non-experts. During the study users had to perform
two exploratory tasks to get familiar with the dashboard, followed by two tasks that gauge
their understanding of the explanations. And the results we found were quite interesting.
Looking at the number of correctly answered insight-tasks we see that non-expert users
struggled quite a bit with the visual explanations, more so than with the textual
one. This isn't all that surprising as graphical information requires more effort to interpret
and might be more difficult for non-experts to do. However, when looking at preferences we still
see that the majority of both groups still prefer the visual explanations. So do users,
and mainly non-expert users, prefer the type of explanation that they perform worse with?
Seems rather strange.. Well, this can be explained by looking at the think-aloud study logs.
There we see that at least seven non-expert users followed a flawed backwards-oriented reasoning.
Under the classification of Croskerry et al., this falls under what they call System-I thinking,
which is described as fast, low-effort and heuristic-based. However, research by Lim et al.
concluded that for inexperienced users, which in our case are the non-expert users,
System-I thinking might lead to confirmation bias, where an incorrect hypothesis
gets confirmed by selectively looking for information that supports it. So that is why
non-experts felt that the visual explanations were more helpful in conveying the information
even though they were interpreting this information incorrectly.
Another issue we discovered with non-experts, was the lack of exploration using visual explanations.
When looking at the research of Tsai et al., they state that unfamiliar information
might induce a reinforcement effect, that causes users to avoid
interacting with various content. This explains why the lack of exploration with
non-experts is stronger with the visual explanations than with the textual ones.
Now when looking at the expert users, we see that at least seven of them (explicitly) followed
a deductive reasoning approach, where they first explore the given information to deduce a possible
answer, and a reason about their correctness of it afterward using their own knowledge.
This deductive reasoning falls under the System-II thinking, which is considered to be slower high
effort and analytical. It is consequently less prone to cognitive biases. So there is a clear
hurdle to overcome regarding non-expert users and their System-I thinking, and them still preferring
the visual explanations. So how can we address this? One solution that we came up with are hybrid
explanations, that essentially combine the aforementioned visual and textual explanations.
While adding this new textual explanation does not introduce any new information, we hypothesize that
it will aid users and correctly understanding the visual explanations that are also there.
So to test our hypothesis we conducted a second user study with 10 new experts
and 10 new non-experts. This time, they had to perform the same two exploratory tasks and the
same two insight-tasks they also had to perform in the first user study. However, this time they
had to do it on the hybrid explanations. We also compared these results with the previous
visual explanation results the ones that had the most issues that needed to be addressed.
So, did the hybrid explanations improve upon the visual ones in terms of correct interpretation?
The answer is most certainly: yes. For non-expert users the addition of textual explanations
allowed some users to correct their wrong understanding of the graph above.
Additionally, it also allowed them to be more confident in exploring the dashboard, so this
also partially addressed the lack of exploration that we found with the visual explanations before.
Expert users also benefited from the addition of textual explanations, albeit in a different way.
For them, the textual explanations functioned more as a reassurance that they were correctly
interpreting the visual explanations. Those users stated that after looking at the textual
explanations once, they felt more confident in using the visual explanations afterward.
So what are the key takeaways from our research? First of all, we see that user expertise is
an important factor by looking at how users perceive your explanations. Different expertise
groups have different needs, expectations, as well as limitations to take into account.
Some of the limitations that we found were more tied to the non-expert users,
which were for example the reinforcement effect that caused a lack of exploration,
as well as the confirmation bias, which led to incorrectly interpreting the visual explanations,
more so than the textual ones. So definitely take expertise of your
target demographic into account when designing your explanations.
Secondly, don't assume that all users will understand your visual or graphical explanations
correctly, or the way you intended it. So make sure you provide additional information
if you know your target demographic consists of mainly non-experts. In our research,
we explore the addition of textual explanations. However, we advise you to also look into different
methods such as adding tutorials examples and so on that can also potentially aid users.
So this wraps up my presentation! I would like to thank you for your attention,
and please feel free to ask any questions. Thank you very much!