- Hi my name is Carrie Cai.
I am a research scientist at Google Brain
and today I'll be presenting our work
on example-based AI explanations.
This was joint work between Google's Creative Lab
and Google's People + AI Research initiative.
The motivation behind this work is that machine learning
predictions can be difficult for end-users to understand.
For instance, given just this humorous example,
you can imagine that an end-user might want to know
why this strawberry was predicted to be an apple.
And with the increasing complexity of algorithms,
such as deep neural nets,
their reasoning can be quite opaque,
even to the developers of those algorithms.
As a result, there's been a wide range
of explainability methods,
trying to increase the transparency of such algorithms.
These range from feature attribution methods,
which for instance, point out which part of an image
was most influential to the prediction,
to higher level concept attribution methods,
which for instance, might explicitly say that it was
the shape of the strawberry that made it look like an apple.
In more recent work, there has been a growing amount
of focus on examples as a potentially effective way
to enable AI explanations.
And this is probably motivated by a lot of foundational work
in educational psychology and cognitive science
which tells us that examples can be quite an intuitive way
to explain concepts to say children, for instance.
Rather than explaining to a child
what the definition of a strawberry is,
we could've said, "Give a child many examples
of strawberries and non-strawberries
and through that implicitly explain
what the essence of a strawberry is."
So inspired by this work on examples,
this research explores two kinds
of example-based explanations.
And we evaluate those example-based explanations
in an in-the-wild study to look at
how they affect end-user experience.
We focus this work on--
So first I'm going to tell you about the two kinds
of example-based explanations
and these explanations we focused
within the sketch recognition domain.
As some of you know, Quick, Draw! is a widely used
and popular platform similar to Pictionary
where users are challenged to draw common household objects,
and they're given say 20 seconds to do so,
at which point they're told whether an AI
has recognized their object or not.
So for example, you might be asked to draw an avocado
and then you're told after 20 seconds whether
an AI has recognized it as in fact an avocado.
So let's suppose that you drew this avocado
and the AI says that no I did not recognize it.
Well how can we use examples to explain why?
You can imagine that one way to do so
might be to show you how your avocado looks
like other objects that aren't avocados.
So that's precisely what we did.
So comparative explanations aimed to compare your drawing
to other objects that are not in your object's class.
So here for instance, your avocado might look like
a pear, an onion, or a potato.
And so what we did was we computed the embedding
of the user's drawing
and then we found the nearest neighbor drawing
from alternative classes.
For example, the pear as you can see is overlaid on top
of the user's drawing of an avocado
so that they can kind of see for themself
how similar their avocado might look to a pear.
But even with this comparative explanation,
you might imagine that well maybe the user really does
still think that their avocado looks more
like an avocado than a pear, an onion, or a potato.
So for that reason we also created
a second type of explanation.
These are called normative explanations.
So normative explanations, rather than comparing anything,
instead they draw a random subset of avocados
from the training set.
So for instance, given this set of avocados
from the training set, a user might then realize,
"Oh maybe my avocado is missing a pit."
So given those two kinds of example-based explanations,
we then conducted an in-the-wild evaluation.
And specifically we were curious whether first of all,
do these example-based explanations have any effect
on end-user experience,
and secondly, are there any differences
between the effects of those two kinds?
To do so, we deployed these explanations to about
a thousand users on Quick, Draw!
and we used a two-by-two between-subject study design.
Two-by-two meaning that the comparative explanations
were either on or off and the normative explanations
were also either on or off.
So let's say you're one of the people in the study,
you drew an avocado,
you see what the result was,
and then you're randomly assigned
to one of these four conditions.
You either see no explanation, a comparative explanation,
a normative explanation, or both explanations combined.
And then to measure the effect on end-user experience,
we asked four survey questions.
So the first question addressed the extent to which
the users felt they understood the system.
The second two questions got at user trust.
So two key dimensions of trust
are capability and benevolence.
So we asked a question about the extent
to which people felt the system was capable and the extent
to which the people felt the system was benevolent.
And then finally, we asked a question regarding
attribution of credit and blame.
So we asked people, "Given the decision that you just saw,
would you attribute the decision more
to your own drawing capabilities
or more to the system's capabilities?"
And of course because for each user,
their drawing was either recognized or not,
we also analyzed whether that fact,
whether their drawing was recognized or not,
whether that had any effect on end-user experience as well.
So first looking at the results on capability,
we found that there was an interaction effect
between normative explanations and whether or not
the drawing was recognized.
So if a user's drawing was recognized,
we saw no significant difference
between any of the explanations.
However, when the drawings were not recognized,
those who had seen normative explanations
felt that the system had higher capability
than those who had not seen a normative explanation.
And we also saw a similar trend on the question
regarding system understanding.
So when drawings were not recognized,
those who had seen normative explanations
rated the system as having--
rated themselves as understanding the system better
compared to those who had not seen
any normative explanations.
So at this point you might be wondering why
normative explanations might've had an effect
whereas comparative explanations we did not
see a significant effect.
And when we looked at the data, we found that
comparative explanations were sometimes quite surprising.
So let's say a user drew this toe,
and the top three predictions were
a hula hoop, donut, and bracelet.
And sure it's true that the drawings are quite similar
to the toe that I drew,
but they're quite semantically unrelated
to this concept of toe or any sort of human anatomy.
And so for that reason it could've led to surprise
and it could've also exposed
some of the algorithm's limitations.
In fact we found that users tended to spend a little longer
viewing comparative explanations
compared to normative explanations
and this could've been partly
because of that element of surprise.
It could've also been because perhaps there was some
notion of cognitive load.
Differences where with normative explanations,
notice that all the pictures were from the same category.
You just look at a bunch of toe pictures.
Whereas, in the comparative explanations,
the user would have to compare across multiple categories
and that could've potentially contributed to this longer
duration of time they spent looking at the explanation.
So to summarize the research contributions,
we explored two kinds of example-based explanations
and we also conducted an in-the-wild evaluation,
where we found that these example-based explanations
do have some effect on end-user experience but that these
effects differ depending on the type of explanation.
So comparative explanations, on the one hand,
sometimes led to surprise and may have exposed
some of the algorithm's limitations,
whereas normative explanations tended to improve
user perceptions during system errors.
So depending on your goal, you may want to consider
these different kinds of explanations differently.
This work also points to a future
where leveraging human intelligence
could play a key role in improving explainability.
So whereas in the past, we often lean heavily on machines
to do the heavy duty lifting of providing an explanation
and humans being somewhat passive recipients
of those explanations.
But with this work and with I think some of the other work
we've seen in this session,
we envision a future where this is much more of a
collaborative work between the human and the machine,
where the human also brings something to the table.
They bring their own intelligence,
their own past experiences to interpret
the examples that they're given.
And so in that sense, perhaps in the future,
machines would only need to be able to present
just enough information for humans to understand.
And with that thanks for your attention.
Here's my contact information and
I hope you'll go check out the paper.
Thank you.
- [Audience Member #1] Yeah, one thing is I'm not sure
how users actually understand the tasks.
So they might understand the task in a comparative way
or in a normative way.
So if you say to somebody, "Draw an avocado",
what they probably hear or what they mean is,
"Draw a representation of an avocado that's good enough
for a human to recognize in a scene."
So if I told you, "Draw an avocado so you can include
it in a salad bowl."
Then maybe the only oval thing you would see
in a salad bowl would be an avocado,
so maybe that avocado is good enough.
So it depends on whereas the normative one,
obviously anything that looks more like an avocado
would have the better chance of being recognized,
so it's always good advice to say,
"Make it a better drawing."
But whether the drawing's actually good enough
for the purpose, that's something that's up for grabs.
And if people are thinking it's good enough for the purpose
of a human and then your algorithm comes along
and they might feel a little tricked
because you didn't tell me that I was supposed
to draw an avocado was good enough for the algorithm.
- Yes so, one bit of context around this was that
I think that's a really good point,
I think that the context of the task really matters.
In this case, the context was that they were told up front
that they were playing a game
and that they're going to draw drawings
and then they're going to see results on whether
an AI has recognized the drawing.
So that was the context around the task
and I do think that the context
around the task really matters.
So in the future it might be interesting to see
in which situations do different kinds of explanations
have more impact.
Game-based setting might be different from
like a critical high stake setting as well.
Thank you.
- [Audience Member #2] Hi.
- [Moderator] Just one last question.
- [Audience Member #2] I'm just curious if you found
any differences between cultures when you compared
the normative versus comparative explanations,
so were there any differences between people of one culture
feeling that the normative explanations were better
as compared to the other ones?
So did you find any impact of culture in how good
people perceived these explanations?
- So we did not analyze the differences
because for privacy reasons we didn't collect
location information.
However if you go on the Quick, Draw! site,
there's actually a whole slew of research
that people around the world have been doing
on the Quick, Draw! data
and some of that work does look at some of the
cultural similarities and differences
which have been really interesting.
- [Audience Member #2] Thank you.
- Thanks.
- [Moderator] Okay I guess it's time to end this.