hi i'm agat and i'm here to present our
work at the intersection between
computer vision model explainability and
identify
bugs and failures in these models
so as most of us might have heard
already models might suffer from various
types of failures some can be funny one
such as confusing whether something is a
chihuahua or machine
but some others might be much more
critical for safety for instance this
car started to slow down just because it
confused the moonlight
with a yellow traffic light
another failures might also pose
societal harms
um
like the this vision api that confused
thermometers with guns when a darker
skin color hands
would be holding these
thermometers
and so these different types of failures
are due to different types of perks for
instance in the data sets train
touch were used to train the models or
in their architecture used for these
models
in reaction to that
the machine learning testing community
has developed automatic methods to
identify these failures and bugs
however our affirmative studies showed
that their these methods are typically
not used by practitioners
instead other works look into the
explainability of models
and
certain explanability methods are used
by practitioners according to a study
yet no
in-depth study has been performed to
show how exactly these explanations are
used for debugging and that is what we
do in this work
especially there are different types of
explanations for instance based on their
medium scope audience and we don't know
which ones are actually useful
for instance we're looking to local
explanations
providing
visual indications of the features the
model might be using to predict one
class for one sample
so these selling symbols
and there can also be global
explanations such as the one from seca
that provides
textual information about features the
model might be using generally to
classify one
and so specifically in this paper we
asked how could diverse explainability
methods be used to support the failure
identification and back characterization
processes of deep learning computer
vision models
and to answer the question we decided to
take a research through design approach
and to build a design probe
for that we first needed to identify the
functionalities that the probe should
allow for
how they should be realized within their
user interface and what kind of
interactions or workflows
this interface should be providing
and so to do so
um we perform the literature analysis a
formative study and various co-creation
sessions that are discussed uh in more
detail in the paper
and next we wanted to identify the
different potential uses of
explainability methods for debugging so
using our probe so we implemented it and
perform various users to these results
what does the probe actually look like
so it's divided into multiple
tabs different interfaces providing
different functionalities
let's see a few of them
for instance this one provides
information about the performance of the
model to be debugged
especially you can see this confusion
matrix
and when you click on a cell of it you
then see explanations related to the
classes concerned by the matrix
for instance here you can see local
visual explanations silency maps that i
mentioned before
but also if you click on one of these
images textual explanations
that relate to it
you can also see of course global
explanations here that
show how the model might distinguish
between the two concerned classes
displaying all these explanations in the
same interface i'll offer the comparison
between them
and then we have
different kind of interfaces for
different parts of explanation before it
was about two classes
and now about all the classes of the
data set
so these global explanations
um
with default parameters to compute them
or more variability for someone who is
more
used to them
you can also have different kind of
interactions for instance for querying
the explanations
or queen even uh images on which the
model was trained these query
explanations for instance mimic content
fractures that uh some participants of
our affirmative study were asking for
um and so now to the user study
so we provided one use case bird
classification
and we made sure to
include different types of failures in
bug in this use case
so depending on the whether the model
was making correct or incorrect
inferences and using correct or
incorrect features
and we interviewed 18 practitioners
um
so who went through this use case and
debugged the model and then we also gave
them post questionnaires based on the
user engagement scale
and ratings of each interface
we got various results but i will just
explain a few of them here
so first of all um
where we could see that the bug
identification process was rather
successful with the explanations we gave
in average participants identified 3.5
bugs
however the results were quite disparate
between participants type of
explanations used a type of background
they had
this is because some of them had
different views on what
failure is especially for feature
figures
but also because others needed maybe for
instance more guidance
or less confidence in their actual
process
some are so misinterpreted the
explanations which led to identify the
wrong books
so this shows different opportunities
for future work
developing more better user experiences
providing warnings
for instance not to neglect certain
types of books
and providing more guidance in general
we also went into more detail into each
of the type the explanations that were
used
and especially identified that they are
complementary uh most global local
explanations were used
for different steps of the debugging
process
and similarly different medium f
explanations were used
visual uh
more frequently but also textual
explanations that were
maybe less familiar
the require functionality was also
useful although more advanced and less
used functionality
so this again uh shows avenues for
future work for instance to develop
types of explanations that have been
less researched until now
so to think about
the extent
of automation that one needs for
guidance and at the same time of freedom
to actually
further identify bugs
so in summary we provide a design probe
first and then we performed
user studies which gave us insights on
the use of different kind of
explanations that are already existing
but also
showed avenues for future research
opportunities