Hello, welcome to my talk.
My name's Andrew, this is joint work with Nina, Elisa, Elena, and Finale.
So, this talk is not just about the interpretability of machine learning models,
but also how we evaluate the interpretability of machine learning models;
we're introducing an interpretability evaluation method.
That means we also need to think about how we evaluate those evaluations,
which gets a little crazy, but bear with me.
So, the starting point of this story is that we train a machine learning model,
let's call it F, using some machine learning method.
We then want to explain that model to some user or group of users,
and of course to do that, we need to use some kind of interpretability method, or visualization.
And maybe we don't just have one model, but multiple,
and we want to determine which is the most interpretable to this group of users.
To do that, we employ some interpretability evaluation metric
that gives us a quantitative score for each model, or perhaps a ranking.
And there are a number of ways people try to do this in practice;
some involve seeing how well the user can predict the predictions of the model,
others involve evaluating the coherency of the
user's mental model of the model in some other way,
or asking them to perform a downstream task
that requires some meaningful form of model understanding.
However, when we're doing research on interpretability evaluation,
we need some way of evaluating whether the evaluation itself
is measuring something meaningful.
So, although the main contribution of our paper is an interpretability evaluation method,
we're also going to innovate a bit on how to evaluate evaluations.
Another issue here is that most prior ML interpretability research focuses on classifiers,
regressors, or, more generally, what are called discriminative models.
Discriminative models are characterized by having relatively complicated high-dimensional inputs
(like images, timeseries, text, or tabular data with lots of features),
and relatively simple outputs
(often just a single probability).
There has been a ton of work on methods for learning interpretable discriminative models,
or interpreting existing discriminative models,
and also a moderate amount of work on how to best evaluate their interpretability.
However, there's been a lot less for generative modeling,
which is kind of the other main branch of machine learning
(examples include autoencoders, topic models, or GANs).
There's been relatively little research on how to interpret generative models,
and almost none on how to evaluate their interpretability.
One exception, though, is the literature on disentanglement,
which provides a number of quantitative metrics
for evaluating how well learned generative models
match the ground-truth model that generated the data,
which is assumed to be interpretable.
However, these kinds of metrics only work for synthetic datasets,
and they haven't received much study in an HCI context.
Our primary contribution is a novel method for both visualizing and evaluating the
interpretability of generative models, which we call interactive reconstruction.
In interactive reconstruction, we ask users to manipulate the
underlying raw dimensions of generative models
to reconstruct target instances to some degree of precision.
We then measure whether and how efficiently they learn to perform this task over multiple trials.
We design this task in order to overcome potential pitfalls
identified by interpretability researchers and more foundational work in HCI theory.
And, as promised, we attempt to rigorously evaluate our evaluation metric in several ways.
The first way we evaluate our metric is by
constructing datasets and models with ground truth,
that is, where we can justifiably assume,
a priori, that certain models are more interpretable than others.
We then test how well our method (and baseline methods) recovers these ground-truth rankings.
Although I'll refer to the paper for more details,
we find that we can recover this ground-truth ordering
from a number of different metrics associated with this interactive reconstruction task,
such as the average rate of completion for each trial within a time period,
or the area under the curve of their error over time.
The same ends up not being true for baselines,
for reasons we explore in much more depth in the paper, and back up with qualitative data.
When we we're dealing with real-world data where there is no ground-truth,
we evaluate our metric in a second way,
by comparing quantitative data from experiments on Amazon Mechanical Turk
with qualitative evaluations of the same models in thinkaloud studies of different groups of users.
In general, we find really close agreement in basically all respects,
including multiple subjective assessment tools,
the consistency of the labels that users assign to model dimensions,
and their raw qualitative feedback. Overall, we really take pains to validate that our metric
is actually measuring model understanding, or at least something intimately correlated with it,
and that this continues to hold for a variety of models, trained with a variety of methods,
on a variety of datasets, with a variety of characteristics.
So, overall, this lets us provide the first
major comparison of interpretable representation learning methods
on non-synthetic data, and also lets us evaluate how synthetic
disentanglement metrics relate to human factors.
If you're interested in learning more about this paper or our group,
you can find more information at the links below.