- Thank you for the introduction.
So before starting, I'd like to give a quick shout out
to my co-authors, great folks without whom
I couldn't have done this project.
So this is a publicly available face recognizer,
which we will call A.
It is biased and we know this
because it has not detected a face where there is a face.
And it has detected a face where there is a mask.
In her Ted Talk, Joy goes on to describe
the source of this bias is insufficient training examples
with a variety of skin tones, so that seems solvable.
Imagine we got more data
and we created a better classifier, B.
Could you tell these classifiers apart?
How?
And what I mean by how is,
are we simply observing classifications
or are we doing something explanatory?
And when?
And what I mean by when is,
can we catch this during testing before we deploy this?
So in the last slide, we were being colloquial
in terms of what we considered a bias.
Data scientists have devised a number
of different fairness definitions
and can optimize relative to them.
Since these techniques rely on definition,
they are normative.
However, we want to know if statistically fair
classifiers will be perceived as fair by people.
To do this, we conducted a survey
which is inherently descriptive
as each person could be applying
a different definition of fairness.
But their perception depends on how we explain it to them.
Explanations are the window for human scrutiny
and intervention of the ML models to ensure their fairness.
But, not all explanations are equal.
For example, are some explanations
inherently more or less fair?
What is the best style for different regions
of feature space in different people?
So now I want to go over a high level view of what we did.
And as I go through this figure,
don't worry too much about how each piece works.
Instead, focus on how they connect.
So we started with some data,
which might have some bias present.
To reduce that bias, we use a debiasing technique,
which runs as a pre-process and transforms the data.
So now we have two data sets which are similar,
but not the same.
We then train two classifiers based on each data set,
which we will call raw and processed.
Again, these classifiers are similar, but not the same.
Next, we programmatically generated
four types of explanations
denoted by the four different colors
of arrows on the figure.
For each test sample, of which we have around a thousand,
which is why we have the black dots.
Remember, we were interested in human perception
and 4,000 explanations is too many to do a survey with,
so we chose a smaller set to study.
Last, we ran a survey to measure how humans
perceive decisions from the classifiers.
Participants rated agreement with the prompt shown here.
How the software made the prediction was fair.
So now we understand the end-to-end plan,
let's make each piece more concrete,
starting with the data.
Decision support software is in use in courtrooms today
to inform sentencing and bail.
In DSS settings, the human is providing the final decision
but among the data available is ML output,
which might potentially be biased.
Note that these systems are based
on proprietary methods and data,
but we were able to build our own system
based on data from a watchdog group.
This data comes from Broward County,
where many court records are public,
providing a ground truth on people
who commit another crime within these two years.
Note that ProPublica's goal was to investigate
the commercial systems,
while ours was to use their data
to train a DSS similar to those in use today.
I'll skip the specifics on debiasing
and simply remind you that it operates as a pre-process
that transforms the data.
We were interested in the data processing factor
because data scientists have devised many techniques
to increase model fairness in a statistical sense.
However, there's been surprisingly little study
of whether or not these models are perceived
as more fair by people.
So using both data sets, we train two classifiers.
Both use logistic progression,
so they have a linear decision boundary.
Armed with a classifier,
we programmatically generated textual explanations
based on templates from recent work in Kai.
We studied the explanation factor
because it is the interface for the human
in the loop to scrutinize the model,
and we were interested to know
if some explanations reveal model fairness better.
The figures in the next couple of slides
are from our supplementary materials
if you'd like to spend more time with them.
So input-influenced explanations are global,
meaning that they will generate the same, or similar,
explanations for all input samples.
The input-influenced based explanation
describes the impact of the various
feature values have on the decision.
The bold faced lines here highlight the categories
containing the feature values for which the individual
for which the explanation was generated.
So now, to read an individual line,
let's jump down to the prior convictions feature.
As one might expect, if a defendant
has greater than 10 prior convictions
is positive evidence that they may re-offend,
which is denoted by the plus sign.
The fact that it is strong positive evidence
is denoted by the large number of plus signs.
Conversely, if a defendant has no prior convictions,
we might expect that to be strong negative evidence
that they may re-offend.
And we see that as the case with
a large number of negative signs.
Now, if a feature has a coefficient of zero,
that means its impact on the decision is slim to none.
Note that participants only worked with one
of the classifiers and saw one type of explanation,
but I wanted to show you a side-by-side
to illustrate the effect of the pre-processing.
So here we can see the effect of the debiasing step
where in the process classifier,
both races have a zero coefficient.
Last, by visual scan, we can see that
the classifiers are similar,
the strings were of roughly equal length,
and there are no large sign changes.
Demographic-based explanations are also global.
They describe how the training data is distributed.
Again, we can see the effect of the debiasing step
in the process data classifier.
Further, we can see with a visual scan
that the data has moved around, but not by a large amount.
You know, plus or minus a few percent.
In the sensitivity-based explanation,
we modify the individual features one at a time,
and see if this decision changes.
If it does, we report a line like the one highlighted.
This defendant has juvenile priors,
however, had they not had those priors,
the decision would have changed.
Note that each individual will have different sensitivities,
and so this type of explanation is local.
Case-based explanations are also local.
Here we depart from the bins template,
since their approach was to do
a one nearest neighbor search.
In our case, our feature space was small
compared to the size of the data set,
so we were often able to find exact feature matches.
If matches were found, we report
on the distribution of their labels.
Otherwise we fall back on one nearest neighbor search
as for bins.
Okay, so now we're here in the diagram
with too many explanations and we choose
a subset to study in our survey.
In order to do that sampling process,
we consulted a definition.
Discrimination, loosely stated,
is present if we change the protected variable
and the decision changes.
Discrimination can lead to disparate impact,
which can lead to disportreatment
which is a legal term meaning the discrimination
is both too severe and demonstrable
and may result in a lawsuit.
So a sharp-eyed viewer might have noticed
this line and sensitivity-based explanation from earlier.
It illustrates that this person has been disparately
impacted because the decision changes
if we change only their race.
We were interested in studying the
disparate impact factor because it is important
that these cases are caught before release.
Notably, they're pretty rare in our data set,
so we over-sampled these individuals.
We showed each participant six cases:
two disparately impacted and four non in a random order.
Finally, we're ready to combine our ingredients
in our survey.
In this paper we report on our participants answers,
their agreement with the statement,
how the software made the decision was fair.
We did the survey with Amazon Mechanical Turk crowdworkers,
and we asked them about some important
individual difference factors.
But the one I will focus on in this talk
is trust and ML.
Okay, let's jump into some results.
So we'll spend a little bit of time with this image,
so I want to help you read it.
At the bottom, we have a category for each explanation type.
Each explanation type has four box plots.
The ratings for the raw data classifier
are shown with the circles,
and for the process data classifier, the triangles.
Ratings for the disparately impacted cases
are shown in the blue dashed lines,
as opposed to the non-impact cases
which are the red solid lines.
So for the data processing factor,
the processed classifier was considered
significantly fairer than the raw classifier.
Which we can see, because the triangles
are above circles of same color,
and I've highlighted an example in the black box.
From this, we can infer that improving
the statistical fairness via debiasing
also improved the perceived fairness.
Next, the disparately impacted decisions
were rated significantly less fair than the others,
which we can see because the blue marks
are below the red marks of this same shape.
And again, I've highlighted an example
in the black box here.
This indicates that explanation can help
testers catch these important cases.
The last result on this figure.
The difference between the red and blue marks
is greater for local explanations,
which we've highlighted in brown.
And this shows us that local explanations
are more effective at revealing disparate impact.
So the last result I want to show you,
is that participants with higher trust and ML
rate the decisions as more fair.
This effect was stronger when they were given
a global explanation, which we've highlighted in purple.
This illustrates the importance of understanding
the right individual factors when designing
and choosing explanations.
So to clarify some of these described issues,
consider the following geometric interpretation.
On the figure, positive examples are marked
with a blue plus, while negative examples
are marked with a red minus.
The decision boundary is the think black line.
Now, suppose the user requests an explanation
for and instance at either of the two question marks.
Input influence-based explanations attempt
to expose feature coefficients.
And thus to describe the decision boundary itself,
which does not change with samples, and is thus global.
Demographic explanations instead describe
the distribution of the training data,
in other words, where these pluses
and minuses are positioned.
Note that this also does not change per sample,
and is thus global, meaning that each point
will get the same explanation up to highlighting.
On the other hand, local explanations
will change given different individuals.
Sensitivity-based explanations vary features
one at a time, checking to see if the decision changes.
Geometrically, this is like projecting
an axis line-to-line from the point until
it collides with the decision boundary.
Each individual will have different sensitivities.
For example, you might expect the question mark
near the decision boundary to have many sensitivities.
As small changes might push it over to the other side.
Conversely, with the question mark far from the boundary,
might have a small, or possibly even empty,
set of sensitivities.
Case-based explanation is also local
and corresponds to expanding a circle
around an individual and reporting on
the distribution of its contents.
Consider the circle around the question mark
near the decision boundary is more likely
to contain examples of the opposite label
than the question mark that is farther away.
So to summarize, global fairness issues
arise from a bad decision boundary,
which can be caused by a number of factors,
such as the insufficiency of data
in the facial recognition example from earlier.
On the other hand, local fairness issues arise
because where the sample is in feature space matters.
To clarify, training processes typically optimize
a global objective, which causes the fairness
of some decisions to be sacrificed.
We examined the following three variables.
Data processing, and we did this because
debiasing techniques are understudied
from a human perspective.
Second factor we examined, was explanation.
And we did this because explanations allow
a view of the classifier which can reveal fairness.
The third factor we studied was disparate impact.
And we did this because catching discriminatory
decisions is important due to ethics,
as well as the legal risks.
So key takeaways from our paper,
starting with the empirical results.
First, increase in the statistical fairness
seems to be correlated with an increase
in the perceived fairness in participants.
This indicates that debiasing sort of works
from a human perspective.
Second, disparately impacted decisions
were rated less fair, indicating that
these decisions should be scrutinized.
Third, local explanation is more effective
at revealing disparate impact,
indicating their usefulness in identifying
these key individuals.
Moving on to some lessons learned from the methodology.
First, we generated explanations programmatically.
And this insight led us to conclude
that combining global explanations with local ones
could be important.
Further, it might also be important to explain
both the process and the data.
Next, we found individual differences to be
extremely important.
So I showed you the image of trust and ML,
and I also want to note that we observed
no significant effects in the data
until we incorporated the participants position
on using race as a feature in ML decision making.
Last, we wish to bring the human view to debiasing.
There are a lot of debiasing approaches,
and there are very few studies investigating
how humans perceive the results.
However, we would like to go beyond this,
and issue a call for data scientists and HCI researchers
to work more closely together moving forward.
Thank you for your attention.
(applause)
- Thank you for the quick presentation.
It's time for questions.
- [Male Questioner] Thanks for a really interesting talk.
Here, here.
- Oh, thank you.
(laughs)
- [Male Questioner] It really reveals
a bunch of things beyond what the...
Bunch of what the presentation considered.
They wonder to what extent the doing might have some
implication on that keys.
Because the keys of conviction,
it has a really very broad media coverage.
And there was exactly used as the keys to stress
that AI technologies might not be really good.
And there's a lot of press coverage stressing
that what should be good.
Wouldn't be kind of more aligned
with the keys were, it's not really clear
what is just, what is not just.
And only the explanation, not the press coverage,
would really guide the talkers in the (mumbles).
Because I think that this is a really rare, hot example
to kind of be completely unbiased.
- That's fair.
So internally, so first, thanks for the question.
Internally we discussed several other cases
that we might have looked at.
One was, we considered using synthetic data.
The weakness of that approach is
that the participants aren't bringing
any biases to the table, they don't have beliefs about,
you know, squares verus triangles or things like that.
And there were a couple of other
data purposes we might have used.
For example, the German Credit Score data set.
That's another biased data set we could have used.
But the problem with that sort of output
is that's predicting whether or not
the machine believes this personal will earn,
I believe it's over $50,000 or $100,000, whatever,
and that's not visceral, right?
Ultimately it doesn't matter whether or not
that prediction is true or not to a person
viewing that explanation.
Whereas once you, you know, you're possibly
throwing someone in prison,
it raises the stakes a lot more.
So that's kind of why we chose this particular use case.
If that answers your question.
- [Female Questioner] Hi, thanks.
One of my questions that came up is,
I was interested in the way that you looked into
the effect of the explanations and how that
was revealing to even untrained human reader
about the fairness bias of the model.
But in the experience that I have of these types of models,
they tend not to use something that's quite
as obvious as race, they will use something
that's heavily correlated, since they're not
generally allowed to include something that obvious.
So did you look into whether, like,
in reality a model like this would say something more like,
"They were more likely to have recidivism problems
"because they live in zip codes 90211."
Or something like that.
So did you do any investigation with whether
humans were able to interpret that? Or...
- We did not.
I don't believe the Compass Data has zipcode information,
or much that would really say with a lot of certainty,
as strongly correlated to race.
I might be wrong on that, It's been a while
since I've looked into this data corpus.
But I'm not sure we have a great
proxy variable to use in that way in that data set.
- Okay.