Hi, I’m going to talk about how we design machine learning models when annotators disagree
on what the right answer ought to be.
Imagine you’re training an AI model to remove toxic comments from an online community.
Should this training example, which is making some aggressive comments about religion, be
an example of content to remove?
or an example of content to keep up?
This is a tough one.
It turns out that in the public dataset we took this from, half of people feel each way
about this comment.
So in today’s typical supervised learning pipeline, we’d then model a sort of aggregate
pseudo-human, predicting the majority vote label while ignoring annotators who disagree
with the majority.OR maybe you could do a bit better, and decide that your AI should
predict a 40-60 distribution.
But if you’re a practitioner who’s then got to decide whether to remove this comment
or not, what do you DO with that information?
There’s not really a good answer.
So lets ask a different question: whose voices is our model emulating here?
It turns out that if our training data were entirely labeled by white males, the distribution
slightly tips toward leaving the comment up.
But, if our data were labeled by groups that are often targets of online harassment — women,
Black individuals, — you’d see the distribution swing toward labeling this comment as toxic.
And voice matters.
Healthy spaces and communities have their own distinct norms and values.
Non-parents shouldn’t decide which topics are fair game in a parenting forum.
And the political problems taking place in the star wars universe require a very different
standard of discussion than those taking place in a forum on saudi arabia.
One model trained from one dataset won’t fit all these communities, and just as importantly,
nor will it allow for an opportunity to figure out what WILL fit.
Because today, whose voice the AI emulates is a decision baked into this training data
here, and then swept under the rug in AI systems.
As a result, for many practitioners, the answer of whose voice their AI is emulating is left
implicit in the data collection and training procedure.
In this talk, I’m going to introduce Jury Learning, a supervised learning architecture,
a technical and normative approach, that lets practitioners articulate exactly the voices
they want determining the behavior of their classifier, and in doing so, often changes
an AI’s predictions.
This is an important problem because, in user-facing tasks ranging from not just toxicity detection
but also from poster design to cancer treatment, ONE THIRD OF experts disagree with a TYPICAL
decision.
So if we could make voice explicit, we could also weigh treatment opinions from doctors
with a particular expertise.
Or to select design decisions from designers trained in a particular school of thought.
Or, maybe a practitioner isn’t sure WHO to side with, and could reason over the decisions
that different people would make.
What we need, and what our research introduces, is an interactive system where, when annotators
disagree, we can choose which voices our classifier listens to.
So how could such a system be possible?
Well here’s our key insight: datasets are ultimately made up of individual people, so
instead of modeling some aggregate pseudo-human, let’s model INDIVIDUAL PEOPLE.
We’re gonna demonstrate that it’s possible to model individual voices in your training
dataset, enabling us to then design a system for interactively exploring, tuning, and shifting
the behavior of the classifier, by explicitly choosing which annotators our classifier will
emulate, in what proportion.
Let’s walk through the jury learning architecture.
Given an input, practitioners start by defining a jury composition, selecting from characteristics
in their dataset.
For instance, if a dataset contains each annotator’s political leanings, then the practitioner
can say they’d like, for instance, 2 liberal jurors.
We’ll also choose two jurors who are parents with high school diplomas.
And we now have these four empty JUROR SLOTS, and we need to choose individual annotators
from the dataset to fill them.
We’ll start with ALL the annotators in the dataset.
Again these are real, individual people.
And just like in any typical dataset, they’ve each only annotated some small subset of the
examples in the full dataset.
We can now filter to all the relevant annotators in our dataset; jury learning currently requires
that datasets contain these explicit labels about annotators.
We then randomly select a matching annotator from the dataset to fill each juror slot.
And the AI predicts how each individual juror would vote, using all the data we have about
their annotations and the annotations of other similar jurors, giving us this jury’s verdict.
I’ll talk about the model architecture that powers this in a minute.
And of course this can work with regression too, where we predict a numeric value instead
of a yes or no class.
So randomly selecting one set of jurors gives an idea of what one particular set of people
might think.
But jury learning ENABLES something impossible with real-world juries: we can re-sample jurors
many times, creating many PARALLEL juries And then each parallel jury has its own predicted
verdict.
And so we get this DISTRIBUTION of jury verdicts.
Each dot is a re-sampled jury.
For a single final decision, we can take the median, giving us a median of means estimator.
And, by modeling individual annotators, we’ve enabled interactive sensemaking about the
nature of disagreement.
For instance, we can plot the distribution over our model’s predictions for EVERY annotator
in the data set who we COULD have selected as juror, versus our model’s predictions
for the actual jurors we selected, showing us that, for instance, democrats are far from
monolithic in their views about this comment.
Alright, so thats the interaction.
Now walking back to that prediction step, how can we predict each annotator’s response
to some unseen example?
We assume we have a typical classification dataset: each of these jurors are again just
annotators, and like in a typical dataset, we assume annotators have some mostly disjoint,
occasionally overlapping set of training examples they annotated, and a bunch of training examples
they didn’t annotate.
And of course we have this not just for annotators we selected as jurors, but for every annotator.
AND for every annotator, we have a bunch of characteristics.
And we want to predict each juror’s response to some unseen example.
To do this, we designed an architecture that co-trains a transformer-based text CLASSIFIER
to use as content embeddings within a Deep and Cross Network recommender system, which
learns embeddings for each annotator and characteristic.
And our technical evaluation found this architecture achieves state of the art performance.
Alright.
Before we continue, it’s important to make clear that while we view Jury learning as
a step forward, it is by no means a panacea.
It will create or, at least bring previously hidden problems, to the forefront.
One challenge: jurors can be biased!
For instance, selecting a woman doesn’t necessarily mean you’re getting a juror
who supports women’s rights.
our perspective here is that there’s two orthogonal and complementary worldviews here.
Today, most approaches assume the world is broken and applies post-hoc statistical transformations
to try to de-bias model outputs.
With jury learning, we’re saying it could also be really useful to select or train the
people who’s voices should influence the model, because its often more tractable to
do the latter.
A second consideration: who determines the jury compositions?
Let’s first note that every dataset ALREADY HAS an implicit jury, they already have some
existing distribution of annotators influencing classification decisions.
What jury learning does is make that fact explicit and visible and easily changeable.
It provides a new way to communicate and debate whose perspectives are being included.
Our take is that when using jury learning, practitioners should make jury compositions
transparent, and provide stakeholders with a voice in selecting them.
So now I want to touch on our field evaluation: we gave jury learning to 18 moderators and
one question we asked is: do moderators author juries that are more racially diverse than
the composition of a typical dataset?
Here’s what the composition of a typical dataset looks like: its 74% white.
But we can see that, when we gave moderators jury learning, they chose to create juries
that represent a much broader cross-section of society.
Another question we can ask is: do the predicted verdicts of juries composed by moderators
make different decisions than a standard state of the art classifier?
We found that 14% of classification decisions change.
The most likely to flip were comments that were the most divisive.
And these comments on the edge are really important because behaviors on edge cases
become de facto platform policy.
So that was Jury Learning, and we hope that when annotators disagree, jury learning empowers
practitioners to make explicit, carefully considered choices about which voices their
models reflect.
Thank you!