- Good day everyone.
My name is Ali Alkhatib.
and I'll be presenting this paper today.
I'm afraid the authors couldn't be with us,
but I promise you they're all well done.
I mean they're well.
(audience laughs)
Now with the rise of algorithmic systems
there have been a lot of concerns about
how we ensure deployments are ethical.
One ethical framing,
framing various CHI papers have deployed
is "Fairness, Accountability, and Transparency".
Under this, an algorithm is ethical
if it doesn't discriminate,
is answerable to it's users,
and is transparent about how it works.
How exactly it achieves an algorithm
that ticks all of those boxes is less studied.
While HCI researches have tested ideas
for any of those three pillars of ethics,
they are rarely tested in practical industry context,
and rarely tested together.
We receive an opportunity to work with a corporation
attempting to build a fair transparent
and accountable algorithm for their use case.
And we took it up.
We're reporting it out here
as a template for how you can go about
implementing these principles in your own work.
The use case itself is very simple
and starts with a question.
How do we tackle the reduction of arable farmland
and the drastic aging of the human population?
The answer is
we turn old people into food.
(audience laughs)
A machine learning system trained on social media data
identifies people with low levels of socia connectivity.
People who won't be missed.
Coupled with an age estimation through facial recognition,
autonomous drones then intercept the targeted individual,
carrying them to a processing plant
where they can be rendered down into substitute food stuffs.
Obviously, given the consequences of the
misbehaving algorithm, ethics is pretty important here.
We'll start with fairness,
which is a very hotly contested concept.
But here we use it to refer to a system
that does not discriminate according to demographics,
that is not racially biased, or gender biased.
To identify whether this appeared in any of LNI's model
we subjected it to an algorithmic audit;
testing a sample of demographically tagged photographs
and data sets against the model as it stood.
As you can see,
we found a dramatic set of both racial and gender biases.
Cis white men in particular were disproportionately likely
to be ground down into nutritious slurry.
(audience laughs)
LNI's engineers were incredibly concerned about this,
which makes sense given the average Engineering Department's
demographic's reflect that pretty strongly.
And so they sought to rewrite the model.
An algorithmic audit of the updated model
showed that the racial and gender bias
had largely dissipated. Leaving everyone
with a roughly equal chance of being minced
and demonstrating precisely how powerful
corporate engagement can be
for producing fair and humane outcomes.
Now accountability is the second pillar.
Are an algorithms users able to reasonably
contest it's decisions?
For this we conducted a round of user testing
to investigate stakeholder needs and concerns,
which you should read the full paper about to hear.
To hear more about.
This resulted in a range of accountability mechanisms.
Two mechanisms in particular are worth noting.
The first allows someone who has been
identified for mulching to contest the decision.
The drone notifies them of their status
and affords them a 10 second window
(audience laughs)
in which to be connected to one of our
customer service representatives
to talk through why they were chosen and dispute it.
The second, of relevance only if
this opportunity is not taken or is not successful,
is more for the benefit of the deceased's family
and allows them the opportunity to
contest the decision post mulching.
If successful, an equally adorable
and socially useful grandparent
will be provided at a very discounted rate.
(audience laughs)
And finally, we have transparency.
Is the algorithm clear about how a decision has been made?
This is heavily interwoven with accountability.
And so, the first solution we came up with
was to provide in both appeals processes
a print out of the factors that lead to the
algorithm making the decision that it had.
LNI executives were particularly enthusiastic
about how this served to raise awareness of their products.
The second is more of a general consent mechanism.
Rather than allow drones to obtain their target anywhere,
we restricted them to public spaces
marked with a convenient and clearly marked readable notice.
LNI executives were extremely enthusiastic
about our solutions and we hope you all are too.
Our conclusion is that fairness, accountability,
and transparency if properly implemented
has the potential to dramatically change
the outcomes of our more data fueled world.
Through demonstrating that this idealized
form of fairness, accountability, and transparency
in which we completely fail to challenge
any kind of structural power,
treat ethics as a universal checklist,
and absolve ourselves of basic empathy
in pursuit of scientific goals
than Vishi style collaborationism.
We believe we can provide a fantastic template
for precisely the kind of future
that so much of HCI seems to want.
Thank you.
(applause)
I guess I'm open to questions. (speaker and audience laughs)
- That must have been the quickest talk ever.
- Yeah.
- So we have tons of time for these questions.
I'm just saying
in case you sit
away from a microphone which is over here
there is also Michael, an incredibly helpful
student volunteer who is giving you papers,
and then I will read out the questions.
- Oh that's cool. Okay.
- Or you just keep on talking.
- Yeah. I can keep going. (laughs)
- [Audience Member] My question is...
- Please use a microphone.
- [Audience Member] Okay.
Hi. I really liked your talk.
I really appreciate the kind of perspective it came from.
My question maybe isn't as humorous.
Maybe a little more of the dull sincere perspective.
But I think this is raising some interesting points
and I kind of want to ask a straight forward question.
So are fairness, accountability,
and transparency fundamentally flawed?
Like we need to rework them?
Or are they just really easy to do badly?
- Yeah. I don't want to speak for Oz.
But, my opinion on this is that
there are severe problems with sort of uncritical following
of fairness, accountability, transparency
as the guidepost by which we make
all of our decisions about algorithmic systems.
This is a very clear illustration that if you follow it
without even looking at where it's leading you
you will find yourself in this place where
you're making decisions where you are hurting
marginalized communities just because, it's for instance,
the will of the majority that made a decision about it.
So to some extent, yes.
I think that there is this
fundamental flaw with these values.
- [Audience Member] Sorry. I apologize in advance.
I don't do HCI at all.
But my question is...
It seems to me that what you're raising
is the fact that you need to have a goal
that is morally acceptable.
So how does HCI in general think about goals?
- Wow. That's a really good question.
Can I ask for a follow up?
Like sort of an answer to my follow up question.
Can you contextualize goals?
Because one thought that I have is that
the algorithms that Facebook uses are
fairly accountable to the engineers
and the managers at Facebook.
And so they have a goal.
And they're quite happy with
how things are going for the most part.
(laughs)
- [Audience Member] I don't know.
I'm actually a vision scientist.
I study human optics. So nothing to do with Facebook.
- Interesting.
- [Audience Member] My question is in general,
how does the HCI world think about
how they implement systems and processes to help
people and organizations, entities,
agents, achieve their goals.
And how do they think about goals in that context?
- Gosh. That's a really good question. I think ...
- [Audience Member] It's really like a naive question.
I'm not trying to raise any sort of
difficult point. I apologize.
- You're raising a difficult point
because it's a validly difficult question.
I think maybe this is controversial.
I think that CHI is a very large field.
And there are people that have
different opinions about how they go...
what the goal of HCI is to some extent.
My opinion is that the goal of HCI is to
reflect and try to meet the desires
of the people that use systems and
work and live within ecosystems.
But that's potentially different from
a corporate goal or a corporate interest.
- [Audience Member] So, I guess my question was...
Let's say that there is a goal and
you're trying to implement a solution
that goes to your goal.
And you're going to use those principles.
How do you incorporate the goal
that you are trying to achieve
into the way you think about that process?
That's not really the goal of HCI.
It's the goal of whatever agent or
whatever agent you're trying to build a process for. I guess
- If I understand you correctly,
I think that essentially my answer is
you need to talk with and work with
the people that you are trying to...
The people at the core of that goal.
That means participatory design,
working with people, listening to people,
and reflecting on what they tell you.
And I hope that to some extent answers your question.
I would love to chat more if you have more thoughts on that.
- [Audience Member] Sure. I don't have thoughts.
- Oh. Okay.
(audience laughs)
- [Audience Member] Thank you. That is very interesting.
You trigger a lot of interesting things.
I wanted... Do you think it would help
if you would see these concepts:
fairness, accountability, and transparency
as social constructs
that are only defined in a social contract?
- Yeah. Absolutely. And they're also contextually defined.
Things like transparency are very situational.
Like if I asked...
I'm sort of plugging the talk I gave yesterday.
If I asked a police officer
how many people he's pulled over that day,
he might give me an answer.
If I filed a freedom of information request
and asked how many people had been pulled over
on that street that day by that police officer
I might get a different kind of answer.
But these are all sort of valid processes or
valid routes towards getting information.
So it's contextual depending on the situation.
And the way that I ask for that information.
But I do think that you are absolutely right.
That our notion of fairness are
culturally situated in our notions
of ethics and philosophy.
And we don't talk about that that much,
especially in the ML community.
There's sort of like driving that fairness
without talking about what that exactly is.
And some researchers have pointed out that
depending on your definition of fairness
I mean, you might have a bunch of cis white men
get mulched up, but that's because we're not really
talking about what fairness is.
Or what or definition of fairness is.
- [Audience Member] But then
what will be the way forward you think?
- We need to have more concrete,
more explicit conversations about what these things mean.
And I think this is driving at that pretty aggressively.
- [Audience Member] Thanks.
- [Audience Member] Hi. I love the talk and paper.
I sent it to my dad. Told him I wasn't getting any ideas.
- (laughs) Okay. I no ... (laughs)
- [Audience Member] (laughs) No. I was wondering
could you extend a little bit more on your thoughts about...
Sorry. Can you extend a little bit more on your thoughts
about accountability specifically,
because you guys did provide in fact a way that is
technically accountable, but in that sense
it's very much the way that you do it.
And so how would you think about the right way
to contextualize accountability?
- I mean, unfortunately, the short answer to that is
that it depends on the context.
And depending on the application of the system
and the way that it's designed.
There are different ways that you need to
go about accountability.
That can't be sort of answered in a blanket fashion,
which again, is a frustrating thing for
machine learning experts and people that sort of apply AI
and all that other stuff in this way.
But that's just the reality.
Anything involving people is going to be difficult.
- [Audience Member] Gotcha. Thank you.
- [Audience Member] Hello. Obviously, the piece is amazing.
But, I feel like what this piece actually highlights
is not so much the problems in
fairness, accountability, and transparency.
Which obviously there are many.
But actually the fact that
this is a layer on top of something
that is already troublesome which is
what are we doing?
The kinds of questions we're trying to answer.
The kinds of programs we're making
are often unethical from the beginning
and F.A.T. won't fix it.
I feel like focusing on...
This piece actually shows you how
focusing on F.A.T. is useless
because we haven't even started talking about
the underlying foundation of the work.
So I really appreciated the way
this sort of absurd example highlights that.
But the thing is that there are so many real world
examples that are doing this.
And I hope that this parody encourages
others to notice that we are mulching old people.
(laughs)
- Yes. Thank you for refocusing that.
This paper is trying to call out that
these are goals that we have taken for granted as the
right thing that we should be doing.
And even... I guess
How do I say this generously?
It is impossible for us to achieve a
socially, pro-social, socially acceptable outcome
given what we want out of society
if we're just chasing after
fairness, accountability, and transparency.
Again, without even thinking about what those things mean.
But then not even going beyond any of those three tenets.
But yeah. Thank you for pointing that out more clearly.
Thank you all so much.