[MUSIC]
When I was nine, >> [LAUGH]
>> My favorite pastime wasn't recess or
video games or even sports.
It was statistics.
>> [LAUGH] >> That taught myself alone in my bedroom
sorting basketball cards from best to worst,
using the numbers on the back of the cards.
It was back then, that I've learned a very important lesson of stats 101, that
I could make the numbers see just about anything I wanted.
>> [LAUGH] >> My childhood idol
was a basketball player named Anthony Penny Hardaway.
He played in an era with perhaps the greatest of all time, Michael Jordan.
But this introverted fourth grader was determined to build a statistical
model that proved that in fact Penny Hardaway was the best player in the world.
>> [LAUGH] >> Now,
there were several common statistics on the back of a basketball card,
the most important of which is points per game.
Several players, including Jordan, averaged more points than Penny.
So I started with a different metric, rebounds per game.
And I eliminated anyone who averaged less than five rebounds.
Now this narrow the pool but some survivors,
including Jordan, still average more points than Penny.
So I narrow it even further with assists per game.
But Michael Jordan remained.
>> [LAUGH] >> But
now that year I knew that Michael Jordan missed several games due to injury.
Penny Hardaway, on the other hand, played in all 82 games.
Surely, I convinced myself health and durability matter.
>> [LAUGH] >> For the best player in the world, and
I eliminated anyone missing more than four games.
>> [LAUGH] >> And so,
after games played rebounds, assists Penny Hardaway finally led the pack.
I call it, The Penny Algorithm.
>> [LAUGH] >> It is how I prove that my
favorite basketball player was indeed the best basketball player in the world.
Simple, indisputable statistics.
>> [LAUGH] >> That 20 years later,
I can tell you that the logic that the Penny Algorithm used is actually
quite similar to the logic that your spam filter uses to keep your inbox clean.
In fact, many machine learning algorithms transform our daily lives
using such logic.
But you probably shouldn't trust the model built by nine year olds.
And more broadly, and more importantly, as I'll show,
we should all have the healthy distrust for models and algorithms.
Take Amazon's hiring algorithm abandoned in 2017 for gender bias.
This algorithm learn the characteristics of Amazon's best employees and
apply that pattern to hundreds of thousands of job applicants.
The problem was that at the time, Amazon's top employees were predominantly male.
So the algorithm learned to discount female applicants.
Even when they've removed gender from the model,
the algorithm learned to use proxies for gender and
discounted applicants for phrases such as women's swim team.
Or attendance at an all women's school.
Now many people hope the objectivity of machine learning,
will counter things like gender bias in the workplace.
But the truth is,
even machine learning algorithms can be vulnerable to human bias.
On our mission to fight biasing algorithm is Joy Boom Weenie, an African American,
computer scientist and the creator of the algorithmic Justice League.
As a graduate student using the same facial recognition software
that's in our phones today.
She noticed that the algorithms that worked on her Caucasian male
lab partner did not work on her dark skin or dark features.
The problem, was that the algorithm that the model was trained
on a dataset that lacked meaningful diversity.
And the result was that the same algorithm that works 99.7%
of the time on my face works just 65% of the time on Joy.
We all hope that the technology we create will have benefits that can be shared by
all people.
But even the benefits of machine learning are vulnerable to human bias.
Last fall, consumers were outraged when Apple Card offered better terms to
male applicants than female applicants.
Sometimes 20 times better,
even before collecting relevant financial data such as credit score or income.
The Internet exploded with similar such examples once this story surfaced.
But the problem is that those algorithms do exactly as they're told.
They maximize revenue by optimizing for things such as click through rate and
engagement.
Both of which vary by gender.
Oftentimes, the goal of our model obscures its consequences.
And the truth is,
machine learning algorithms can have significant unintended consequences.
Now, this technology has transformative potential to live billions out of poverty
and to accelerate human progress, like few technologies ever before.
Indeed, it's why I've spent most of my time at Stanford studying it.
But unlocking this potential comes with risks and trade offs, and
knowing when to trust the model and
when not to requires that we first understand these risks and trade offs.
But assuming you're convinced to have a healthy distrust for algorithms,
how then do we know when to trust them?
Here are three questions to ask.
I call them Penny's Trustworthy Framework.
>> [LAUGH] >> Number one, what is this model true of?
Every model just like every one of our perspectives is built upon assumptions.
Bias often lurks within these assumptions.
Number two, who or what is missing?
Removing gender from an algorithm does not prevent gender bias.
In fact, it's often how bias is found measured and mitigated.
In number three, what are the consequences of this algorithm.
Any one of us could find ourselves in a position through our actions and
decisions to prevent the scaling of unwanted bias.
Consider my own experience.
While I was in the United States Air Force I built a model to predict how students
would perform through a nine-month flight training program in the F-15.
This model allowed us to identify struggling students early and
invest in them appropriately.
As flight commander, I mention one such student that the model predicted
with high certainty just would not graduate the program.
I knew this data better than anyone.
I'd spent hundreds of hours training this model myself.
But I also knew that student is a human being.
I knew he was a husband.
I knew he's a brand new father.
I knew that he struggled with air sickness on every flight,
just as I had four years prior.
While the algorithm encourages us to give up hope, deep down I'd empathy for him.
Should I trust the model or not?
To the rise of machine learning and artificial intelligence,
many of you will face the same question of whether to trust the model or not.
I decided not to.
But what matters more than my decision, is that in the future each of us
takes responsibility for this decision, and ownership of the consequences.
But before you decide,
remember Penny's Trustworthy Framework, >> [LAUGH]
>> And that shows you three questions.
What's this true of, what's missing and what are the consequences?
Because making better decisions with algorithms starts with trusting them less.
Thank you.
[APPLAUSE]
[MUSIC]