Transcriber: Naynne Rissardo Reviewer: Emma Gon
Let’s play a game.
The game is simple and it’s not really a game at all.
For one, there's no winners or losers, and we're all on the same team.
And I’m going to make a computer play most of it for us.
But since it’s my talk, we’re going to play it anyway.
We start simply with an equilateral triangle
whose vertices we’ve labeled one, two and three.
We choose a random point on the interior of the triangle,
and then we roll a dice.
On a roll of one or four, we choose vertex one.
On a roll of two or five, we choose vertex two.
And out of all of three or six, we choose vertex three.
We draw a point halfway
between our starting point and the vertex we chose.
And then we do it again and again and again.
Each time starting at the new point,
choosing a vertex and drawing another point.
Now, so far, this game doesn't really seem to have a point.
Actually, it has at least three points so far,
but it doesn't seem very interesting.
So in order to try to make it a little bit more engaging,
we're going to increase to 100 points.
But before we do this, I want you to take a moment
to think of what you believe the end result of this process will be.
Have an image?
100 points doesn't give us a ton of information,
but it does suggest some ideas for it.
And in order to make things a little bit more engaging,
we'll go to a thousand points, as you can see up here.
And a thousand points gives us a little more information.
To get more of a sense of what happens will increase by a power of ten
going to 10,000 points and then to a million points.
We’ve created a complex mathematical structure,
a fractal called Sierpinski Triangle.
And the process you’ve observed is a process called the Chaos Scheme,
coined by mathematician Michael Barnsley.
Barnsley was creating a processto develop fractals using probabilistic odds.
It is unlikely when I asked you to imagine what you thought might occur,
that you imagined that this structure would occur.
In fact, it’s difficult to understand how the set of simple rules
that we generated led to the complex structure that emerged.
It might surprise you to know that we see this sort of behavior
all the time across a diversity of scientific and mathematical subjects.
A simple set of rules repeated time and time again,
an algorithm leads to self-organizing, complex behavior.
I do not believe that is an overstatement
to say that algorithms are one of the most important drivers
of human behavior in our modern world.
But before we discuss them, we should probably define our terms.
So, algorithm: a finite sequence of well-defined instructions
used to solve a problem.
You may be thinking that algorithms are not very important to your life,
that you don’t encounter them.
You are likely not a mathematician like me or a computer scientist.
And you might think that these structures don’t come up for you.
But imagine something that you do probably every day -
run a Google search.
Google dominates a lot of our life these days,
but its original and perhaps still most used structure is its search engine,
particularly its PageRank algorithm.
Most estimates place Google at about 92.4% of the search engine market
that there are 3.5 billion Google searches per day
and that there are 70,000 Google searches per second.
What this means is for the vast majority of information on the Internet
and for the average user, the Google PageRank algorithm
is the arbiter of what information that an user sees or does not see.
And its PageRank is a critically important piece of information.
The Google PageRank algorithm works off a set of simple principles.
When it looks at a page, it first evaluates the number of backlinks,
the number of pages that links to that page.
Next, it evaluates those pages for authority and relevance,
and it compiles all of this into a score and then delivers search results.
This is, of course, a gross oversimplification,
but we’re not concerned today about the details of the PageRank algorithm,
but rather about the influence that it wields on our lives.
In particular, we might be worried about the concept of bias,
whether the Google PageRank algorithm
prefers certain information over other information.
If you’ve been paying any amount of attention,
you know that this has been the cause of an immense amount of public concern
over the past couple of years
It's resulted in congressional hearings,
in academic papers and miles of Twitter and Reddit threads.
The problem is that, like most algorithms,
Google's PageRank algorithm is a proprietary secret.
It is unable to be known to us.
And so we don’t have the option to understand exactly how it works.
We can only work with the algorithm from the outside in
trying to work with it to understand how it treats different types of information.
This makes it very difficult to generate one to one,
apples to apples comparisons between similar subjects
to see if the algorithm treats them similarly
because we don’t know what it values.
The result? A simple process that we run 3.5 billion times per day
controls a large amount of access and influence to information,
and we do not know how to directly evaluate its effect on us.
Google is not the only source of algorithmic bias in our world.
And we'll consider three scenarios today about ways that algorithmic bias happens.
We'll consider pre-existing bias feedback loops an emerging bias
and technical limitation bias.
Preexisting bias is bias that's created by the existing institutional
and cultural ideologies that permeate our world.
The engineers and programmers of algorithms are influenced
by these prejudices, and so when they program things,
they can uphold and institute existing marginalization.
In the worst case scenario,
the future uses of that algorithm can continue to perpetuate
inequality beyond the original use.
A great example of this, the Amazon resume screener tool,
a tool created to determine
whether candidates would be successful at Amazon or not.
The majority of programmers of that algorithm were male,
and it was trained on a predominantly male training set,
which means that the algorithm prejudiced towards male candidates
and systematically prejudiced away from qualified female candidates.
The result - Amazon’s discontinuation of the program
due to the fact that the bias could not be overcome.
Emergent bias in feedback loops happen when we take algorithms
from a certain scenario and apply them across new and changing contexts
without accounting for those contexts.
In particular, feedback loops happen
when an algorithm generates real world data
is then fed directly back into the algorithm to generate more data.
Consider your social media feed.
If you’re old like me, your Facebook feed.
If you're young, like you, your TikTok feed,
which generates data based on clicks to generate engagement data.
Now this bias is towards content that is explosive, potentially extremist,
though maybe not actually true or informative
and it can trap people in ideological feedback bubbles
where they only see their own viewpoint
and it can shuttle people towards extremist viewpoints.
In addition, marginalized communities are already underrepresented on those feeds,
and so they generate less engagement.
But generating less engagement means that the algorithm biases against them,
causing them to show up less, causing and generate less engagement,
causing them to show up less,
perpetuating a cycle of further marginalization.
Finally, we might consider technical limitations
bias that occurs due to constraints on programming,
computational power and design of algorithms.
Police forces across the US are using facial recognition algorithms
at a greater and greater rate.
But due to limitations in the image processing of those algorithms,
we know that they bias against Asian and black faces
at a rate of 10 to 100 times of their white counterparts
and women more than men.
The result - hundreds to thousands of false arrests due to misidentification
in meshing people of color and women
in a justice system that saps their psychological,
emotional and financial resources.
This bias not only limits the implementation of this algorithm,
but it undermines the very fabric of the justice system
that it is meant to uphold.
What should we conclude from this?
Algorithms are clearly a part of our lives,
and so they are a matter of personal and public concern.
They give rise to complex and unexpected behavior
that has both positive and negative implications for their implementation.
And algorithmic behavior is difficult to analyze.
We are unable to understand entirely how they influence us.
We have handed over an immense amount of our decision making power to algorithms.
They decide what restaurant we should go to, what Netflix show we should watch.
They tell us the route that we should take using our GPS.
They recommend what stocks to pick.
And in their most powerful implementations
they recommend to world leaders what choices they should make
that could affect hundreds of thousands of lives.
Given this, as citizens of a digital and civic world,
we should expect more of the companies
and organizations that implement algorithms.
We should ask for transparency in how algorithms are implemented.
We need ongoing analysis of how algorithms are implemented
and how they work, and we should be conscious of algorithms affecting us.
We should understand how they affect us.
But beneath all of this, beneath the computational nature
of our modern world, we should remember humans.
Humanity and human emotion are lost
when we give too much of our decision making power
over to algorithmic processes and data modeling.
Human emotion, human compassion and common sense
are powerful decision making tools
that we need to use to lean into instead of away from.
There is no algorithm that will full capture the hopes,
dreams, needs and desires of an individual human.
And so as you leave this talk,
as you head into a world full of Facebook feeds and Google searches,
I hope that you remember the human.
I hope you remember your own hopes, needs, desires.
And I hope that next time
you let the algorithm do a little less deciding for you.
Thank you.