Translator: Judy Klass Reviewer: Peter van de Ven
Moldova recently held elections for mayor of its capital city.
A few weeks before, a video was posted to social media.
It appeared to show a news broadcast in Arabic, from Al Jazeera.
According to the Moldovan subtitles, one of the candidates for mayor
wanted to lease the capital to the Prince of Abu Dhabi for 50 years.
Those subtitles, it turned out,
didn't, in any way, match what was actually in Arabic.
The video itself was also manipulated to match the bogus translation.
It was eventually debunked,
but even this simplistic fake was shared over half a million times
in a country with a population of less than 4 million.
Our civilization is built upon our ability to distinguish fact from fiction.
Think about it.
What would happen to our communities, our businesses, our governments
if we can't trust what we read, hear, or see?
What we know, what we believe to be true determines how we act,
how we elect our leaders, how we make decisions,
change policies, and start wars.
But soon, things are about to get real weird.
Right now, as I'm talking to you,
the technology exists to steal your voice from previous recordings
so I can make it sound as if you said anything I wanted you to.
We're entering an era in which our enemies
can make it look like anyone is saying anything at any point in time
even if they would never say those things, but someone else would.
Someone ...
like Jordan Peele.
So I can take a video of you
and replace all of your facial expressions with those of another person
so that it looks like you are speaking, frowning, and shouting
along with that fake voice.
I can even have that fake you appear to interact
with a real environment of an existing video.
And then I can create a social media army of automated accounts
to spread this video,
complete with realistic photos of people who have never existed.
I'm a technologist in engineering, and I find this incredible
and terrifying.
Welcome to the world of AI-assisted audio and video manipulation -
to the world of what is now called
Deep Fakes.
All this was possible before,
with skilled actors and enormous production budgets,
but it could soon be as easy as downloading an app.
So I got my first degree from MIT in 2009,
and even before then,
I've been wrestling with techno social demons -
sort of walking this fine line between loving new tech
and curiosity about its negative impacts.
And in May 2016, this became all-consuming.
I connected some dots that deeply concerned me.
It seemed that the extent of online misinformation was not only growing,
but that all the incentives were aligned
to make this a much, much bigger problem very quickly.
I felt like I was watching an epidemic,
except no hospitals were even aware of the problem -
not to mention actually trying to slow the outbreak.
I decided I couldn't live with myself
if I didn't at least try
to get Facebook and friends to address this growing threat.
I dropped my technology consulting work, found allies, and focused on this.
Over two years later, and companies are finally taking these problems seriously,
but the epidemic of misinformation
is evolving into an even more dangerous strain.
Consider for a moment how someone might use deep fakes for money or mayhem.
We can use recent history as a guide -
and I don't just mean the U.S. 2016 election.
There were nine lynchings in Indonesia
shortly after a rumor spread online about kidnappers.
False Facebook posts tore apart Muslim and Hindu communities in Sri Lanka
and enabled ethnic cleansing in Myanmar.
How much worse could it be
when a provocateur can easily create video proof that a rumor is true.
Or will criminals walk free
because they can claim that a real video was manipulated?
What happens when a significant chunk of the population
just doesn't trust anything,
because anything important can be believably faked.
It's not the one-off fakes that we should worry about the most,
it's the way this technology is a threat multiplier.
It can increase the overall level of outrage, hatred, and violence
in a world full of powder kegs just waiting for a spark.
And at the same time, this tech weakens our ability to contain any explosion.
When people can't discern reality, it's hard to govern, to use diplomacy.
Whether you're concerned about climate change or pandemics,
Skynet or drug lords,
you know that countries need to work together to address global challenges.
But how can we work together if we can't trust anything?
OK, by now, I've probably perhaps terrified you.
I'm sorry about that.
(Laughter)
You need to know,
but our market for reality could fail.
Dystopia isn't fully inevitable, though.
We can still prevent the worst outcomes -
if we start now.
But what can we do?
The natural thing to do about fakery is to create detection systems.
Unfortunately, in the long run, this may not work.
The easier fakes are to detect,
the faster it is to train an AI to make an even more convincing fake.
Forgers are likely to win this battle in the end.
That said,
it is a very useful step for now.
But even the best detection systems alone won't change the impact of a fake video,
and some evidence suggests that the obvious idea of labeling fakes
can backfire.
So what's a more comprehensive way to think about this?
Well, the goal here is not to stop all fakery.
It's to prevent our institutions from crossing over that threshold
into so much failure that they can't even function,
to avoid tipping into infopocalypse.
To do that, we need to shore up the marketplace of ideas.
We need to keep the ROI of fakery as low as possible
and ensure that reality stays impactful.
This means protecting our knowledge pipeline,
addressing that flow of information
from creation to distribution, to belief, to action.
How easy is it to create a manipulated or a real video,
to distribute that video so people actually see it.
What impacts whether or not we believe the video?
What determines if we take actions based on our beliefs?
If I was to go through each step of this pipeline
and all the things we can do to try to impact it,
that would be 15 more talks, so not going to do that today.
Instead, let me use an example
to illustrate how we can think about creation and distribution
so I can show you the kinds of thinking we can bring to the table.
Let's pretend that Steve wants to discredit a politician, Lucy.
Steve wants to make a video showing Lucy admitting to blatant corruption.
The first that we can consider is creation.
We can't stop this technology from existing,
but we can make it harder to create fakes for malicious ends.
What if investors, engineers, even app stores
decided they didn't want to enable deep fake technology?
If it only took Steve a few taps to download and use an app
to create that fake video of Lucy, he's far more likely to do it.
Making it just a little harder to create a fake
is not gonna stop the Kremlin.
It won't stop anyone with a computer who really wants to manipulate video,
but it might help in Myanmar
or any other country where most people just have phones.
And even small obstacles to usage
can have a significant impact on what people actually do.
We also need to make it easier
for a witness to film a real video of corruption
and show that it's real.
Camera and phone companies can help here.
We may be able to make authenticity infrastructure
that lets people prove where and when a recording was captured.
But OK,
now that our corruption video has been created,
either fake or real,
what determines who sees it?
Distribution platforms.
This means social networks, video uploading sites, messaging systems,
recommendation engines,
and all of the ways that these sorts of things can be manipulated.
This is all about the incentives,
and right now in our world,
sensationalism is rewarded because it attracts attention,
it makes money, manipulates beliefs, and it drives people to act.
A convincing fake can be naturally viral.
Wouldn't you click on a video of Trump or Clinton taking a bribe?
A platform like YouTube shouldn't stop people from being curious,
but it can try to ensure that it doesn't recommend
sensationalism and forgery more than reality.
And it can invest in systems to recognize and reward good actors over bad.
This is also where that forgery detection and authenticity infrastructure
that I mentioned earlier can help,
and more traditional journalism and fact-checking.
(Applause)
So after our video is distributed,
then the audience either believes it and acts on it, or they don't.
Here we need to involve almost every aspect of society,
from actors to diplomats and especially teachers.
There are a lot of minefields here.
We're going to need to make some very tough calls
on what a healthy ecosystem looks like.
These proposals are far from perfect.
Some may even be worse than the original problem.
We have to keep our eye on the prize -
functioning civilization, functioning democracy -
and ensure we have the checks and balances in place to facilitate that.
And one thing that can help with creating those checks and balances
is integrating responsibility into our processes and our organizations.
Researchers and technologists have repeatedly messed up - badly.
We built systems that have caused immense harm
and have only acknowledged a problem long after the damage has been done.
This can't keep happening;
what we create has tremendous impact,
and the good doesn't always outweigh the bad.
Companies like Google already have dedicated security teams.
They make sure that your data isn't hacked,
and they have power.
They can say to any team, "You need to fix this now,"
and stop a product from being rolled out to billions of people.
We need this for more than just security.
Every company that has a significant impact on public discourse
should have well resourced and empowered teams
100% focused on identifying the ways
that their systems can have negative impacts.
(Applause)
Individual product teams can also have impact advisory boards.
The goal would be to involve external experts in different regions
that could be consulted,
as a product changes and as the world changes,
to answer that question:
"How might things go horribly wrong?"
We need similar support for responsibility in AI research.
If we want companies and researchers to be responsible,
we can't just hope they are going to magically understand
the complex consequences of new technology.
We need to ensure that they have the incentives, the infrastructure,
and the resources to make responsibility possible.
I could go on, but look, I got just one goal here today,
and that is to tell you not to trust anything you see or hear,
just like my favorite news outlet, RT QUESTIONMORE.
No, RT, as in Russia today, a Russian propaganda channel.
No, of course, that is not my goal.
If we don't trust anything, the manipulators win.
Democracy requires trust.
As I give this talk,
all of this AI technology is not out of the lab yet.
It's far from perfect; we have some time.
My real goal is to ensure that we prepare ...
now;
that we work to improve the ways that we distinguish what's real and what's not;
that we create crucial new institutions, technology, and governance;
that we make reality as easy, efficient, and incentivized as possible.
Thank you.
(Applause)