[Narrator] This
is Donald Knuth.
[♪ classical music ♪]
[slides click]
Professor.
[slides click]
Author.
[slides click]
Math savant.
[slides click]
[♪ dramatic organ music ♪]
Pipe organist.
And this
is Donald Knuth's life's work,
"The Art of Computer Programming"
clocking in at well over 3,000 pages.
It's considered by many
to be a founding text of computer science.
[♪ rousing classical music ♪]
Bill Gates once said, "If you can read the whole thing,
send me a resume."
[mouse clicks]
[film projector rolls]
[♪ upbeat music ♪] But in 1964,
when Knuth was still in the middle
of writing volume one,
there was no guarantee
his opus would ever see the light of day.
[film projector rolls]
That's because the same perfectionist streak
that drove Knuth
to analyze his college basketball team
and optimize his home's kitchen
around the trash can
was getting in the way of actually publishing anything.
Checking and rechecking every page,
Knuth blew through deadline after deadline.
His editor demanded progress,
his family missed him
and still, volume one remained unfinished.
Finally, he arrived at a solution.
If he could not make his book perfect,
he would make it perfectible.
And so on page 12 of the preface,
he added a short note.
[typewriter keys clack]
”I will greatly appreciate receiving information
about any errors noticed by the readers
so that they may be corrected as soon as possible
in future editions.”
[paper rips]
It worked.
[♪ upbeat music ♪]
As soon as the book hit shelves,
error reports started coming in.
Mathematicians corrected flawed equations.
Knit pickers pointed out punctuation errors.
With each find,
Knuth mails out a reward of 256 cents.
That's 1-0-0 in hexadecimal,
in case you didn't know.
So far, Knuth’s got more
than $22,000 worth of checks.
They've even become a bit of a collector's item.
More get framed than cashed.
And with each new edition of his book,
fewer and fewer errors remain.
Flash forward to today,
and the software engineers responsible
for the apps and services
billions of people rely on
face a conundrum similar to Knuth's.
[traffic noises]
[machines buzz]
[keyboard clicks]
How do you make your code perfect
without delaying progress indefinitely?
You follow in Knuth's footsteps
and start rewarding the people that hunt down your mistakes.
[♪ anthemic music ♪]
When it's your job to keep billions of people safe online,
you have to live and breathe
and see the internet just like the attackers do
because the only way to stop a hacker
is to think like one.
[♪ soft music ♪]
This is Eduardo Vela,
Security Engineering Lead at Google.
[Eduardo] Yeah. Hello. [laughs]
[Narrator] Eduardo doesn't have a Knuth check,
but he has found thousands of errors
in all kinds of software—
including Google's.
[Director] Does Google have bugs?
[Eduardo] Sure. Google has bugs.
Google has vulnerabilities.
Everything that we do in everyday life
that relates to software,
we are putting some trust
on whoever wrote that code.
[♪ upbeat music ♪]
We as Googlers,
we recognize the responsibility
of the faith that people place in Google.
We have a team of people
that will look at the codes,
that will look at the products
and we look for bugs.
But then inevitably,
there is going to be something
that we didn't know about.
[Narrator] Coming up with new ways to keep bugs
out of Google's code is a full-time job.
One that's held by Christoph Kern,
Principal Engineer on Google's Security Foundations Team.
He knows more about bugs than just about anyone.
[Christoph] There's basically two kinds of bugs.
There's functional bugs
where the program just doesn't work correctly,
like some UI element.
You click on the button,
nothing happens,
that kind of thing.
[error noise]
And then there's security bugs
where the program doesn't work correctly,
but it has a security implication where,
for instance, somebody else might be able to get data
that they're not supposed to have.
[♪ soft music ♪]
[Narrator] He's talking about bugs like—
[Christoph] Memory corruption vulnerabilities,
buffer overflows,
injection bugs,
cross site script injection,
SQL injection,
predictable identifiers,
various authorization vulnerabilities.
I don't know.
Let's leave it at that, maybe. [laughs]
[Narrator] Fair enough.
But if we already know about all these bugs,
why do they keep popping up?
[Christoph] One particular challenge
with a software that's being delivered
over the internet is that it's so malleable, right?
Many web-facing applications basically
get delivered a new version every couple weeks
or even every week.
So the software is constantly changing.
Every time there is a change,
there is a possibility of introducing a subtle flaw
that could potentially have security implications.
[Narrator] So change causes bugs
and code is always changing.
Faced with this problem,
there are two approaches companies can take—
hope no one finds their bugs
and threaten to prosecute those who do,
or think like Knuth
and ask the community for help.
[♪ suspenseful music ♪]
[Camille] So there are a lot of people who,
for the intellectual stimulation of it all,
tend to search for vulnerabilities in systems.
[Royal] When you think back to this
community of hackers,
one of the things that they
loved doing
was finding something
that no one else has found before.
[Tim] I think it just comes from
an innate sense of curiosity,
wanting to figure out how things work.
[Eduardo] Literal translation of hacker
in Spanish is
"pirata informático,"
which means "information pirate."
I think it represents better what we do
when we talk about bug hunting
or vulnerability researchers.
You are looking for clues,
you're looking for hints
and you're trying to chase weird behavior
into something that is like a bug or vulnerability.
That's why it's called bug hunting
and yeah,
it's like hunting
[laughs] for bugs.
We have this program
called a Google Bug Hunters Program
in which we ask people in the world
that are able to find security issues
to tell us about it.
[Narrator] Across more than 100 countries,
thousands of amateur and professional hackers alike
have answered the call,
filing thousands of bug reports every year.
Over time, a few of these hunters have risen to the top,
the best of the best.
[Eduardo] Bug hunters are from all around the world.
They come from as many countries as you can imagine.
Sometimes it's very difficult to ship them gifts,
and that's usually how we found out
where exactly they are from.
[♪ western music ♪]
There is one guy named Callum. He's from the United Kingdom.
[Callum] I hack companies in my free time. [laughs]
It's the easiest way to say it. [laughs]
[Eduardo] There's Yesenia from Mexico.
[Yesenia speaks in Spanish]
[Eduardo] We have João Lucas Melo Brasio.
He used the money that we gave him
for rewards
to build companies.
Now he has many companies. [laughs]
He has houses and [beep].
Sorry.
[laughs]
He has houses and "stuff."
[Narrator] Look at the top of the leaderboard,
and you'll find Tomasz Bojarski,
the number one ranked bug hunter in the world.
[Tomasz] I'm number one since 2016,
and I'm not really putting any effort
into keeping number one.
I don't know why. People are so lazy, I guess.
[laughs]
[Narrator] But hot on Tomasz's heels
is a new generation of hackers.
One that entered the hunt before they could even drive.
Meet Ezequiel Pereira,
hacker since homeroom.
[♪ mellow music ♪, school bell rings]
[Ezequiel] When I was in high school,
I decided to try
to find like, vulnerabilities
in the high school website.
Bringing the site down
or editing some page to say,
"Oh, there are no classes," or something like that.
Then I got caught.
They suspended me for a month
and made me clean the high school
until the end of the school year.
[mop drags]
[mop bucket creaks]
[water splashes]
And that was not fun at all.
That was not fun at all.
[Narrator] Youthful hijinks aside,
it didn't take long for Ezequiel
to start putting his skills to good use.
[Ezequiel] In 2018,
I reported a security vulnerability in Google Cloud.
Suddenly, I get an email.
"Congratulations.
Thank you for reporting this vulnerability to us."
Called my mother.
[phone ringing]
"Hello, by the way,
Google told me that
a vulnerability that I had reported,
they would be rewarding me with $10,000."
Suddenly, she screamed
[Ezequiel’s mom screams through the phone]
[laughs] and I had to put away the phone.
[♪ dramatic music ♪]
[Reporter] Authorities are still deciding whether
to file charges against the hackers.
[Narrator] As long as there's been an internet,
there have been people like Ezequiel,
but there hasn't always been a way
for their skills to be rewarded.
At least not ethically.
Early resistance to the idea of
paying for bugs
drove hackers to the darker corners of the web,
where bug brokers that operate outside the law welcomed them
and their discoveries
with open arms.
It's a problem that still exists today.
[Tim] These days, you can go to websites,
you can look it up and see
what the price of a certain exploit is
and also,
if you're a security researcher,
you can submit that exploit into
what's basically known as the gray market
where you would sell that vulnerability
to a bug broker
who would then go sell it
to unspecified clients,
usually at a much higher price,
and the clients we're talking about here,
nation states
or people with very deep pockets.
[Director] Why would they be buying something like that?
[Tim] Almost certainly to use to exploit users.
[Narrator] In the early days of bug hunter programs,
rewards were given out just a few times a year
from a limited prize pool.
[♪ dramatic music ♪]
To counter the growing appetite of the black market
and to find more of the errors hiding in Google's code,
Tim and Eduardo had to change tactics
and increase the stakes considerably.
[Tim] We thought, "Wouldn't it be cool
if we said we'd pay infinity million dollars for bugs?"
We spoke about it. And we're like, "Well, why not?
Like, would there be a case where we
would not pay for that type of bug?"
No?
Okay.
Then aren't we basically saying
there's infinity million dollars on the table?
[Narrator] Since moving to an unlimited war chest,
bug finds have gone parabolic
with new records being set every year.
But it's not just about financial incentives.
Here's Katie Moussouris, CEO of Luta Security.
She's an expert in what makes bug hunters tick.
[Katie] Having a steady stream
of high-quality security researchers,
that takes a whole bunch more community building,
and that is something that
I think Google really excels in.
They have their own very highly skilled security researchers
interacting with their counterparts
on the outside of Google all the time.
[Narrator] That starts with a handwritten thank you note
or at least a handwritten email.
[Tomasz] The engineer who is taking the bug,
he actually writes a message to himself and says,
"Nice catch!"
[João] "Nice catch!"
[Ezequiel] "Nice catch!"
[Tomasz] They always send you that.
[Callum] The classic Google line
the "nice catch," yeah.
They've got an emoji now. [laughs]
[Yesenia speaks in Spanish]
[João] And sometimes when you receive a,
"Whoa!
Very nice catch!
Oh my God, you are a superhero!!!”
It's nice.
[Tomasz] And I love that because
it's individual thing to you, right?
For the bug.
[Katie] And that sort of
direct engineer to engineer interaction
is one of the most powerful ways to attract outsiders,
way more than money.
[Narrator] The respect Google engineers have
for the hunters is about more than technical prowess.
It's about gratitude
[♪ upbeat music ♪]
because the bugs they find aren't just stamped out.
They're also studied.
[Katie] The best hackers in the world are ones
who continually learn from other hackers.
We are exchanging ideas,
learning new techniques
and expanding upon each other's knowledge.
[Narrator] Bugs reported today will be used
to strengthen the preventative measures of tomorrow,
allowing the code of the future
to start out a little closer to error-free.
[Christoph] Sometimes you see a report
from a bug hunter
where somebody found a really subtle problem
that actually betrays a fairly detailed understanding
of how the application works.
You wonder, "How do they figure this out," right?
And you're like, "Oh, this is pretty cool."
And then quite a few of those bug hunters end up
getting hired [laughs] because they
sort of emerge as somebody
who really has a particular knack for this kind of work.
[Narrator] Yep.
Sometimes the error-finders end up as system designers.
[Ezequiel laughs]
It's a journey Knuth might appreciate.
[♪ dramatic music ♪]
Today, engraved in the entry way of his home
are the words of Danish poet Piet Hein.
"The road to wisdom?
Well, it's plain
and simple to express:
Err
and err
and err again,
but less
and less
and less."
A fine message
for the next generation
of software engineers
and the bug hunters
that will bring their code
a little closer to perfection.
[Director] Eduardo, we are all good.
Thank you so much. [Eduardo] Awesome, yeah.
[Director] This is wonderful.
[Eduardo laughs]
[Eduardo] All right. Goodbye.
Have a nice day. [laughs]
[light switches off]
[♪ anthemic music ♪]
[Tim] Zero-day is a type of vulnerability in a system
that attackers know about
but defenders don't.
[Royal] And so we're gonna dedicate a team
to finding
the hardest-to-find vulnerabilities
and get them fixed before they're abused.
[Parisa] At Project Zero,
we rigorously,
ruthlessly
break the internet.
[Natalie] My motto is “Hack Everything.”
[Tim] The weakest point for Google
might be a non-Google product.
The implant allowed them to pull chat history,
photos,
GPS locations.
[Sen. Fred Thompson] Within 30 minutes,
the seven of you could make the internet unusable
for the entire nation.
[Tim] If the company doesn't fix the bug in 90 days,
we put it all online.