JOHN BOHANNON: This is Michelle Casbon.
She's an engineer at Google on the Kubeflow team
MICHELLE CASBON: And this is John Bohannon.
He's the director of science at Primer, an AI startup.
JOHN BOHANNON: My team at Primer built a machine learning system
that reads millions of news articles and tells you
what it learns.
MICHELLE CASBON: And I worked with them
to deploy this system of Kubeflow
to make it portable and scalable.
JOHN BOHANNON: The goal is to create a machine
to help humans learn about the world
and overcome their biases.
Humans are biased.
MICHELLE CASBON: Consider this riddle.
JOHN BOHANNON: A boy and his father are in a car crash.
The father is killed instantly.
The boy is rushed to the hospital
and prepped for surgery with one of the best
doctors in the country.
The doctor arrives, sees the boy on the table,
and says, "I can't operate on this patient.
He's my son."
MICHELLE CASBON: Who's the doctor?
JOHN BOHANNON: It's a paradox, right?
Father's dead, so--
MICHELLE CASBON: It's the boy's mother.
JOHN BOHANNON: For those playing at home, don't feel bad.
85% of people are stumped by this question.
It even worked on Michelle when she first heard it.
MICHELLE CASBON: Bias is powerful.
All humans have it, and we all have
to work against it constantly.
JOHN BOHANNON: Every time you meet someone,
you can't help but make assumptions.
MICHELLE CASBON: But you're probably biased.
JOHN BOHANNON: Wait.
That one's actually mine.
MICHELLE CASBON: And that one's mine.
Oh.
JOHN BOHANNON: You're totally biased.
MICHELLE CASBON: Well, any system built on human decisions
will inherit human bias.
JOHN BOHANNON: Consider Wikipedia.
It has grown into the world's most important general purpose
information resource, but there's no one in charge.
Information is added to Wikipedia day
after day by millions of volunteers.
It just boggles the mind that such a thing can exist
and that it actually works.
Wikipedia aims to be the summary of all human knowledge,
but it has biases, just like the humans who create it.
MICHELLE CASBON: If you think about all the new things we
learn about the world each day, keeping Wikipedia up to date
seems impossible.
It's not just that important information is missing
from most articles, but it's that many important articles
haven't been written at all.
JOHN BOHANNON: Which shouldn't be that surprising, right?
MICHELLE CASBON: Well-- so there are about five
million Wikipedia articles.
JOHN BOHANNON: In English alone, yep.
MICHELLE CASBON: And there are some thousands of news articles
published every day.
JOHN BOHANNON: Hundreds of thousands, just in English.
MICHELLE CASBON: So you just have to read all of it.
You find the important information that's in one
set and doesn't exist in the other.
OK.
Yeah, that's hard.
JOHN BOHANNON: Machine learning to the rescue.
I decided to start by focusing on people.
We want a system that just reads all the news, every day,
and learns new information about everyone,
not only people described in Wikipedia
but anyone described in the news.
And if there's enough significant information
about someone in the news who's missing from Wikipedia,
the system should just write you the first draft of the article.
The goal is to make it as easy as
possible for the human editors of Wikipedia to do their job,
and the first step is to shine a really bright light so we
can see where our bias is.
MICHELLE CASBON: For example?
JOHN BOHANNON: Well, for example, only about 15%
of Wikipedia's biographical articles are about women,
and that gender gap is especially bad for scientists.
If you never hear about all the amazing women of science
and engineering, then it reinforces your bias
about those professions.
MICHELLE CASBON: Exactly.
And that's why I wanted to work with Primer on this
in the first place.
JOHN BOHANNON: Right.
And here we are just a few weeks later,
and we're actually catching up on where
we are with deployment.
So where we left off, we had this really cool ML system
with some really classic ML deployment problems.
MICHELLE CASBON: Kubeflow to the rescue.
JOHN BOHANNON: Our first problem was scaling,
so the computing load is all over the map on this one.
On any given day, we're processing hundreds
of thousands of documents-- news articles, scientific papers--
and we're updating information about people in our knowledge
base.
But we also have users who might want
to add thousands of new people all at once,
and that kicks off a back-fill of information
about those people from millions of historical documents.
MICHELLE CASBON: This turned out to be
easy to solve because your team had already set up a Kubernetes
cluster.
Kubeflow is built on Kubernetes is because scalability
is baked into the platform.
The application back-end was built
on top of Postgres, which was a straightforward
migration into Cloud SQL.
Once the data was migrated, I transplanted the full app
into Kubernetes Engine.
I started with a single node cluster
and enabled auto provisioning, which
scales the size of the cluster up and down based
on resource requests.
So now we can deal with unpredictable and bursty
computing loads like yours.
JOHN BOHANNON: OK.
But on top of that scaling problem,
I've also got a complexity problem.
So our data pipeline has a bunch of spaghetti junctions,
and not only do we have thousands
of models to load at inference time,
but we have to update those models
as new information comes in.
So at training, it adds up to millions of model features
getting updated every day.
MICHELLE CASBON: I noticed that.
Well, luckily, this is exactly what Kubeflow is designed for--
composability.
The platform provides patterns for reusing common ML tasks,
like updating models incrementally
in the background.
I started by carving out pieces of the app.
For example, model training was refactored
into a separate service that uses Kubeflow operators.
The monolith is now separated into smaller, distinct pieces.
Now it's a pipeline with three separate steps, train, serve,
and web app.
JOHN BOHANNON: Oh, that sounds much nicer.
OK.
So how's it looking so far?
MICHELLE CASBON: Well, the refactoring immediately
reduced the training runtime, so that translates directly
to cost savings.
And now you don't have to run huge nodes all the time
because GKE takes care of the scaling for you.
Well, the other win is the decrease in code complexity.
That translates to less maintenance overhead,
less time searching for the source of bugs, less time
on ramping new engineers, and adding new features.
JOHN BOHANNON: OK, but that creates a new problem.
MICHELLE CASBON: What haven't I thought of?
JOHN BOHANNON: So now we have all this saved time.
What am I going to do with all my time?
MICHELLE CASBON: You could create some Wikipedia articles
about women of science.
JOHN BOHANNON: Hmm.
Good call.
Oh, and what about the last thing on my wish list, about
vendor lock-in?
We don't want our system tied to one cloud infrastructure.
We want it to be able to deploy anywhere, even on prem.
Oh, and bonus points if the solution
is free and open source.
MICHELLE CASBON: Check and check.
As long as you're running standard, open source
Kubernetes, you can run Kubeflow.
JOHN BOHANNON: Wait, Kubeflow is open source?
What?
MICHELLE CASBON: Yep.
Thanks for listening.
I hope that hearing about this project
lowers the bar for trying it yourself.
The Kubeflow community has some great resources
for getting started, which you can find at Kubeflow.org.
JOHN BOHANNON: And, hey, got a few minutes of free time?
I've already used this machine learning system
to identify thousands of notable women of science
who are missing from Wikipedia.
You can make a difference right now.
Just follow this link and see what the machine
has discovered for you.
Go fill in a gap in the world's knowledge.
[MUSIC PLAYING]