IVAN RAMIREZ: Hey, Fei.
How's it going?
FEI XUE: Hi, Ivan.
IVAN RAMIREZ: Wow.
These micro-kitchens sure are getting bigger.
FEI XUE: I know, so much chocolate.
IVAN RAMIREZ: Anyway, machine learning
has been all the craze lately.
And we're looking for better and faster ways to leverage it.
But we often run into operation overhead,
especially considering all the time and effort
it takes to develop a specific model
and having it run in different places.
FEI XUE: Yeah.
I actually hear this issue a lot.
A common scenario is that a data scientist
would develop his or her model on a laptop,
train that model in the cloud, then serve it in production,
maybe on prem or in a different cloud.
In these different environments, software and hardware
configurations could totally be different.
That makes things break easily when moving from one
environment to the other.
So what if I told you that there would be a way
to easily develop, train, and deploy machine learning
models in a very scalable and flexible way by leveraging
Kubernetes?
IVAN RAMIREZ: Wow, that would be amazing.
Do you think you could show us all of that in five minutes?
FEI XUE: For sure.
I'll be using an open source framework to do just that.
It is called Kubeflow.
It helps you run machine learning models on Kubernetes.
Well, I will show you how to use Kubeflow to first train
a TensorFlow model to recognize handwritten numbers then
to serve that model on an application server.
On top of that, we will deploy a web interface
to interact with the model over public internet.
So Kubeflow actually supports distributive training,
as well as training on GPUs.
But to keep things simple today, I
will demo CPU-only training on a single node.
Let's jump into it.
IVAN RAMIREZ: Let's do it.
[MUSIC PLAYING]
FEI XUE: I have already spun up a Kubernetes cluster.
Kubeflow uses a tool called ksonnet to help
manage Kubernetes development packages.
To get started, first I deploy Kubeflow component
to my Kubernetes cluster.
Some of these components include a Jupiter hub
for model development and TF Job for managing training jobs.
After those are deployed, I can run Kube Control
Get All and see a list of all the resources provisioned
by Kubeflow.
Now that Kubeflow is ready to go,
let's start training in the model.
It's actually really easy.
I have already built a container with existing Python code
and pushed it to the Container Registry.
This Python code trains the model
to recognize handwritten digits then export
the model to a storage bucket.
Now I will deploy the training component.
I set the parameters for the training drop
to tell it where to find the container in the registry.
You can now see in the Workflow page,
I have a container being created to perform the training job.
When it is done, the model will be exported to a bucket.
It is going to take a little while spinning up the container
and doing the actual training.
While this is happening, let's set up a server
to serve the model.
I'm using a TF Serving deployment to do it.
I will point the server to the bucket
where our model will be exported to.
Now, in the Workload page, you can
see that I have a serving deployment called Minstserv.
After that, let's also deploy a web app
UI to interact with the model.
Here I have a simple Flask app that
will connect with the server that we
deployed in the last step.
Again, I have packaged it into a container
and pushed it into the registry.
I use ksonnet to deploy it with an external IP.
Back to the Workflow page, you can see the web
UI deployment is also running.
Now let's check on the status of our training job.
It is still running.
Let's look at the logs.
OK, it is almost done, almost 90% there.
Now we can click on the Refresh button here.
You can see the training is now over,
and the model has been uploaded to the bucket.
With that, we can play with our model.
Let's go to our web app component,
find the external IP, connect to the server.
Here it is.
We can then use that server to test on some random images.
So the first one, we're seeing a picture of a handwritten zero,
and it's giving the correct predictions of zero.
With that, let's test on couple of other random images.
We can see that it's giving the correct result.
So everything we just saw today, from training, to surveying,
to web app UI, is all happening on GKE Kubernetes
with Kubeflow.
IVAN RAMIREZ: Well, Fei, that was amazing.
FEI XUE: Well, thanks.
Well, now you can easily develop, train, and deploy
your machine learning models everywhere on Kubernetes,
thanks to Kubeflow.
I want to give a big shout out to Dan [? Senche, ?]
who developed the demo material.
Please like, comment, and subscribe down below.
IVAN RAMIREZ: And if your company's working on any cool
projects and you'd like to reach out to the team,
go ahead and click on the link down below as well.
[MUSIC PLAYING]