SPEAKER: So this demo is about showing
how Kubeflow and Kubernetics together
can make machine learning really easy, scalable, and portable.
So this demo is about the operational efficiency
that you gain by adopting Kubernetics and Kubeflow.
So the sample use case we have taken here
is that of predicting sentiments associated with reviews.
And so here's a very simple dev app, right?
Let's go over some of the reviews that are sample
reviews that are listed here.
First one says, "It's a typical McDonald's.
Food was typical.
Employees were typical."
Everything is typical.
So there's really nothing positive about this review.
And so we would expect it to be a negative review.
You don't want to highlight it on your restaurant website
as one of the premier reviews.
And the second one here, it says it's good.
"The service is good.
Overall, they do a good job."
So this seems to be a pretty positive review,
so we should probably be highlighting that.
Whereas on the right side here, it
says, "Really excited to hear of this restaurant coming
to Toronto," and so on and so on.
"The service here is not great."
So it's not a review that you want to highlight.
So it's probably a negative review.
So what would you expect--
let's say you throw this task at your engineer,
and then your engineer just builds
something, some back-end system, to classify
them as positive or negative.
Let's see what the engineers built.
Turns out, what the engineer has built doesn't
meet what the humans expect.
It's literally classifying them incorrectly.
Let's go look at the code that the engineer has used.
This is not good code.
It's really dumb code.
I don't know how that got checked in.
But in any case, it's there.
So now let's go improve it and let's go fix it.
So this is where machine learning comes to the rescue.
There's quite a few machine learning models
that you can use to solve these sort of language problems.
And Tensor2Tensor happens to be one really popular model that
can be used for lots of different applications
like language modeling, sentiment analysis,
summarization, and so on.
And so sentiment analysis is one application for this model.
So what we're going to do now is,
we're going to use Kubeflow.
We're going to use this Tensor2Tensor preexisting model
and we're going to operationalize that.
We're going to train that model using public data that's
provided by year.
And then what we're going to do is,
we're going to deploy that model to a Kubernetics cluster
and then see if I can improve this web
app to become smarter and make the right kind of predictions.
So at this point, I'm going to switch to my terminal
here where I'm going to run the demo.
So the first stage of the demo is that I'm
going to switch to Minikube.
Then what I'm going to do is, I'm
going to deploy Kubeflow to my local Minikube cluster.
Kubeflow prepackaged with a few components.
I'm going to show you what components exist.
And you're also going to see that Kubeflow extracts
out a lot of different Kubernetics constructs
that you have to put together in order to build applications
that data scientists and ML practitioners can consume.
They find it really hard to consume Kubernetics directly.
So Kubeflow acts as the glue between the apps that data
scientists like and the infrastructure efficiency
that Kubernetics provides you.
So I'm switching to my local Minikube context here.
The next thing that I'm going to do
is, I'm going to deploy Kubeflow locally.
So it's as simple as running one simple command.
Then the next thing is, I'm going
to show Jupyter Notebooks.
So this is part of Kubeflow, that you have Jupyter Notebooks
that are built in.
So you can go in and log in and create your own Notebooks.
There are many pre-canned images.
And you can choose different CPU.
You can choose different memory.
And then you can also choose GPUs if you happen to have it.
But Minikube doesn't support it yet,
so you probably can't use GPUs locally.
But you get a pretty good set of configurations,
and you can bring your own images.
So what this provides is, it provides you a local Notebook
environment through which you can iterate locally and build
models interactively, locally.
But we're not going to use Notebooks now.
Instead, what we're going to do is, we're going to go back
and we're going to continue training
our model using command line.
And specifically what we're going to do is,
we're going to use a Tf job operator that's
packaged as part of Kubeflow.
The Tf job operator lets you do distributed training
and single-load training in a really easy, portable way.
So similar to how you define a deployment or a job
in Kubernetics, you can define a TensorFlow job using the Tf job
operator.
And then the TensorFlow job, what
it does is takes your TensorFlow application
package as containers and sets up
the runtime environment that TensorFlow needs in order
to make it work in a distributed setting and in a local setting,
and extracts all of that.
So I'm going to configure my TensorFlow
model to run locally.
I've set the data to be available as locally,
and I've set local training to be true.
And the model is now deployed.
I'm going to check if the training jobs are running.
They seem to be running.
Then the next thing is I'm going to check
if the model is actually running by looking at the logs.
OK, looks like the model is running.
But I'm going to stop here with local training,
because it's going to take a long time.
It's probably going to take a couple of days
for local training to complete.
But the point of training locally
is not to build a model, but it's rather to just make sure
that my model works.
As soon as I know my model works,
I have the cloud to the rescue.
Now I can switch to the cloud and use elastic compute data.
And that's what we're going to do next.
And you're going to use the same Kubeflow framework to do that.
So I'm going to switch to a GK cluster.
And I'm going to show you the GK cluster
that I have pre-provisioned.
So here's a GK cluster.
This cluster has GPUs and TPUs already attached.
So I pre-provisioned that.
It's very easy to provision GK clusters with TPUs and GPUs.
It's literally running two commands.
And you get late drivers and everything pre-installed.
So I have GPUs attached here.
And TPUs are available from all nodes.
And I also pre-provisioned a TPU here.
And the TPU is running TensorFlow version 1.7.
So the next thing that I'm going to do
is, I'm going to go back to my terminal.
I'm going to switch context and switch to this GK cluster.
And I'm going to deploy Kubeflow to GKE.
And once I've deployed Kubeflow, then I'm
going to deploy my model to GKE.
So here we're deploying Kubeflow to GKE.
The next step is, let me deploy the model.
So I have to change the model parameters such
that it can run in the cloud.
I've enabled GPUs for the cloud, and also enabled
a set of data that are created to be on GCS.
And I have also disabled local training.
So once the model is deployed--
next off, in addition to GPUs, you
can also use TPUs in Google Cloud.
And now I'm going to deploy the same model,
but this time with TPUs.
So here you go.
I've configured TPUs, a few parameters to make the model
point at a TPU.
And then the models are now deployed on TPUs as well.
Now we're going to wait for the pods to be running.
So I'm going to switch to the GK UI
and I'm going to filter on my demo namespace.
And once I've filtered that, I'm going
to check whether my training pods are running.
So turns out, my training pods are running.
You see here that you have a distributor cluster running.
So you have two workers and a master and a parameter server.
These are TensorFlow concepts.
Don't worry about it if you don't know it.
And there's also a TPU training job that's running.
So if I go into the TPU training job--
if we go into the TPU training job,
then you would know that the TPU training job is running.
And then there's the GPU job here.
And then the GPU job is running.
And you can see here the GPU memory and GPU unitcycle are
also being shown here for you.
And the pod is running, and you can go in here
and you can see the logs.
The training is actually going on already,
and it's saving checkpoints and so on.
So once we go back, then--
OK, so training is going on, but training
is going to take quite a while.
What can we do meanwhile?
So in addition to that, we also get Tensible.
I forgot to mention this.
You get Tensible.
That's going to help you track the progress of your training
both locally and in the cloud.
What I've done here is, I'm showing you Tensible graphs
based on pre-trained models that I ran through training.
And this is what you would see.
So what you see here is, for GPUs, you
see a training rate of about three, roughly.
Like 2.5 steps per second.
And then in the case of TPUs, if you scroll down,
you see a much higher learning rate.
And so this is to give you a rough idea of what
sort of hardware you might use.
And in any case, Kubernetics and Kubeflow
can work with both of them.
So now that we have gone through the training process
and now that training is done, the next office--
we can't wait for training to complete.
And so what we're going to do now
is we're going to take a pre-trained model
and deploy it to the same GK cluster.
And then we're going to see if our restaurant sentiment
analysis UI can perform better.
So I'm going to go ahead and deploy a pre-trained model
locally--
I said, sorry, into my GK cluster.
And once that model is deployed, now I
can go back and see if the model is running.
So I'm going to refresh.
And my model is running.
The pods are running.
So let's switch back to my UI.
So I have the UI here.
But local classification, it did not
do well, as we would expected.
So now, let's see if it does well with the machine learning
model.
Yay, it does well.
It's classifying the first and the third review correctly
as being negative, and it's classifying the second review
as being positive.
So this demo should have shown you
how the overall pipeline for machine learning
can be simplified with Kubeflow and Kubernetics
together, where your infrastructure concerns
and application packaging concerns are
taken care of for you.
And you focus on building great machine learning packages that
are reusable and making a plain machine learning really, really
easy for the rest of the world.