(upbeat music)
Hello, everyone.
Welcome to Build End-to-End Solutions with Vertex AI.
My name is Sara Robinson. I’m a Developer Advocate
on the Google Cloud Team focused on machine learning.
And you can find me on Twitter @SRobTweets.
I’m going to be talking about Vertex AI, which is new.
It was just announced earlier today.
So I’ll start by giving you an overview of what Vertex AI is.
I’ll talk about how you can use Vertex AI tools
to streamline your machine learning model development process.
We'll look at productionizing ML workflows with pipelines.
Finally, we’ll end on tools to use to monitor your deployed machine learning models.
So to start, what is Vertex?
Vertex is a managed machine learning platform
that can help practitioners accelerate their experiments
and deploy machine learning models.
But how does this fit into a typical application development workflow?
Vertex AI sits at your application backend, giving you all the tools you need
to build, train, and deploy machine learning models.
Your application logic will talk to these deployed models,
getting predictions and surfacing those predictions to end users.
Those end users could be in an offline/on-device use case,
or they could be desktop or mobile clients.
And what's all included in Vertex?
These are the products that are part of Vertex AI.
As mentioned, they can take you through your entire machine learning workflow,
from gathering data, to deploying and monitoring your models.
I’ll talk about a few of these products in this talk today.
I’ll highlight tools focusing on these parts of a machine learning process.
We’ll look at tools to help you gather your data, train your model,
then we’ll go to how to deploy models to endpoints.
We’ll look at streamlining workflows with pipelines.
Finally, we’ll look at some monitoring tools.
Just a quick note that Vertex AI supports
both UI based and code-based model development.
While I'll be focusing mostly on the UI in this session,
note that all of the functionality I’ll be covering
has equivalent SDK methods,
so you can implement everything I’m showing programmatically.
Now, we're going to look at how you can use Vertex AI
to streamline your model development.
When I work with customers and external users the first question I’ll typically ask is
do you have your own training data?
If the answer is no, we have a set of pre-trained APIs
that give you access to machine learning models
for common tasks like analyzing images and analyzing text.
All you need to do is send a simple REST API request.
For the duration of this presentation, I’ll focus on the right side of this flow chart.
I’ll assume you do have custom data available
and you want to use that custom data to train a machine learning model
custom to your prediction task.
So if that's the case, the next question to ask
is are you writing this model code yourself?
And whether the answer is yes or no,
we have tools to help you train your models on Vertex AI.
Let's take a look.
So how does model development work in Vertex AI?
I've split this into three steps:
gathering your data, then choosing the training method to use,
and finally, deploying your model,
whether you’re deploying it offline on device to an endpoint for online serving,
or running batch predictions.
Let's start with gathering data.
And for this, we have a product called managed datasets.
With managed datasets, you upload your data.
First, you choose the type of data to work with.
From there, you’ll choose the prediction task that you’ll be solving.
If your data doesn’t fit into any of these use cases, don’t worry at all.
You can still train and deploy your models with Vertex AI.
Once you’ve got your data ready, the next step is to choose your training method.
One option for this is AutoML.
AutoML lets you train custom machine learning models
without requiring you to write any machine learning code of your own.
Vertex AI will find the optimal model architecture for your prediction task,
and giving you access to state-of-the-art machine learning models.
Once your model is trained,
you get access to detailed model evaluation metrics,
and you also get feature attributions
which are powered by Google Cloud’s Explainable AI.
Feature attributions tell you which features in your model
signaled your model’s predictions the most.
This helps you get a bit more insight
into how your model is performing under the hood.
The other option for training your models is to use Custom Training.
With Custom Training, you can train models using a framework of your choice.
You can either use one of the prebuilt containers we have available,
or package your model training code as a Docker container
and use the custom container option for training.
With either of these,
Custom Training lets you run distributed training jobs
and also gives you access
to accelerate your training with GPUs. Using Custom Training,
you can also take advantage of hyperparameter tuning
to run multiple trials of your training jobs in parallel
to find the optimal hyperparameters for your task.
Once you’ve got a trained model, you’ll likely want to deploy it to an endpoint.
Whether you’ve trained your model with AutoML or with your own custom model code,
all endpoints in Vertex AI show up in the same place in your console
and are accessible via the same API.
You can split your traffic between models once you've deployed them to an endpoint.
This is really useful if you want to do something like A/B testing,
or maybe you want to roll out a new model update incrementally
to some of your users.
You can also customize the machine type where your model is deployed.
So I've talked about how you can go from raw data to deployed model.
But in a real-world scenario, there will be many teams
solving a lot of different prediction tasks across your organization,
and we need a way to manage this process. For that, we'll use the practice of MLOps.
So what is MLOps?
MLOps is a practice that defines processes
to make your machine learning development more reliable and productive.
It can help you standardize your machine learning systems,
so that multiple teams and people across your organization
can reproduce those systems rapidly and reliably.
Vertex AI has a variety of tools to help with MLOps.
They’re highlighted here in blue.
I’m going to go into a few of them in this presentation.
Before I do that, let’s look quickly at how MLOps
fits into a typical application development workflow.
So once you have a deployed model,
you have users generating new predictions on that model.
You want to make sure
that your model is still performing as you’d expect.
Then you may want to feed those new predictions into your model,
train a new version of your model, redeploy it and start this process again.
The first MLOps product I'll introduce is Vertex Feature Store.
Vertex Feature Store is a centralized repository of feature data,
which means your feature engineering work can be shared across teams.
Let’s look at what problem this solves.
To do this, let’s imagine you have one team in your organization
building a model to predict whether or not somebody will return an item
that they purchased. Now, let's say one of the features
that is input to this model is a windowed aggregation of purchase data.
Imagine that there’s another team in your organization
that's solving an entirely different task,
but they’re also computing this windowed aggregation.
If these two teams don’t talk to each other,
and there’s no centralized repository for your feature data,
they’ll wasting time and effort by computing the same feature twice.
This is one of the problems that’s solved by Feature Store.
Feature Store also lets you serve your machine learning features
at scale with low latency. And finally, it alleviates the problem
of training-serving skew,
which happens when the way that you compute features at training
differs from how you’re doing it at serving time.
With Vertex Feature Store, you only compute your feature values once
and then you reuse them for training and serving.
Feature Store also lets you track and monitor
for drift and other quality issues. Next, we’ll look at how you can productionize
your machine learning workflows with pipelines.
Before diving in, a quick note that a deployed machine learning model
is only the beginning of your ML process.
Once your model's deployed,
you want to make sure to avoid things like concept or model drift,
which can happen when the environment around your model changes.
And pipelines can help automate this retraining workflow.
We believe that pipelines are the backbone of production machine learning systems.
And by pipelines, we mean your end-to-end machine learning workflow,
broken up into distinct steps for each task along the way,
and these steps are also referred to as components.
To build pipelines, we’re introducing Vertex Pipelines,
which is new today.
Vertex Pipelines is a managed service that lets you build pipelines
with familiar, open-source pipeline frameworks,
like TensorFlow Extended, TFX, or Kubeflow Pipelines.
It’s automated, scalable, and cost effective, so you pay only for what you use.
What we’re looking at here is a pipeline graph,
And in the Vertex Pipelines console,
you can inspect each individual step of your pipeline.
Each step of the pipeline
will generate an associated artifact that’s fed into the next step.
All of these artifacts are stored in a central storage location,
and right in the console, you can click into individual steps,
see the associated artifacts generated from that step,
and you can also track these artifacts that were generated
across separate pipeline runs.
With Vertex Pipelines, you can also add conditional logic to your pipeline.
So let's say that you only want to redeploy a model
if that model’s accuracy is above a certain threshold.
So you can add logic like that,
or whatever type of logic you’d like, to your pipeline.
You can also store metadata
for every artifact that’s produced by the pipeline.
And as I mentioned before, you can track artifacts
across your machine learning workflows.
Finally, we’ll look at how you can use Vertex AI
to monitor your deployed models.
First, an important distinction between MLOps and DevOps.
So unlike with traditional programming,
your model may not be performing as you would expect it to,
even if that prediction code still works.
The environment around your model may have changed
in such a way that it’s not performing predictions correctly.
So this is known as drift,
and we need a way to continuously monitor our models
to catch when this happens.
For that, we're introducing Vertex Model Monitoring,
which is new today.
With Model Monitoring,
you can automatically alert your data scientists and ML engineers
when model performance changes.
To do this, you create a monitoring job,
and within that job, you define an objective.
Once your job has kicked off,
you’ll be alerted whenever performance goes below a certain threshold.
You can use Model Monitoring
to detect things like drift and training-serving skew.
Rather than manually checking to make sure your model’s still performing correctly,
this provides confidence in model reliability
because you'll be alerted anytime something changes.
I covered lots of different products in this presentation, so just to recap,
I showed how you can use Vertex AI throughout your machine learning workflow.
We started by looking at how you can use Vertex
to gather data, build, and train your models.
Then we looked at how you can scalably deploy your models to endpoints.
We talked about how you can automate your machine learning workflows with pipelines.
And finally, we looked at tools for monitoring your machine learning models
to make sure they’re still performing as expected,
and retraining them if they are not. Thanks for watching.
I hope you enjoy the rest of the sessions at Google I/O.
And to get started with Vertex AI and dive right into the Vertex console,
head over to cloud.google.com/vertex-ai. Finally, we have a couple of events coming up
where you can learn more and get hands on with Vertex AI.
The first is at the Google Cloud Data Summit on May 26th.
The second is at the Google Cloud ML Practitioners’ Summit on June 10th.
To register, head over to the link on the slide.
Thanks for watching.
(upbeat music)