PRIYANKA VERGADIA: Hi, I'm Priyanka Vergadia.
And this is AI Simplified, where we
learn to make our data useful.
We're helping an imaginary company
that is in the process of building their machine learning
workflow.
And to accelerate AI innovation, they
are looking at Vertex AI, which provides
tools for every step of the machine learning workflow,
from managing data sets, to different ways of training
the model, evaluating, deploying,
and making predictions.
In the previous episode, we started
by looking at the first part of any ML project, the data sets.
Vertex AI supports managed data sets for image, tabular, text,
and video data.
In this episode, let's move further in the ML workflow
and talk about building and training the models.
The managed data sets we created earlier feed directly
into the model building and training.
Although you can bring external data sets
while building the model, managed data sets
bring some added benefits as I laid out
in the previous episode.
We have two options to train the model.
We can use AutoML which allows us to create high quality
custom models with minimal effort and machine learning
expertise.
And the second option is to write custom model training
code, where you can run custom training
applications in the cloud using pre-built containers
of your own.
Now when should you write your own model code and then
should you choose AutoML?
It depends.
So let's see.
If your use case fits a supported AutoML offering,
then starting with AutoML is a great choice.
And these use cases include cases where you have images,
or video classification, or object
detection, text classification, entity extraction
from the text, sentiment analysis of text,
tabular regression, and classification.
But if it does not fit any of those categories,
or your model takes some mixed input type, such
as you have images and then the metadata
is in the form of text, then it makes sense
to use custom model.
The other technical factor could be
that you have specific requirements for the underlying
model if you need control over your model's
architecture, framework, or the exported model assets.
For example, maybe your model needs
to be built in TensorFlow or PyTorch.
Then you use custom model.
Otherwise, for the basic use cases, just use AutoML.
The next factor could be how experienced
is your team with ML and AI.
If you have a team with limited machine learning expertise
in building custom models, then exploring AutoML
makes sense before looking into the custom model code
because it is the lowest barrier to entry.
Another deciding factor could be your team size.
Now if you have a small data science team and less machine
learning expertise in that team, it perhaps
makes more sense to work with AutoML because custom model
code would require more time and maintenance.
And use AutoML if you want to develop a quick initial model
to be used as a baseline or a proof of concept,
which could eventually end up being your production model.
But if you are looking to improve on your existing
baseline, or heuristics, then you
might want to write your own custom model
to test out the possibilities.
Hopefully, these factors have clarified
which path to take when it comes to building the models.
Now let's see how to build and train the AutoML model
using Vertex AI.
With that, onto the Google Cloud console.
We learned how to create data sets in the previous episode.
But for the sake of completeness,
let's create a data set.
In this example, we're using credit card fraud
detection data set available on BigQuery
public data sets to create a credit card fraud
detection model.
I have included the link to this data set below if you
would like to follow along.
Once the data is uploaded, we can analyze our data
in the rows and columns.
Right from the data set details page,
we can select Train New Model, define our objective,
which is classification here since we are labeling examples
as either fraudulent or non fraudulent.
This is where you decide whether you
want to use AutoML or custom training.
In this video, we're focusing on AutoML.
So we will select that.
Stay tuned for the next episode, where
I will cover a custom model.
Next, we give our model a name.
Under the Target column, we select Class.
This is an integer indicating whether or not
a particular transaction was fraudulent, zero for non fraud,
and one for fraud.
In the advanced options, you can specify a specific train test
split if you don't want Vertex AI to handle that automatically
for you.
In this case, we're using the random assignment.
But you can also specify this manually
when you upload a CSV of your data.
Or you can have the data split chronologically
if it has a time column.
In the Choose Training Options section,
we will go to the Advanced option.
And since this data is heavily imbalanced, less than 1%
of the data here contains fraudulent transactions,
we will choose the AUC PRC option,
which will maximize precision recall for the less
common class.
In this case, we care more about ensuring our model
can classify fraudulent transactions correctly.
Now the last step is compute and pricing.
Here we enter one as the number of compute hours for our budget
and leave early stopping enabled.
Training your AutoML model for one compute hour
is typically a good start for understanding whether there
is a relationship between the features and labels you've
selected.
From there, you can modify your features
and train for more time to improve model performance.
Then we are ready to start training.
From here, we can leave the UI and will get an email when
our training job completes.
All right.
So our model is ready.
Let's explore the model evaluation metrics.
We head to our Model tab and evaluate
the model we just trained.
There are many evaluation metrics here.
We will focus on two, the confusion matrix, and feature
importance.
A confusion matrix tells us the percentage of examples
from each class in our test set that our model predicted
correctly.
In the case of an imbalanced data
set like the one we're dealing with here,
this is a better measure of our model's performance
than overall accuracy.
Remember that less than 1% of the examples in our data set
were fraudulent transactions.
So if our model accuracy is 99%, there is a good chance
that it's just randomly guessing the non fraudulent class 99%
of the time.
That's why looking at our model's accuracy for each class
is a better metric here.
The confusion matrix shows that our initial model
is able to classify 85% of the fraudulent examples
in our test set correctly.
This is pretty good, especially considering
our significant data set imbalance.
Next, we could try training our model for more compute hours,
or gathering more training data to see if we can improve that
from this 85%.
Now this shows us the features that
provided the biggest signal to our model
when making predictions.
The feature importance is one type
of explainable AI, a field that includes various methods
for getting more insight into how animal model is making
predictions.
The feature importance chart is calculated as an aggregate
by looking at all of our model's predictions on our test set.
It shows us the most important features
across a batch of examples.
This chart would be more exciting
if most of the features in our data set were not obscured.
We might learn, for example, that the type of transaction,
transfer, deposit, et cetera can be the biggest
indicators of fraud.
In a real world scenario, these feature importance values
could be used to help us improve our model
and to have more confidence in its prediction.
We might decide to remove the least important features
next time we train a model, or to combine
two of our more significant features into a feature cross
to see if this improves our model's performance.
We are looking at feature importance across a batch here.
But we can also get feature importance
for individual predictions in Vertex AI.
And we'll see how to do that once we've deployed our model.
Now that we have trained model, the next step
is to create an endpoint in Vertex AI.
A model resource in Vertex AI can have multiple endpoints
associated with it.
And you can split traffic between endpoints.
On our model page, there is Deploy and Test tab.
There we click Deploy To Endpoint.
Give your endpoint a name like fraud version one or two.
And then leave the traffic splitting settings as is.
And then select a machine type for your model deployment.
We used an n1-highcpu-2 here but you
can choose whichever machine type you would like,
including one with GPUs.
Then select Done and click Continue.
Leave the selected location settings as is
and then click Deploy.
Your endpoint will take a few minutes to deploy.
When it is completed, you will see a green checkmark
next to it just like my version one here.
Now we are ready to get predictions
on our deployed model.
There are a few options for getting model predictions,
the UI and the API.
Let's check out both one by one.
On your model page where your endpoint is shown,
scroll down to the Test Your Model section.
Here, Vertex AI has chosen random values
for each of our model's features that we can
use to get a test prediction.
You're welcome to change these values if you like
and then scroll down to the bottom of the page
and select Predict.
In the prediction results section,
you should see your models predicted percentage
for each class.
A confidence score of 0.99 for class zero for example
means that our model thinks this example is 99% non fraudulent.
The UI is a great way to make sure the deployed endpoint is
working as expected.
But chances are that you will want
to get prediction dynamically via a REST API call.
To show you how to get model predictions here,
let's set up a notebook.
For that, we navigate to Notebooks option
and select TensorFlow Without GPUs.
Once created, open the notebook instance.
And then we open Python 3 notebook form the launcher.
In the notebook, we run the command in a cell
to install the Vertex AI SDK.
And below that, we add and run a new cell and the example
from our test set.
Next, we need to create the endpoint object
and then change the endpoint ID and project number
to your specific endpoint.
You can get the endpoint from the URL and the project ID
from the project dashboard.
We use that to make a prediction request to our endpoint.
And we see our prediction around 0.99 for the zero class,
which means the model thinks there
is a 99% chance this transaction is non fraudulent.
With that, we've learned about the two options
to train our models in Vertex AI, AutoML and custom training,
and when to use which one.
Then we went into the console and trained an ML model
from scratch.
In the next episode of AI Simplified,
we will look at building and training a custom model.
In the meantime, let's continue our discussion
in the comments below.
And don't forget to like and subscribe
to catch that next episode.