PRIYANKA VERGADIA: Are you playing around
with a powerful complex image classification model
and training it in Colab or Kaggle Notebooks?
Well, it takes too long to train with the free GPUs.
Stay tuned to learn how you can instantly
scale to train with more data, more GPUs, and a larger scale.
Welcome to AI Adventures, where we explore the art, science,
and tools of machine learning.
I am Priyanka Vergadia.
And today we're talking about scaling machine learning
training resources right from within Colab and Kaggle
Notebooks using tensorflow_cloud.
Let's say you're playing around with a powerful complex image
classification model such as NASNet,
and training it in Colab.
It takes a couple of hours to train with a free GPU
that Colab gives you.
Also, it may not even be feasible to train on large data
sets because the training could go over the free 12 hours
window.
Wouldn't it be nice if I could instantly
scale to training on more data and with more GPUs?
And once you have the model trained,
you often need to run multiple experiments to fine tune
and optimize the hyperparameters to continue
to improve your model.
Hundreds of runs are needed sometimes
to find a setting that results in the best accuracy.
If you participate in a competition on Kaggle,
you know that often, very small accuracy differences separate
winners in the leaderboard.
This kind of experimentation takes a very long time
and a lot of resources.
Wouldn't it be nice if you could run
all these experiments concurrently using
more resources?
Well, that's where tensorflow_cloud comes in.
It's a client-side library for training your TensorFlow model
on Vertex AI.
It provides APIs for a seamless transition from local debugging
to distributed training and hyperparameter
tuning within Google Cloud.
You can directly use it from a Colab
Notebook, or a Kaggle kernel.
It handles Cloud-specific tasks such as creating
virtual machine instances and distribution strategies
for your model automatically.
For distributed tuning jobs, tensorflow_cloud sets up
model callbacks to capture model checkpoints,
and TensorBoard logs automatically.
Now, I'm sure you're excited to see how it works.
So let's take a peek into it.
First, we will run the initial one-time setup
to make sure the Google Cloud assets are all configured.
Link to this Colab is included in the description below.
For that, we set up the project in the Google Cloud console
at cloud.google.com, and grab the project name
and the project number.
In the Colab or Kaggle notebook you're using, plug those in.
Link to this notebook is included in the description
below.
Then make a choice, depending on where
you're running the notebook.
I'm using Colab here, so I will set up auth for Colab Notebook
here.
But if you're using Kaggle, then do the same for Kaggle.
You need a billing account attached to your Google Cloud
project.
So the next step is to link your billing account.
When you entered your credit card information
during Google Cloud sign-up, it created a default billing
account for you.
That's the one we are going to use here.
Next, enable the required Cloud APIs, including the Vertex AI
APIs, which will allow us to kick off the training
jobs within Google Cloud, and the Cloud Build
API to build our Docker images.
We create a storage bucket in this step
to store our temporary assets and to save the model
checkpoints.
Now, if you're going to also perform hyperparameter tuning,
then we need a service account, which
is a special kind of account used
by the tensorflow_cloud tuner to create the hyperparameter
tuning jobs in Vertex AI.
In this step, we are creating that service account
and providing it the right access to create the job.
At this point, we have our set-up complete with a project
in Google Cloud, a service account that will create jobs
in Vertex AI, and the Cloud storage bucket that
will store our model assets.
We can reduce the setup with various notebooks.
In each notebook, you will need to repeat the Step
2 to add the Google Cloud auth credentials to your notebook.
This will allow your notebook to connect to the project.
Now that the setup is out of the way,
let's run some distributed training
and see how tensorflow_cloud helps scale the distributed
training resources in Google Cloud.
This example is based on the image classification
we are fine-tuning with EfficientNet
to train a NASNet mobile model.
First step, import all the requirements.
Then, set the project parameters that we
got from our setup notebook.
And authenticate the notebook to our Google Cloud project.
Then we load and prepare the data,
split it into training and test data sets,
add a preprocessing layers API for image augmentation,
image resizing, rotation, flip and contrast.
Next we load the model and prepare for training.
Here, we load a NASNet mobile pre-trained model with weights
and unfreeze a few layers for fine-tuning the model
to better match the data set.
Then train the model using model.fit.
But notice the remote function.
It determines whether your code is being executed
locally or in the Cloud.
And allows for the separate destination of fit parameters
for local and remote execution.
Now we're ready to train the model remotely on Google Cloud
platform.
We submit a training job using this step
where we define the parameters within the run function.
Our distribution strategy, requirements
for any custom models, Docker configuration,
and master and worker machine configurations.
Once done, we can monitor our job in TensorBoard.
This may take a few minutes, since the logs
start to populate once the training begins.
Now that we know how to kick off a training job using
tensorflow_cloud, execute a distributed hyperparameter
tuning job yourself.
It's fairly similar.
You'll be using the CloudTuner function, which
defines tuning parameters for your hyperparameter tuning jobs
that are executed concurrently on Google Cloud platform.
You also get to define the distribution strategy,
the custom modules in requirements.text,
the Docker image, and the number of concurrent tuning jobs
that you want to run.
If you run into any question, watch my walkthrough
in the video linked below.
All right.
In this episode, we talked about scaling machine
learning training resources using tensorflow_cloud.
Give it a try yourself using the links below
and let me know how it goes in the comments.
If you like this video, give us a like and subscribe.
If you have any questions, reach me on Twitter @pvergadia.