STEPHANIE WONG: Training a machine learning model
involves testing a variety of parameters
to figure out which configuration works best.
Before you even start training a model,
you'll need to set up some hyperparameters,
like the number of epochs you want to train for
or the type of optimization function you want to use.
But it can be tricky to figure out which values will give you
the results you want.
Let's take a look at what hyperparameters are
and how Katib can help you tune them.
[MUSIC PLAYING]
Today we'll talk about hyperparameters
and how to use Katib for hyperparameter tuning.
If you want to check out the documentation directly,
check out the link below to read more about Katib
and how to use it.
Unlike the training parameters that are tested and adjusted
during the training phase, hyperparameters
aren't learned by the model.
They're provided before you begin training and can
make a big impact on the results of your model.
You could try adjusting them yourself,
changing hyperparameters, and noting the results.
But when you have many different configurations
you want to explore and training your model
takes a significant amount of time,
it's not feasible to do it yourself.
That's where Katib comes in to help.
Katib helps tune your hyperparameters
by testing a range of values for each one.
You need to define what your objective is, and then
set up an experiment so Katib can
try different hyperparameter values.
For example, if you are training a model with two
hyperparameters-- learning rate and batch size--
and wanted to maximize the accuracy,
Katib could run the experiment to try different values
and see how they affect your objective.
You could also try different objectives,
like minimizing error rate.
Let's walk through actually running Katib.
You can try this out for yourself
if you have a Kubeflow instance running.
Just head to the dashboard, choose your namespace,
and then click Katib on the left side, followed
by Hyperparameter Tuning.
This should give you a big text box where
you can paste a YAML file.
We'll include a link in the description
for the one we're using here.
So go ahead and copy-paste the file here, and then hit Deploy.
This can take a while to run, so let's take a look
at some of the configuration we're specifying.
Here we're defining some key values with our trial accounts.
Each trial is an iteration through the experiment with one
set of hyperparameter values.
This whole file will create one experiment with 12 trials,
as specified in the maxTrialCount.
In addition, Kubeflow will run three trials in parallel,
thanks to the parallelTrialCount.
And maxFailedTrialCount will shut down the experiment
if three of the trials result in failure.
In addition, we're also specifying the objective
we're looking for.
In this case, we want to maximize the accuracy_1 metric.
A goal of 0.99 is also specified,
so the experiment will automatically
stop if we hit 99% accuracy.
Lastly, we can specify which algorithm Katib
will use to tune the hyperparameter values.
Multiple algorithms are supported,
such as grid search or Bayesian optimization,
but we'll keep things simple with the random algorithm.
In this case, Katib will randomly
try different values for our hyperparameters
and measure our objective metric.
Let's take a look at the hyperparameters
in the configuration file.
These are the actual hyperparameters
we want to test.
In this case, we're looking at testing the learning rate
and batch size for our training step.
Each hyperparameter has a type, along with a feasible
range of values with minimums and maximums.
Since we specified random as our algorithm,
Katib will choose random values for these two
inside the feasibleSpace and then run the training job
as a single trial.
Once we've hit the maxTrialCount,
our experiment should be finished.
We can check the status by selecting the Menu
icon next to Katib and choosing HP, then Monitor.
You'll see a list of any experiment
and their current status.
A check mark means the experiment has completed
successfully, and we can click on it to see more information.
This graph can look a little bit complicated,
but it's actually just showing you each trial
over the entire experiment.
The left column is a result of our objective metric,
the middle column is the learning rate,
and the right column is our batch size.
If we look at the line that had the highest accuracy,
we can see the values of our hyperparameters
that lead to this run.
It can be a bit hard to see the exact values here,
so if you scroll down, you can see a table
with all the trials.
Looks like the best learning rate was around 0.0111
and the batch size was 170, giving us
an accuracy of just over 97%.
In this case, we only used 12 trials,
but we can easily change the YAML file
and try another experiment with more trials
or additional hyperparameters.
Katib can help save you a lot of time and effort
in narrowing down the best values
for your hyperparameters.
And it's easy to set up an experiment.
Try it out for yourself, and check out the documentation
and read more.
[MUSIC PLAYING]