[MUSIC PLAYING]
NIKITA NAMJOSHI: If you've been prototyping with large language
models, you might be wondering if there's
a way you can improve the quality of responses
beyond just handcrafting prompts.
So let's dive in and look at what
it takes to tune a large language model
and how to launch a tuning job from Vertex Generative AI
Studio.
As a quick recap, the prompt is your text input
that you pass to the model.
Your prompt might look like an instruction,
and maybe you add some examples.
Then you send this text to the model
to get it to take on the behavior that you want.
Prompt design allows for fast experimentation
and customization.
And because you're not writing any complicated code,
you don't need to be an ML expert to get started.
But coming up with prompts can be tricky.
Small changes in wording or word order
can impact the model results in ways that
aren't totally predictable.
And you can't really fit all that many examples
into a prompt.
Even when you do discover a good prompt for your use case,
you might notice that the quality of model responses
isn't totally consistent.
One thing we can do to alleviate these issues is tune the model.
So what's tuning, you ask.
Well, one version you might be familiar with is fine tuning.
In this scenario, we take a model
that has been pretrained on a generic data set.
We make a copy of this model.
And then, using those learned weights as a starting point,
we retrain the model on a new, domain-specific data set.
This technique has been pretty effective
for lots of different use cases.
But when we try to fine-tune LLMs,
we run into some challenges.
LLMs are, well, as the name suggests, large,
so updating every weight would take a very long training job.
Compound all of that computation with the hassle and cost of now
having to serve this giant model, and, as a result,
fine-tuning a large language model
might not be the best option for you.
But there's an innovative approach
to tuning called parameter-efficient tuning.
This is a super exciting research area
that aims to reduce the challenges of fine-tuning LLMs
by only training a small subset of parameters.
These parameters might be a subset of the existing model
parameters, or it could be an entirely new set of parameters.
For example, maybe you add on some additional layers
to the model or an extra embedding to the prompt.
Figuring out the optimal methodology
is an active area of research.
But the key benefit here is that you're not
having to retrain the entire model and all
of its many weights.
Parameter-efficient tuning can also
make serving models simpler in comparison to fine-tuning.
Instead of having an entirely new model you need to serve,
you just use the existing base model
and add on the additional tune parameters.
If you want to learn more about parameter-efficient tuning
and some of the different methods,
there's a summary paper linked below for those
who are extra curious.
But if you just want to get to building,
then let's jump into Generative AI Studio
and see how to kick off a tuning job.
From the language section of Vertex Generative AI Studio,
select Tuning.
To create a tuned model, we provide a name, then point
to the local or Cloud Storage location of your training data.
Parameter-efficient tuning is ideally
suited for scenarios where you have modest amounts of training
data-- say hundreds or maybe thousands of training examples.
Your training data should be structured
as a supervised training data set in a text-to-text format.
Each record or row in the data will contain the input text--
in other words, the prompt--
followed by the expected output of the model.
This means that the model can be tuned
for a task that can be modeled as a text-to-text problem.
After specifying the path to your data set,
you can start the tuning job and monitor the status in the Cloud
Console.
When the tuning job completes, you'll
see the tuned model and the Vertex AI model registry.
And you can deploy it to an endpoint for serving
or test it out in Generative AI Studio.
That's an overview of parameter-efficient tuning
and tuning models in Vertex Generative AI Studio.
Check out the links below to learn more
about generative AI on Vertex and learn more
about large language models.
Thanks for watching, and definitely
let us know in the comments what you're
building with generative AI.
[MUSIC PLAYING]