Hi. I'm Sanjana Reddy,
a machine learning engineer at Google's Advanced Solutions Lab.
There's a lot of excitement currently around generative
AI and new advancements, including new vertex AI features
such as Gen AI, Studio Model Garden, Gen AI API.
Our objective in this short session is to give you a solid footing
on some of the underlying concepts that make all the Gen AI magic possible.
Today I'm going to talk about the attention mechanism
that is behind all the transformer models and which is core to the LEM models.
Let's say you want to translate in English sentence
the cat ate the mouse to French.
You could use an encoder decoder.
This is a popular model that is used to translate sentences.
The encoder decoder takes one word at a time
and translates it at each time step.
However, sometimes the words in the source language
do not align with the words in the target language.
Here's an example.
Take the sentence Black cat ate the mouse.
In this example, the first English word is black.
However, in the translation,
the first French word is chat, which means cat in English.
So how can you train a model to focus more on
the word cat instead of the word black?
Add the first time step to improve the translation.
You can add what is called the attention mechanism to the encoder decoder.
Attention mechanism is a technique that allows the neural network
to focus on specific parts of an input sequence.
This is done by assigning weights to different parts of the input sequence
with the most important parts receiving the highest weights.
This is what a traditional RNN based encoder decoder looks like.
The model takes one word at a time as input
updates the hidden state and passes it on to the next time step.
In the end,
only the final hidden state is passed on to the decoder.
The decoder works with the final hidden state
for processing and translates it to the target language.
An attention model differs from the traditional sequence
to sequence model in two ways.
First, the encoder passes a lot more data to the decoder.
So instead of just passing the final hidden state number
three to the decoder, the encoder passes
all the hidden states from each time step.
This gives the decoder more context
beyond just the final hidden state.
The decoder uses all the hidden state information to translate the sentence.
The second change that the attention mechanism brings is adding
an extra step to the attention decoder before producing its output.
Let's take a look at what these steps are
to focus
only on the most relevant parts of the input.
The decoder does the following.
First, it looks at the set of encoder states that it has received.
Each encoder Hidden State is associated
with a certain word in the input sentence.
Second, it gives each hidden state a score.
Third in multiplies each hidden state by its
soft-max score as shown here.
Thus amplifying hidden states with the highest scores
and downsizing hidden states with low scores.
If we connect all of these pieces together,
we're going to see how the Attention Network works.
Before moving on, let's define some of the notations on this slide.
Alpha here represents the attention rate at each time step.
H represents the hidden state of the encoder RNN
at each time step h subscript
B represents the hidden state of the decoder
RNN at each time step. With the attention mechanism
the inversion of the Black Cat translation is clearly visible
in the attention diagram and ate
translates as two words, a mange,
in French. We can see the attention network
staying focused on the word ate for two time steps.
During the attention
step we use the encoder hidden states and the H4 vector
to calculate a context vector a four for this time step.
This is the weighted sum.
We then concatenate H4
and a 4 into one vector.
This concatenated vector is passed through a feedforward neural network.
One train jointly with the model
to predict the next work.
The output of the feedforward neural network
indicates the output word of this time step.
This process continues till the end of sentence token
is generated by the decoder.
This is how you can use an attention mechanism to improve
the performance of a traditional encoder decoder architecture.
Thank you so much for listening.