Let's take a look at Hugging Face,
the GitHub of Large Language Models.
Hugging Face is a company and also a
community of open source ML projects,
most famous for NLP.
Its Hub hosts models, datasets, and spaces
for demos and code. These are easy to
download,
under different licenses.
There are also a number of libraries it
provides.
The datasets library provides ways to
download data from the Hub.
Transformers makes it easy to work with
pipelines, tokenizers, models---these core
components of NLP---and to download
pre-trained ones from the Hub.
Evaluate is what it sounds like.
On the right you can see that the
popularity of Transformers has really
been rocketing up in the last couple
of years based on Stack Overflow
questions.
I'd argue that the reason is Hugging
Face offers very simple APIs, but under
the hood they have a ton of power
because they're using deep learning
libraries like PyTorch, TensorFlow and
Jax.
Let's take a look at Transformers first.
What does the pipeline look like here?
We're back to our summarization problem:
article on the left, summary on the right,
and the LLM in the middle.
Using Transformers, you can simply import
the pipeline class, say I want a default
summarization pipeline, and throw my
article at it, and I'm done.
Under the hood it's picking a default
LLM for me, configuring it,
and trying to do the right thing.
In general though, we'd want to configure
it further ourselves, so let's open up
the pipeline. Here are common components
it might have.
That raw input might go through
something called prompt construction.
We'll talk more about prompts a little
later in the module. For now, just know
that some LLMs---not all---require further
instructions beyond the raw user input.
For summarization, it might be as simple
as pre-pending summarize colon to that
article.
That'll then go through a tokenizer
which will encode the text as numbers,
which is what our model the LLM will
expect.
The model outputs an encoded summary,
and the tokenizer (the same tokenizer)
decodes that summary.
I'll also note that this slide glosses
over potential pre- and post-processing
which might happen in the
pipeline, but it gives the key components.
Looking at the tokenizer a little
bit more closely, we can see that in the
bottom left,
it outputs that encoded data as input_ids,
which is the actual encoded text, and
then this attention_mask. We're not
going to talk much about attention in
this course. Suffice to say for now, it is
metadata describing the text which is
output as input_ids, and you need to pass
it along to the model, which expects this
metadata.
On the right you can see us using this
AutoTokenizer class.
This is one of multiple Auto classes
that Transformers offers, which sort of
does the right thing when you give it
the name of a model or tokenizer that
you want to load.
Given that tokenizer, we can pass in the
articles,
and then here's an example of some
configurations you might specify:
max_length, whether or not to pad,
whether or not to truncate the input. These are
all around forcing variable length
input text into fixed-length tensors which
the LLM will expect.
You're going to adjust these based on
the model you pick and also the task.
At the bottom, return_tensors is just
saying we're using PyTorch.
Next comes the model.
We're going to pass in the input_ids and
attention_mask and get encoded output.
On the right you can see us using
another one of these Auto classes,
this time for sequence-to-sequence
language models.
We won't go into a ton of detail about
the different classes of language models
here,
but this is essentially saying a
variable length sequence of text
(like our article)
being transformed into a variable length
sequence of text (like our summary).
Give it the model name. It loads the
right model for us, and then we can pass
in input_ids, attention_mask.
Here I'll note a little more detail
about the metadata.
This is where, you recall with the tokenizer we
were specifying some parameters to handle
variable length inputs. This is the
metadata that will help the model handle
those variable lengths.
Next come a few inference and output
parameters: num_beams, which we'll explore
more when we get to coding,
is saying I want to do beam search to
generate the output text. There are a
number of ways to do inference, and this
is one of them.
And then min and max length are saying:
I want between 5 and 40 tokens in my
output summary.
This is going to be adjusted of course
to your requirements for the task.
That's enough about Transformers,
so let's say a word about datasets.
The library offers one-line APIs for
loading and sharing datasets. We'll talk
about NLP, but there's also audio and
vision.
The load_dataset API not only lets you
specify the dataset name, but also a
version number. That can be valuable for
maintainability of code.
These datasets are hosted on Hugging Face Hub,
and that lets you in the UI
filter by things like task, size, license,
language, and so forth, and also find
related models which can be very useful.
This has been a whirlwind tour of the
Hugging Face libraries and tools.
Next we're going to get into the task of
selecting the right model.