So the first thing I want to talk about
is, why LLMs? Why look at Large Language
Models? We often hear a lot of questions
about LLMs: is the hype around them real?
Is this an iPhone moment for the
industry, or is it sort of a hype moment
where everyone gets excited about a
technology that ultimately doesn't work
that well?
Are LLMs a threat or an opportunity
in different areas of business, in
different areas of software development?
How do you use LLMs to gain a strong
competitive advantage in whatever
industry you're in? And finally, how do
you quickly apply LLMs to your own data?
So we're going to talk about all of
these really quickly, starting with
the first one. And we believe,
a lot of people believe, that LLMs are
quite a bit more than hype. They are
revolutionizing every industry that
involves some form of human-computer
interaction and some kind of
language, data, or interfaces, or in
general working with language.
Here are some examples of
recent stories about the impact of LLMs.
So Chegg is one company that saw its
shares drop significantly because it
found out that users were using ChatGPT
to answer questions instead of using its
service, so it's had an impact. Then
they have to think about
how to adapt their business in
this age of LLMs. Many companies
are building brand-new products using
LLMs. For example, you.com is a startup
that does search but really is focused
on a search assistant that can look at
personalized results for you: look at the
content of those pages and give you high
quality, complex results and
cited sources. It's only really possible
using LLMs, and even existing machine
learning tools like GitHub Copilot are
becoming much more powerful with the
latest generation of LLMs. GitHub
Copilot is this coder system that helps
you auto-complete code essentially, but
now it can also do things like fix your
code by looking at errors, generate tests,
and so on---all new functionalities thanks
to more powerful language models.
At the same time, LLMs are not that new
as a fundamental technology.
They've existed for years,
even in the current form with
deep learning, with
models such as BERT that were out
there.
So why should you care about them now?
And really the story here is
that the quality of these models has
gotten significantly better as we've
gained experience with them. So at the
beginning, people would train
language models, and we'll talk
about this in detail. You could train a
model on a bunch of docs, and then you
have to customize it a bunch to get it
to do a specific use case like for
example
looking at product reviews and
figuring out whether they're positive or
negative, or figuring out what
aspect of the product
they're about. So even though you had
this underlying technology, you had
to do a lot of hand coding, data
collection, machine learning to
actually turn it into to a useful
application. And as we've gotten better
at building and tuning LLMs, we've
managed to make LLMs that you can just
give instructions to in natural language,
that can now do a lot of tasks---useful
tasks with very little human effort to
set them up.
So it's really kind of a
combination of two things that are
enabling all these powerful applications.
Even though the machine learning
part of this has been the same
for a while, the first is that
accuracy and effectiveness have hit
this tipping point where, first of all,
many use cases are now possible thanks
to the accuracy. But also importantly,
it's accessible to everyone thanks to
the ability to instruct them in natural
language. And also increasingly there's
been more and more high quality data and
tooling available. You know, a few years
ago if you wanted to build something
using LLMs, you had to crawl the web and
collect clean datasets and do all
kinds of stuff from scratch. Today you
can find a lot of great open
datasets that have been collected; you
can find entire open source models that
are very high quality in
particular use cases;
and of course sometimes you do want
to do a lot of computation, but you can
get a lot of great commercial services
like GPUs from your cloud provider to
make that happen.
So what is an LLM specifically? Well,
it's a large language model. Large of
course is in the eye of the beholder, but
it's one of these things
where it's increasingly called
large simply because it's larger than
the ones that people tried to use
before, and essentially one of the
main ways that it's large is that it's
also trained on very large amounts of
data.
So what exactly is a language
model though? That's maybe the
more important question.
A language model is a statistical
model that tries to predict
words in text,
in some kind of language. So the way
to set this up, it's
basically a statistical problem: you have
a collection of text and you say, hey
if I show you some words, like say
the beginning of a sentence but not the
end, can you predict what the missing
words are? And this is sort of a
prediction problem. This is the
kind of thing that we've been doing in
machine learning and statistics for a
long time. There are lots of ways to try
to fit a model to the data to predict
these words, but it turns out that
in the past few years, we've
developed ways of doing it that can get
better and better accuracy, especially as
you feed in more data. And this
ability ultimately gives you models
that learn useful things about the world
or can be used to sort of query
interesting information about the world.
So just to give you a sense of the scale
that these operate on,
a typical person might read
a few hundred books in their lifetime,
like say 700 books on average.
That's a lot of books; it takes a long
time. But it's not much compared to what
we feed into a large language model
today. A large language model...
One book is about 80,000 words which when
encoded into the form that a language
model takes is a 110,000 tokens.
A token is sort of a
piece of a word; we'll talk a lot about
them later. So, your typical model
like say ChatGPT is often trained on
trillions of tokens today, which means
tens of millions of books if you do the
math. So that's a lot more books than a
person would ever look at, and if you
have a way of turning these into
a useful statistical model, you often end
up with something that can be
applied to real world tasks. So just as a
very simple example of how you can use
this language modeling ability to
capture knowledge about the world.
Imagine that I wrote the sentence:
avocados are... and then I left a blank
space, and I asked the model: can you
predict which word is more likely in
there? If I have a good
statistical model of language, I can use
it to learn some things about avocados.
For example, if I wasn't sure what color
avocados are, I can try putting in a
bunch of different colors and asking
how likely is it
that it's that answer, and the model will
say it's much more likely that the word
after blank is green. Avocados are green,
versus avocados are blue. So in some
sense, the model learned something about
the color of avocados, even though it
doesn't really know what an
avocado is. It just knows about the
probabilities of words.
Likewise you can figure out
avocados are delicious, avocados
are probably not intelligent,
you know all kinds of things like
that. And if you take this ability to
the extreme, use it in more
complicated ways, you can get these
models that will do things like
generate text with specific
properties, like generate a song about
sunset in Santa Monica, or classify data.
You can ask it, is this a good
or bad review, and it will do that, and
these are all based on sort of the
knowledge about the world and about
reasoning that it gleaned from seeing a
large amount of text.
So what does that mean for you as a
developer and user?
Well, LLMs can automate many tasks
that previously required a person to do
them in detail, that usually involve
imprecise
language or knowledge about the world,
that's kind of hard to codify and
put into a computer. And this can help
both accelerate innovation and build new
features, new interfaces to
the software, and just increase ROI and
efficiency of a business. So just some of
the examples of things you can do with
it. You can speed up software development
because a lot of software is
code; it's in a language, and you can
get these assistants that are
good at specific parts of it.
You can democratize AI itself. Many
users can now use LLMs by just talking
to them and asking them to do things,
instead of doing a heavy duty machine
learning project. You can open up these
rich use cases like assistants,
analyzing documents for business,
all kinds of things like
that,
generating ad copy and stuff like
that. And of course you can reduce
development costs of applications, and
you can reduce monotonous tasks that
people have to do: get the easy stuff to
a model, and get people to focus on the
hard stuff. So we're just beginning to
scratch the use cases. You'll
see a bunch more throughout the course.
Now unfortunately with LLMs, there's no
perfect or magic bullet model. So once
you get past the idea
of, what could I use this for,
and try to actually use it in an
application, the right model will
depend a lot on the requirements for
your application. And there'll be a lot
of trade-offs required and potentially
quite a bit of custom development, so
here are some of the factors you want to
take into account when thinking about an
application using LLMs. The first one is
the model's quality. What
kind of quality can I get from it? Can I
find a model that I can use out there
that's high quality, or do I have
a way to improve it myself? Or do I need
to sort of wrap the LLM around in a
bigger application that uses this
unreliable component to do something
reliable at the end? So that's an
important piece. The second one is the
serving cost. This depends a lot on what
you're trying to do.
This is the cost of just running the
model and getting a prediction. For
example if your application is
reading a couple of documents per day
and extracting some information from
them, it's okay if it costs a
dollar, ten dollars, whatever for the
LLM to process one document. It's
totally fine. If your application is
placing ads on a web page that
are customized to each user, or as they
click through the site for your store
selecting recommendations for
them,
you can't afford to spend
many dollars each time someone looks at
a page. You need to serve
millions, maybe hundreds of millions,
of instances of this per day
at a moderate cost. So you'll really
optimize for cost. Serving latency is
another important factor. Again if you're
doing some offline analysis, it's okay if
it takes minutes for your application
to run. But if you're doing something in an
interactive web page, it better
run
in a few milliseconds to get
a high-quality application.
And then the final one that can
be tricky, and is important to think
about, is customizability. If you have an
important application, you're going to
want to keep improving its quality over
time, debugging it when something goes
wrong, and generally sort of making it
better and customizing it. So you need to
think about, in different solutions, what
knobs do I have to control
how well this is doing,
to monitor it, to make it better,
to prevent certain kinds of bad behavior
that I don't want it to have,
and so on. So this is an important
thing to think about as you design your
LLM application.
So who is this course for?
In this course,
we wanted to really
make it useful for practitioners and
bridge the gap between two things that
you can find out there today in other
courses. So one of them is just black-box
solutions, like a third party
LLM-as-a-service API that you can call,
where the tutorial is really
easy: it just says okay just put in your
text here, and call this
function, and you'll get a result back.
But it's really hard to then take this
and
customize it, put it into a bigger
application, and make sure you can get
the best quality, the best
performance, and so on because it's
basically one model for everyone.
So that's on the one hand. And then on
the other hand, there's the first
principles of how LLMs work, how machine
learning works, how to think about
building a pipeline that's reliable, that
you often find in academic courses
on theory and algorithms, and so on.
So we wanted to build this
practical course that's in between, so
that when your boss asks you,
can you do something with LLMs in my
application, and you really care
about making it work well and getting it
right and so on, you have the tools
to build your own
application using existing components
out there, and refine it and make it
really great over time. And we hope
this gives you a sense of both some
of the research and academic foundations
of this space, but also some of the
engineering things you can do to
reliably create high-quality
applications to the best of our
knowledge of what we know how to do
today.
So that's a little overview of the
course. We hope you enjoy it. We had a lot
of fun putting it together, and we're
very excited to see what you build
with LLMs!