So Suj, if you look in your LinkedIn profile lately and notice
there are a ton of job openings for prompt engineers.
Absolutely.
And that's why today we're going to do
a deep dive on what that is.
But first, to give a little context,
let's talk about what large language models are used to do
For a review, of course, everyone is familiar with chatbots
and that, we see that all the time.
It's also used for, well, using summaries.
For example, another common use case
or information retrieval.
Those are three different cases.
But for our viewers,can you explain
how that applies in prompt engineering?
Sure, prompt engineering is really vital
in communicating effectively with large language models.
What does it mean?
It is designing, coming up with the proper questions
to get the responses you are looking for
from the large language model.
Because you want to avoid hallucinations, right?
Hallucinations are where you get essentially
false results out of a large language model.
And that's because ...
Because the large language models are predominantly trained on the Internet data.
And there could be conflicting data,
conflicting information and so on.
Great, okay, I got that.
So we're going to look at this from four different approaches.
So let's get straight to it.
Yep.
we're going to look at the first approach, which is RAG, or Retrieval Augmented Generation.
We've had videos about this already on the channel,
so I have kind of a basic understanding of it
where you take domain specific knowledge
and add it to your model.
But how does that actually work behind the scenes?
Can you explain that to me?
Absolutely.
So the large language models, as you know,
are trained on the Internet data.
They are not aware of your domain specific knowledge base content at all.
So when you are querying the large language models,
you want to bring awareness of your knowledge base
to the large language models.
So when you say knowledge base here,
you're referring to something that might be specific to my industry,
specific to my company,
which I'm going to then be applied to the model.
Absolutely.
And so how's that work again?
So to make this, bring this awareness to the large language models,
we have to have two components.
One is the retrieval component,
which brings the context of your domain knowledge base
to the generated part of the larger language model.
And when they work together
and when you ask the questions to the larger language model,
it is now responding to your questions
based on the domain specificity of your content.
Okay, I think I got it now.
This retriever, that can be really as simple as a database search, right?
Exactly, it can be a vector database.
Okay, I got that.
But could you first kind of give me a quick example
of how you've seen that applied in an industry?
Absolutely, let's take the example of a financial information for a company.
Right.
If you were to directly ask a question through the large language model
about the total earnings of the company for a specific year,
it's going to go through its learning and Internet data
and come up with a number that may not be accurate.
Right?
So, for example, the annual earnings,
it could come back with $19.5 billion
and which may be totally incorrect.
Whereas if you want to get the accurate responses,
then you bring the attention to the domain knowledge base
and they ask the same question.
Then the large language model is going to refer to your knowledge base
to bring that answer, and this time it will be accurate,
say, for example, $5.4 billion.
I see, because this is a trusted source
that it can then integrate in with this larger model.
Correct.
Okay, so now we're on the second approach to prompt engineering,
COT, or Chain-of-Thought.
And I sometimes think of this as the old saying,
explain it to me like I'm an 8-year-old,
but could you give me more a practical explanation of what that really means?
Absolutely. I think the large language models, like any 8-year-old,
also need guidance on how to arrive at those responses.
Right, and before I jump to the Chain-of-Thought approach,
I want to recommend something, right?
Any time you are working with the large language models,
consider two things:
The number one is the RAG approach.
Content grounding.
Content ground you work large language models, right?
And then take the approach of prompting it,
guiding the model through the prompts
to get the responses that you need.
And COT belongs in that second category.
As well as these other three, right?
Absolutely.
So let's talk about Chain-of-Thought. Okay. Right.
Chain-of-Thought is all about taking a bigger task
of arriving at a response,
breaking it down into multiple sections,
and then combining the results of all those multiple sections
and coming up with the final answer.
So instead of asking a larger language model,
what is the total earnings of a company in 2022,
which it will give you just a blurb of a number like $5.4 million.
You can actually ask the large language model,
give me the total earnings of a company in 2022 for software,
for hardware, and for consulting.
Say, for instance. I see.
So you're asking me to be more precise with the idea that you'll be able
to get individual results that will ultimately combine.
Combining.
I see. So for example, you cited, we'll just make up some numbers,
if I had 5, 2, and then continue the rest.
And 3, for example.
And the final answer will be 5 + 2 + 3.
That would be the output.
But the larger language model is now arriving at this number
through reasoning and through explainability.
This was .. these were three separate queries essentially,
three separate prompts.
So the way I tell the large language model is
I give the problem
and I explain it on how I will break down the problem.
So, for example, I say,
what is the total earnings of a company?
And if the total earnings of a company for software is 5,
for hardware it is 2,
for consulting it is 3.
Then the total earnings is 5 + 2 + 3.
Let me see if I can net that out to make sure I got it.
So in RAG, we were talking about being able to essentially
improve based on domain knowledge,
but then to improve on the results that that generates.
We then apply this technique,
the "explain it to an 8-year-old" technique,
which then makes the result even better.
Okay, that was Chain-of-Thought,
which is, I understand, is a few shot prompt technique
where you basically provide some examples to improve the end result.
And I think the ReAct is kind of the same genre, but it's a little bit different. Can you explain to me the difference?
Absolutely, so ReAct is also a few shot prompting technique,
but it is different than the chain of thought.
In chain of thought we were going breaking down the steps of arriving at a response.
Right, so you were reasoning through these steps and arriving at the response,
whereas ReAct is goes one step further. It's not only the reasoning with that,
but acting based off of what else is necessary to arrive at the response.
So this data, though, is coming from different sources.
We weren't talking about that in the latter case with chain of thought.
Mm hmm. And they are? So they are, for example,
you have a situation where you have your content,
the domain content in your private data base, knowledge base.
Right, but you are asking a prompt where your question is demanding responses
that are not already available in your knowledge base.
Then the ReAct approach has the ability to actually go into a private
at a public knowledge base
and gather both the information and then arrive at the response.
So the action part of the ReAct is its ability to go to the external resources
to gain additional information, to arrive at your responses.
I got it. I got it.
But there's one thing that confusing me just a teeny bit
is that in RAG, that looks awfully similar.
But they're not the same. Where's the difference here?
So the difference is they both are using the private databases, right?
Knowledge bases.
But in large language models, I want you to think about 2 steps, right?
One is content grounding.
That's what RAG is doing.
It is making your large language model, aware of your domain content
where ReACT is different is has the ability to go to the public resources,
public content and knowledge bases to bring additional information to complete the task.
Okay, before we wrap, can you give me an example of ReAct?
Absolutely, so let's go back to the financial example you were looking at in the previous patterns.
We were looking at the total earnings
of a company for a specific year.
Supposing you come back with a prompt
where you are asking for the total earnings of 2010 and 2022, right?
Your 2022 information is here in your private database ... knowledge base.
But 2010 information is not there, for example.
It's over here in the public one.
Exactly. I got it, okay.
So the large language model in the ReACT approach,
now takes the external ...
takes for the external resources to get that information for 2010
and then brings both of them and does the observation.
I see, so that's going to produce a result that takes into consideration this,
whereas before it might have produced, essentially, a hallucination.
Hallucination. And a couple of more things, right?
The ReAct gives you the result in a 3-step process, right?
When you are asking the prompt in a ReACT mode,
you have to, first of all, split that prompt into three steps.
One is the thought, right?
What are you looking for?
And the second one is action.
What are you getting from where, right?
And the third one, finally, is the observation
that is the summary of the action that is taking place.
So, for example, Part 1 will be, "Retrieve the total earnings for 2022" Right?
And the part ... so Action 1 will be
it will actually go to the knowledge base to retrieve 2022
and Observation will be 2022 value.
Now Part 2 is "retrieve the value for 2010 from an external knowledge base"
now and have that value there.
And Observation 2 will have that value
and then Part 3 will be comparing them
to arrive at which is a better total learning for you.
I think I've got it, that's great.
We only have one more to go
If you really want to impress your colleagues,
you want to learn about this next one,
which is Directional Stimulus Prompting, or DSP.
Different from the other ones, and how so?
DSP is a fun way,
and a brand new one that I want to introduce to the audience,
of making the large language models give specific information,
giving it a direction to give specific information from the task.
So, for example, you ask a question and say, for example,
"What is the annual earnings of a company?"
But then you want don't want the final number,
but you want specific details about annual earnings
for, say, software or for consulting.
So you give a hint and say software and consulting.
And the large language model, first of all, will get the earnings.
And then, from that, extract specific values for software and consulting.
This kind of reminds me of the game
where you're trying to get someone to draw a picture and what do you do?
You provide a hint, and in effect,
this provides you a better result in the same fashion.
Absolutely, so it is a very simple technique,
but it works very, very well when you are looking for specific values from the task.
So try it out!
Well, thanks, Suj. I now understand what DSP is,
but could you kind of net out, how do you combine these different techniques?
You should always start with RAG
to bring focus to your domain content,
but you can also combine COT and ReAct.
You can also combine the RAG and DSP
to get that cumulative effect.
Excellent. Okay. Well, thank you very much.
I hope you come back for another episode in prompt tuning.
Absolutely. Thank you Dan.
Thank you for watching. Before you leave, please click subscribe and like.