>> You're not going to want to miss this episode of the AI Show.
We talk all about making enterprise GPT real with
Azure Cognitive Search and Azure OpenAI Service
with my friend Liam Cavanagh. Make sure you tune in.
[MUSIC]
>> Hello and welcome to this episode of the AI Show
where we're talking all about making enterprise
GPT real with Azure Cognitive Search and
Azure OpenAI Service with my friend Liam.
Liam it's been awhile since you've been on the show my friend.
>> It has been awhile that hopefully I can make it up
with a couple of cool things to talk about today.
>> Well, anytime there's GPT and stuff and search,
you know, I'm super-excited Liam for those who don't know you,
why don't you just tell us who you are and what you do.
>> Absolutely. Hi everyone my name's Liam Cavanagh.
I am the Group Product Manager,
for Cognative Search and I lead the Product Group building out
search and chart experiences or organizational data.
>> I love it. Because we've
talked about cognitive search and I
even did full demos with skills and
it's the greatest thing ever since sliced
bread but now you're telling me
about GPT stuff and when people hear GPT they're think ChatGPT.
Can you tell us a little about how
you're putting these two things together?
>> Absolutely, in fact,
Seth I haven't demos.
Do you mind if I switch over to
the demo so we can walk through that.
>> Let's go to the screen right here. Here it is.
>> Cool, so in this demo,
what I'm going to show you is basically what
we've heard from customers ever since ChatGPT came out.
What we heard is that they love the experience.
They love the ability to be able to
interact with it in a natural way.
But we heard what people really wanted to be able to do,
at least in organizations,
is being able to do that over their data,
using their own enterprise data to be able to
interact through that and be able to ask natural questions.
>> Interesting.
>> When you ask what search,
that's where we tie this in.
Is that search is such a great place to be able to retrieve
useful information that we can then
use as part of the ChatGPT experience.
>> Can I ask a question though because and this is important
because I want to make sure we're understanding everything.
Are you getting customer data and putting,
unlike redoing the model or is it something different than that?
>> Now, that's a great question because you're absolutely not
taking enterprise or organizational data
and injecting it into a model.
That can actually be in many cases with the amount of
content organization is very costly, very time-consuming.
A pattern that we're using is called
RAG retrieval, augmented, generation.
>> I see.
>> What that means is that we're
retrieving information from the search service.
All that information is stored within
an organization's search service that they own and
it retrieves that and then it augments
the answer using the Azure OpenAI model.
>> I see, well, I'd love to see this in practice.
>> Cool. Why don't I actually walk through
an example in this demonstration will
actually share a link to where you
can actually download the code to reproduce this yourself.
But what I want do is show you
an example where imagine if you will,
you are in an organization
and you want it to be able to take all of
your HR documents such as
this one which talks about the employee handbook.
We have this which is one of
the many health plans for a fictitious company called Contoso.
Imagine you wanted to be able to use that information,
but be able to have a chat experience over it.
I'm going to go back into
this information or into this application.
By the way, we have used Azure Cognitive Search to
ingest all of that content so that it's searchable.
Let me choose one of
these example questions is
such as what's included in my health plan.
What's happening right now is the first stage is it's
going cognitive search and it's trying to retrieve for
that RAG pattern trying to retrieve
potentially useful information that then
injects into Azure OpenAI
as a prompt and tries to answer my question.
As you can see here,
I think it did pretty well.
>> I think that's cool.
>> Thanks. If we see here,
there's a couple of things that I really want to highlight.
First, there was no hallucination.
There is nothing where the model made up and answer.
It was told to only answer if you can find it in that content.
Let me actually even click on one of
these citations where you can actually
go and view and actually validate that it's correct.
That allows us to tie these together Seth.
>> That's cool. I think the cool part about this is
if I understood this right and
I want to make sure that I'm understanding.
You're basically taking the query that the person or
the chat message and you're searching in
Azure Cognitive Search for stuff related to that message and then
you're putting that text into
the prompt that then the large language model,
in this case ChatGPT or gpt-35-turbo,
uses in the prompt to generate
an a grounded answer and not just make stuff up.
>> You got it. In fact,
we can actually see this.
We have in this demo,
we have a developer mode.
You can actually see this.
If I click on the supporting content,
you can see what the search service had brought
back as potentially useful snippets of information,
which we then injected into a prompt,
what we call an Azure OpenAI
prompt that explains to the model what to do.
It also says, I want you to only use these sources of information.
It allows me to do that and in fact I can just continue on and
just say what dental and I can ask that,
to further discuss this.
You'll notice a couple of really cool things
here is that not only did it answer my question,
but I didn't re-ask that about my plan.
It just remembered from the context that I have
a Northwind Health Plus Plan and
it continued that on through that discussion.
>> That's interesting. How does it remember what you said before
because the way I understood these things as you're just
giving it a prompt and it returns texts.
How are you retaining the conversation?
>> That's one of the great parts about the ChatGPT model is
that we can actually continue to keep previous information,
previous discussions into that prompt so that it continues
forward so that the contexts from
before continues into the further discussions.
>> This is cool because it gets into this notion of
prompt engineering but with the RAG pattern where you're
retrieving information to augment
the prompt or you're putting the messages in there because it's
ChatGPT model is very good at conversations and then it's
giving you answers grounded in
the data plus what the person is asking.
>> Exactly.
>> That's cool. Is this hard to set up though?
>> It's really not.
In fact, let me show you one other example and then I'll show
you how this is all backed
then how easy you can deploy all this through some of the code.
Let me just clear
this out and I'm going to ask a different question.
I going to say, can you tell me about performance reviews?
While it's doing that,
I'm actually going to take that question and I'm
going to go in and I'm going to try it in a different way.
I'm going to go into the Bing Translator and I'm going to paste it
in and copy it in Spanish.
Let me take that exact same question and put it in.
I'm going to ask the same question in here.
Now, what you'll see in a second is that this comes back
and what it'll do is it will answer my question in Spanish.
Even though none of the content is in Spanish,
it was able to handle that and so to back to
your question Seth what we've
done is we've provided this GitHub repo,
and I want to show you something.
Within this GitHub repo we have the prompt that backs
this and I'm going to show you
just one sentence that we added to the prompt.
We said, if the question is not in English,
I want you to translate it,
and I want you to then answer it.
You all of a sudden have a multi-language support
with a single sentence that you added to your prompt.
>> Now you're seeing my face.
It's basically like that's menace.
Because one of the things that I've seen is that it's like,
ChatGPT is answering things incorrectly.
Well, of course it was trained on data that was
from before now number 1.
Number 2, there's some data that it never looked at like what's
our Contoso employee handbook
because for those of you that may not know,
Contoso is a fictitious company.
It probably didn't crawl over
documents about Contoso even though it may have.
They've always been asked about how can I make it
answer questions about my stuff.
Now what you're saying is
with RAG, retrieval-augmented generation,
it's specifically with cognitive search
you're able to pull stuff back and
ground the model and
even give it instructions in English to be like,
and by the way, you should translate this and that's bananas.
>> It's really fun to play with it Seth and it's really great.
A fun fact that we had of this is
that all of this content you
mentioned Contoso being a fictitious company.
The health plan, all the
documents purely generated using Azure OpenAI,
we just asked it to create content.
It created over 100 pages for a health plan,
it was amazing, so super fun to work with Seth.
>> Oh my goodness, it feels like as
programmers we're cheating nowadays.
It's like this is got to be cheating.
This is like the fast mode of this but this is really cool.
What's the reaction in or are there
any other details that you'd like to share?
>> This has been a really popular tool for us.
You can hopefully see
how many people have actually taken a look at it.
But I want to go back to your other question about deploying it.
It's literally a matter of downloading this GitHub repo.
It has a lot of information,
but all you really need to do is just do a couple of
simple steps to deploy it.
Just easy to login to your subscription.
You can initialize it and deploy it so in three simple steps,
you can get this up and running and going.
I hope that you,
and some of the viewers,
we'll take a look at this Seth.
>> I don't know what to say.
It's like you've thought of everything.
It's like I'm going to blow your mind number 1,
then I'm going to blow your mind again by
programming in language and then I'm going to blow
your mind by saying just do azd up and you can have this
too in your own subscription.
This is really cool anything
you want people to go read or look at?
>> No, I'd love it if they take look at this,
give it a try. Give us feedback.
Tell us how it's working.
What are the types of things that you would like to see in this?
Have you seen any really interesting things,
so I'm looking forward to hearing what people have to say.
>> Amazing and I was trying to
lob one up because you told me to put
the blog post and a link and there's
a blog post on that for you all to go look at.
>> Thank you that's great linked it.
>> Liam's awesome love working with them.
Also for sure,
minimally, you should go look at the GitHub repo,
browse around the code and see if it's really as easy
as Liam said to set this up in your own subscription.
Any parting words Liam?
>> Now, I'm just super happy to be back and hopefully we will be
back a little bit more often with
some additional cool things coming down the road.
>> I'm going to hold you to that.
Thank you so much and my friend thank you
so much for watching we're learning all about
how you can make your own enterprise GPT real with
Azure Cognitive Search and Azure OpenAI Service.
Thank you so much for
watching and hopefully we'll see you next time. Take care.
[MUSIC]