>> You're not going to want to miss this episode of the AI show.
We talk all about generating
the right answers with Azure AI search.
What's new since Microsoft Ignite make sure you tune in.
[MUSIC] Hello and welcome to this episode of the AI show.
We're talking all about generating the right answers
with Azure AI search with my friend Farzad.
How are you doing my friend?
>> I'm doing well. I'm really excited for today.
>> Fantastic. Tell us who you are and what you do.
>> Yeah, my name is Farzad Sunavala.
I am a Product Manager on Azure AI Search,
formerly known as Azure Cognitive Search.
>> Oh, so things have changed since Microsoft Ignite.
Can you fill us in on some of those details?
>> Since Microsoft Ignite,
we're pleased to announce that we have a new name for our product.
Formerly known as Azure Cognitive Search,
we are now Azure AI Search.
I know. Fancy, right?
>> Yeah.
>> Another name change that we have is
we previously had a feature in
Azure AI search called Semantic Search and we
are now calling that Semantic Ranker.
>> Yeah, and I like this because if I
remember right when we talked
earlier with Liam a couple of weeks ago,
the semantic search, what it actually did is it took
the keyword search and
the vector search and then it re-ranked those things.
Ranking makes more sense to me. Am I getting this right?
>> Yeah, 100 percent, correct.
We know that vector search alone,
it's good, but it's not the best possible retrieval mode.
We have hybrid search, which is even better,
but hybrid search with semantic ranking on
top to get that best possible precision and ordered ranking.
That's what you want to go for and we're pleased
to announce that in Azure AI search.
>> Amazing. Were there any other announcements
that you could just sort of rattle off to us?
>> Yeah, let's talk about it.
Talk about Azure I Search,
Vector Search, and Semantic Ranker.
We're pleased to announce that at Microsoft Ignite,
we announced the general availability of vector search.
We're ready to support you.
Go use it in your production-ready applications.
We're really excited to
see a bunch of your usage with vector search,
hybrid search, and so on.
Additionally, we had another public preview announcement
at Microsoft Ignite and so we have
a new feature that I'll dive into in a bit called
Integrated Vectorization and so talk about end-to-end ingestion.
I think this is a cool really feature.
I'm excited to show today
to really make the whole rag pattern process.
I'm using Azure AI search, chunking,
vectorization a whole bunch easier.
Then lastly, we also have generally available the semantic ranker.
It's been in preview for a while but now it's GA, go use it.
Get that best possible retrieval in Azure AI search.
>> That's awesome. This is all good stuff.
But oh, and there you go. There's the slide.
Because Liam, he foretold,
he's like you should watch for another announcement regarding
perhaps pricing and this
is the pricing announcement. Can you tell us about that?
>> Yeah, this is the really exciting piece of
the semantic ranker which is now GA. As we know,
hybrid plus semantic re-ranking
is the best possible retrieval mode.
We want all of you to leverage it for
your generative AI applications and so we made
significant changes to our business model and so it's
going to be a lot more economical now than it was in preview.
You get a 3,000 requests per month and then additionally,
it's only $1 per 1,000 requests thereafter.
Big, big changes to the business model.
A lot more economical for you to use.
>> That's awesome.
Yeah, that's amazing.
If you're building like LLM-inspired apps,
you're using Azure AI Search it's very
economical to do
the retrieval augmented generation pattern with this new model.
>> Oh, 100 percent.
I had a black blog post
that Liam I think talked about earlier on the AI show that
actually showed how much better semantic re-ranking
is with hybrid search and so now with this new business model,
you could use it for all your rag scenarios
anytime you interact with
Azure AI Search and grand your data to
an LLM so really excited about this one.
>> Awesome. What's next?
I know there was some feature.
Yeah, this is the thing that I want to ask you about
because Integrated Vectorization sounds interesting,
but can you help us explain what it does?
>> Yeah, so Integrated Vectorization is really just about
end-to-end processing for your generative AI
or really just building the rag pattern.
In Azure search, previously during
our preview of vector search, it was really manual.
A lot of times if you add,
a bunch of long PDFs,
a lot of long documents,
you had to manually chunk it maybe you
adjust the sentence overlap window is a very manual process.
A lot of customers loved using Azure I
Search and we have a capability called
the Indexer and AI enrichment and skills.
A lot of them wanted to say,
hey, this is just going to be amazing.
If we could just have a built-in skill that'll
automatically chunk these documents and then not only that,
but are probably my personal favorite embedding model,
the Azure Open AI Embedding Model.
If we could just automatically send those chunks within
the enrichment process in Azure AI search get
those vector representations of
your data for your chunks and
then just automatically throw them into
your search index fields and then go crazy quoin
and so that's exactly what Integrated Vectorization is it.
>> This is really cool.
>> Yeah.
>> [inaudible] Because I build some samples, you know,
with the prompt flow that uses
Azure AI search all the time but I had to
write my own script and it's not the funniest of code to write.
Just like, okay, what's the overlap window on the paragraphs?
How much text should I put?
You're saying that now there's
a new way to do this all in an integrated way.
>> Yeah, it's all, you just set
it up one time when you're setting
up your indexer and which skills you want to use,
which we have a split skill for
Chunking and Azure opening eye
embedding skill as well as
a custom embedding skill if you want to use
your own custom embedding model and
then once you set that up one time you're done you can
schedule your indexer to grab
your documents from your Azure data source like Blob storage,
SQL Cosmos, whatever it may be and then you're done.
It's hands off and you have your rack solution just work for you.
>> That is impressive.
What else do you have to show us?
>> I do have a demo that I think
is going to be really interesting.
Inside of our Azure search vector samples,
we have a bunch of different code samples here.
Everyone nowadays is a Python guy,
so I'm going to show a Python code sample.
As you can see here, all I'm doing is importing
our Azure search documents library with
our latest pre-release version.
Importing Open AI Azure Storage Blob.
Because I have some documents I want to store in Blob storage.
Over here I'm just going to briefly skim through this part just
a bunch of libraries I need from Azure search documents.
I configured some environment variables here from
my Azure AI search service as well my Blob connection string.
Over here what I'm going to do is
I'm just going to connect to Blob storage.
All I did is inside of
this documents folder, in my Blob container,
I think I put like about five or six PDFs
I actually want to do vector search retrieval over.
>> Question because people are going to ask this,
is this is available in multiple languages, is that right?
What languages are available with this [inaudible]?
>> Yeah, Azure AI search is available in Python,
Dotnet, Java and JavaScript type.
Cool. Because I know everyone's going to be like,
well, can I do this in my language?
Yes, you probably can. I'm sorry, I keep going.
>> You're good. Happy to show.
Yeah, so then once I connected to
my Blob storage inside of this super notebook,
now I want to create a data source inside of Azure AI search.
All I could do here is if you follow this code block,
just use this search indexer,
data container and data source connection method.
I'm passing in my blob connection string
in my container name and then boom,
I get Azure search Integrated Vectorization sample Blob,
which is probably a really long name for a blob data source.
But that's okay. Then once I have that.
>> If I could pause right here. Effectively you're
telling Azure AI search,
Hey, my data lives over here,
and here's a connection to it.
That's persistent in Azure AI search, is that right?
>> Hundred percent.
>> This is much better. Sorry, I'm
going to put myself on the screen here.
Just because I make the index 1 time with the vector,
with my goofy code that I
use for whatever Contoso outdoors company,
that doesn't mean that it's going to continue to be updated.
Is this a situation that makes
it so that this can continue to be updated?
>> Correct.
>> You said it so fast.
I thought you were going to be like yes,
you can upload documents whenever you want to
blob storage and it will pick it up, is that right?
>> 100 percent, that's all right now.
>> Let's keep going.
>> Cool. Now that I connected
my data source Azure AI
search in this case Blob you can do Cosmos,
you can do SQL, ADLS, whatever you want.
Now I'm just going to create my search index.
This part stays standard with creating any Azure AI Search index.
My schema relatively simple parent ID because I want to
keep the parent document at least IDs,
title because I want to maybe display the title of
my PDF actually in my UI,
in my rack solution chunk ID, chunk and vector.
These are probably the real important ones
that is actually going to be
used by a skill set that I'm going to show later.
>> Because I want to make sure sometimes people
lose at least I lost the story in this.
Effectively, this is like a paragraph of
text from your documents your PDF might be 10,000 pages long.
This is just a single chunk,
and the parent ID is referencing.
This is the PDF,
this is the chunk number.
This is the title of the thing, is this right?
>> Yeah, 100 percent that's exactly right.
>> Yeah. One of the things that I tell people about this is
this is super useful specifically
when you want to economize the stuff that you put
into a prompt for retrieval augmented generation,
because if you overspill the prompt,
it's not going to tell you
anything it's just going to be really wrong.
>> Yeah, that's precisely right.
>> Awesome.
>> Cool. Moving forward at this part I'm not going to
spend too much time on because I
think you've seen it a lot of times.
This is the vector search configuration.
Over here.\ I just configured a couple but I'm only going to use
my HMSW vector index on
my vector field property that I configured over here in my schema.
Then I also have this thing called vectorizer.
This is new, this is
another cool feature that's part of the integrated vectorization.
We know that we're going to be doing
Azure Open AI Skills which I'm going to
show in the next code block.
But as part of this vectorizer this
allows me to take advantage of query vectorization.
Now when I pass in text or image whatever it may be,
what it's going to do automatically when I
start showing an example of searching documents,
it already has my Azure OpenAI configuration over here.
I no longer have to use the same function to generate
a query vector then pass it back
to Azure AI Search it'll just handle it for me.
Pretty convenient as well.
>> Nice.
>> Then real quick my semantic can figure over
here because I want to take advantage of that semantic ranker.
That's the only prerequisite for that.
Then this is the cool part,
the new part, so creating a skill set.
An Azure AI Search has a bunch of skills,
what I'm going to do first is use the split skill.
The split skill as it says in
my description here split skill to chunk the documents.
What I want to do is I go ahead and look at
the documentation for what
the split skill contract requires In Azure AI search.
I'm just going to go ahead and really just pay attention to
these two properties since these are the ones
that you probably want to care about.
Text split mode pages,
that means I want to split by page.
But what these two parameters mean over
here maximum page length, 2048.
This unit is in characters of text.
This means that every 2048 characters inside of
the PDF that I haven't blow up split,
I'm considering that a chunk.
Then this other parameter page overlap length like we talked about
earlier That's the sentence overlap in characters unit as well,
so 20 characters before,
20 characters after each chunk.
Then lastly, just doing input and output to output to
the pages field inside
of the cognitive search index through the skill set pipeline.
Then from here now I'm just doing some field mapping.
Now that I have my chunks inside of this pages,
what I want to do is I want to go
inside all of these pages and then
generate and pass these chunks
into my Azure OpenAI embedding skill.
Then I'm returning back the embedding from
the response that I get back from
Azure OpenAI embedding 8002 model.
Then I'm mapping it to the vector field that I
defined in my Azure AI Search index.
>> Can I interrupt just one second.
This is important because first when I saw
2048 I was like is that the number of tokens?
But no, this is the number of characters with
a 20 character overlap and you can totally change that yourself.
The second question I have are
these skills now part of Azure AI Search?
Because I know you can build your own custom skills,
but these are actually part of
Azure AI Search now. Is that correct?
>> Yeah, this split skill actually existed for a long time
we just made some updates to it to
cater for more chunk like scenarios.
>> Makes sense. Another thing that
may be confusing to some people I know it was to me initially,
these skills you define once and then you can reuse
them for an index over time is that right?
>> Yeah, exactly.
>> Keep showing I'm sorry about that.
>> Cool. No worries.
Pretty much almost done, then we can
get to actually querying the index.
This index projections, all this really is,
once the skill set chunks
your documents and generates the vectors.
Some customers like having their chunks
in a separate AI Search index
and some like having maybe with
their parent documents or parent ID's.
That's all this index projections code is really doing.
It's pretty much just getting the output from my previous step in
my skill set process which was
the Azure OpenAI Embedding skill
now just mapping it to the different fields.
This from pages over here is coming over here to my actual chunks.
These are the actual chunks of text.
Then vector over here is the actual vector field,
the 1536 dimensions of floating points.
>> Then over here I'm getting actually from Blob storage,
I want to keep the title so I'm just doing a field map from
my Blob storage metadata field
just so I can display the title as well.
>> This is just like a way of aggregating all of the data that
all the skills generated into put this into this field,
this is into that field. This is cool.
>> Yeah, precisely.
That's it then all I have to do is create my indexer.
This part it's pretty cool because this is where you
can have like a schedule.
You could have your indexer run like on a weekly basis,
on a daily basis.
If you're constantly moving data inside my Blob storage container,
it'll automatically run execute
these skills the chunking
and the vectorization that I just defined.
It's automatically going to be mapped to my AI Search index.
>> This is question about that in terms of economy,
dos it remember things it's already indexed?
>> Yes, it does.
>> That's cool. I was wondering that because
generally if you delete
something it means it doesn't have to do anything.
But if you update something,
it does have to do something or if you add something.
But if you leave something there you don't want to spend
the tokens through eight to two if you're
doing the embeddings and that will take care of it.
>> Yeah, for sure if
you already spend like a bunch of money factorizing things
and it's a duplicate the index is smart
enough to detect that, yeah, you're good to go.
>> I love it.
>> Cool. Once I created my indexer,
I wait I think it was about a minute or
two then it goes ahead does the chunking,
does the vectorization grabs my PDSM blob,
now I can start doing some queries.
This is pretty much the pure vector similarity search,
you can see over here I have a query which is more comprehensive,
North Wind health plus versus North Wind standard.
Over here is my vector essentially my request body,
I have a vectorizable text query.
I'm just going to pass in the pure text.
It's going to my vectorizer that I defined earlier.
I'm just going to say, hey, K = 1 Just give me one chunk back.
Do it over my vector field.
I also have this field exhaustive equals true.
Just because I want to find
like 100 percent recall of the ground truth value.
But if you want you could turn
that to false and just do a ANN search as well.
Here I go, I get my cosine similarity score,
I get my content and it tells me exactly North
Health Plus offers more
comprehensive coverage than North Health Standard,
which some additional information from my chunk. That's it.
>> This is amazing
because just from the search alone I'm getting really good answers
imagine if you put that into a language model to generate
even more customized responses because I've
always told people you stop using LLMs like databases,
use them like language calculators use Azure AI Search as
your database and then put those chunks into your search request.
This is amazing my friend.
Where can people go to find out more about this stuff?
>> Yeah, precisely. You can honestly visit all
of these resources by visiting our block post.
We announced at Microsoft
ignite the general availability of vector search GA,
semantic ranker as well as a hint to
the integrated vectorization feature that I just showed today.
I recommend visiting our block post at aka.ms/AISearchIgnite2023
>> Right here. I put it up on the screen for you, there you go.
By the way, this is absolutely amazing.
It's something that I've been
struggling to know how do I set this up?
Because anytime I do these samples
particularly with retrievable augmented generation,
I'm doing the chunking stuff it's
just like a one time thing for the one sample.
But the reality is this thing needs to be
a living thing and this does it all for you.
>> That's right.
>> Amazing. Thanks so much for being with us my friend.
>> Yeah, no problem I love be here.
>> Awesome. Thank you so much for
watching and learning all about generating
the right answers with Azure AI Search
the new news since Microsoft Ignite.
Thank you so much for watching
hopefully we'll see you next day here.
[MUSIC]