Up next on this special edition of Microsoft Mechanics, we are joined by Distinguished Engineer
Pablo Castro to look at how you can bring state of the art search capabilities to your custom
applications and content management systems. ...including the latest enhancements with
semantic search to for ranking top search results and “answers” that uses machine
reading and comprehension to surface answers to your search queries.
So, Pablo, it’s a pleasure to have you on Mechanics.
Thanks Jeremy, happy to be here It’s real privilege to have you
on the show as one of the leaders in intelligent search for Microsoft! Congrats
on today’s announcements. Before we get into this though, for those of you new to
Azure Cognitive Search, it’s a PaaS solution in Azure that allows you to integrate sophisticated
search capabilities into your applications. This is what large industrial manufacturer Howden,
use to be able to quickly drill into the details of customer equipment requests,
so that they can respond with accurate bids. The Azure Cognitive Search platform helps you
to quickly ingest, enrich and explore structured and unstructured data. You can integrate Cognitive
Search into, your customer facing mobile apps, ecommerce sites and line of business apps.
So, Pablo, Azure Cognitive Search really brings together some of the
best work from Microsoft across search and AI and makes it available to everyone…
It really does, we’ve been lucky to be able to harness a lot of the amazing work of Microsoft
Research. Combined with our extensive partnership with the Bing Team, we’ve developed a lot of
advancements for Azure Cognitive Search And this has been a multi-year journey
Everything starts with data ingestion – so that you can bring data in from any source,
You can automatically pull data in from an Azure data source
Or push any data you want to the search index, using the push API
Of course, this content isn’t uniform, it exists in different formats, it's anything from records
to long text and even pictures, So, we taught search to extract
and index information from many formats, applying machine learning techniques,
to find the latent structure in data For example, we extract key phrases,
tag images, detect language, locations and organization names and you can also
bring in your custom skills and models. This combination of cognitive search with
cognitive services, in fact makes search smart enough to
understand the content that it is indexing Right and in fact a few years back on the show we
showed a great example of this. We took the John F Kennedy files, comprised of handwritten notes,
photos and typed documents, and with Azure Cognitive Search we could understand the data,
and surface insights that had never been seen before...
Yes, the sophistication of data ingestion, the smarts to understand and index the content, along
with keyword search is something we’ve had for a while and we’ve seen many of you take advantage
of this for lots of interesting scenarios Today we are announcing the next steps on
this journey first with the new Semantic Search capabilities that include, Semantic relevance,
captions and answers in preview today Can you explain what these help you to
do with an example? Absolutely.
I have this demo application that’s fronting a Cognitive Search index with a dataset that’s often
used for evaluation purposes called “msmarco”. Let’s search for “what’s the capital of France”
You can see that the results match the keywords in our search, but it looks like the ambiguity
the word “capital” in particular caused top results to be a bit all over the place – we
see Capital Punishment, Capital Gains, the capital of Kazakhstan, Removals in France.
Now I’ll enable something brand new, semantic search, and this auto-enables spelling as well.
With Semantic Search, the Azure and Bing teams have worked together to bring state-of-the-art,
learned ranking models to Azure for you to leverage in your custom search solutions
Now if I go back to the page you’ll see the results side by side, keyword search
on the left and semantic search on the right You can see how on-point the new results are
They are about France and they discuss its capital – these are links brittanica and worldatlas
Note that this huge improvement in quality
only required me to enable this option Now let’s take this to the next level with
semantic captions and answers as well Let me go back here into settings
and enable both of these features Now not only do we see relevant results,
but we can see captions under each result that are meaningful in the context of our query
We can also see an actual answer proposed by Cognitive Search
So, now you can create the same kind of experiences that web search engines offer,
but for your own application and your own data So the answer is presented directly in the context
of search and you aren’t having to click on a link to find your answer. So what does it take
to add Semantic Search to an app? It’s not that hard to get it running
Let’s first walk through how to ingest and enrich data in Cognitive Search, and then we can dive
into the new semantic search capabilities. First thing to do is to create a Cognitive
Search service, I already have one created and I’m here in the portal with it open.
I’m going to use “import data" to start the process.
You can see we support many Azure stores, in this case I’m going to point to an existing blob
storage account with unstructured data in it In the next step I can enable one or more
Cognitive Services or custom models to enrich the data I’m ingesting. So I’ll add enrichments
For example I can enable OCR, entity extraction, computer vision and more.
Finally I have a chance to customize my index definition, and to setup indexer options.
At this point I’m done and have an ingestion process that will run automatically, detect
changes, enrich data, and push it to my index. I already created an index before do
we don’t need to wait for this process. Let’s go into this index and give it a quick try.
I can search for say “France” and I can see results coming back
Now, in your application you’ll typically use our of our client libraries or the HTTP api.
Here I’m in VSCode, and this is a typical HTTP request for the Search API.
Let me run it so we can see results for the same search we did earlier.
You can see we get the keyword search results
Now I’ll just change the queryType to “semantic” and re-issue the query
You can see that now I’m getting the new more relevant results thanks to semantic relevance.
That one line was all I needed. A few more tweaks will enable captions and answers.
Since this option doesn’t require re-indexing, you can easily try
this in existing applications as well. And these are very easy API calls, but
behind this at the service level, there’s a ton of complexity…
Right, we take care of the data science to give you state-of-the art search results
without having to create your own custom ranking models from scratch.
And at the same time, we also take care of the infrastructure to run our ranking and
machine learning models efficiently and fast So all of what you’ve shown would take a ton of
effort to build yourself! What improvements have been made to make this possible?
So, let’s start by explaining traditional keyword search, here you would match each
word in a search query against an inverted index, which allows for the fast retrieval
of documents based on if they contain words or search terms that you are searching for.
This returns documents that have those words. The problem with this is it only returns exact
matches, and ranking is often only based on the relative frequencies of words.
Sometimes that's what you want such as searching for part numbers,
like the example you gave earlier with Howden when you know precisely what to look for.
However, when searching through content written by people,
you want to capture the nuance of language So we added a few key components
to improve both search precision and recall First we added a step so as a Search Query
comes it passes through our new spelling correction service to improve document recall
Then we use the existing inverted search index to retrieve all candidates and we pick
the top 50 candidates using a simple scoring approach that's fast enough
for scoring millions of candidates quickly We then added a new prep step for these results
by running another model that picks parts of document that matter the most based on the query
From there the results are re-ranked again via a much more sophisticated
machine learning model that scores document relevance in the context of the query
Still, there’s a lot more to it in terms of how the intelligence works! In the examples
you demonstrated, you showed how the semantic search wasn’t just matching words but understanding
the context of the words relative to the surrounding words. What makes this possible?
This is where we take advantage of the recent advancements in natural language processing
Let’s put this in to context in terms of what this means for ranking and then answers:
The first thing we need to do is to improve recall to find all the candidate documents
In our search a key concept was the word “capital” The search needs to understand that the word
Capital could be related to states or provinces, government, money, finances
or a number of other meanings. So, to go beyond key word matching,
we use vector representations where we map words to a high dimensional vector space,
These representations are learned, such that words that represent similar concepts are
“close” together in same bubble of meaning These represent conceptual similarity,
even if those words have no lexical or spelling similarity to the word Capital
How does it find the relationship between words?
Now that we have solved for recall we need to solve for precision in the result.
Here transformers, which is a novel neural network architecture, enable us to think about the sematic
similarity not just of individual words but of sentences and paragraphs.
This uses an attention mechanism to understand long-range dependencies in terms,
in ways that were impractical before, particularly for models this large
Our implementation starts with the Microsoft developed Turing family of
models that has billions of parameters. We then go through a domain specialization
process where we train models– to predict relevance using data from Bing.
For example, when I search for “What is the Capital of France” reading the whole
phase it’s able to identify the dependency between capital and France as a country that puts “Capital"
in context and quickly returns the result with high confidence, in this case Paris
Separately, we also build models oriented toward summarization
aswell as machine reading and comprehension. With captions we apply a model that can
extract the most relevant text passage from a document in the context of a given query.
And for answers, we use a machine reading and comprehension model that identifies
possible answers from documents and when it reaches a high enough
confidence level it will propose an answer. And Microsoft takes care of the infrastructure
to run these models, but as you say they are ‘large’, so what are you doing
to operationalize these models in production so that you avoid slow search results?
You are right, these models can be compute and memory hungry and expensive to run, so we have
to right size them and tune them for performance all while minimizing any loss in model quality.
We distill and retrain the models to lower the parameter count so that they can run
fast enough to meet the latency requirements of a search engine.
THEN to operationalize this, we deploy models on GPUs in Azure and when a search query comes in,
we can parallelize on multiple GPUs to speed up scoring operations to rank search results.
Great so all of this offers a great foundation then to achieve high precision and relevant
search results. There’s a lot behind the few lines of code that lights up these powerful capabilities
for custom search inside of apps! How can everyone watching learn more?
Try it out for yourself – you can sign up for the public preview of Semantic Search today
at aka.ms/SemanticPreview And we have more guidance on
how to get started with Azure Cognitive Search at aka.ms/SemanticGetStarted
Thanks Pablo! And of course, for the latest updates across Microsoft,
keep streaming Microsoft Mechanics, subscribe if you haven’t yet, and we’ll see you again
soon. Thanks for watching!