(gear sounds)
- Coming up we explore the latest
Azure Search capabilities that leverage hosted AI Services
to allow you to quickly index unstructured data in
any format so you can knowledge mine untapped information
inside your organization.
We're gonna walk you through a real-world example
and demonstrate the steps that you can take to make
previously idle data inside your organization
fully searchable, making it faster to discover and visualize
relationships across your data and add your own
custom AI models and classifiers.
(techno music)
- I'm joined today by Cindy Noteboom, welcome.
- It's great to be here.
- So last time we covered the topic of knowledge mining and
AI we had CVP Julia White on the show who actually
showed us the potential of Azure Search against the publicly
available John F. Kennedy assassination files
made up of hundreds and thousands of handwritten notes,
photos, and even typed documents to create a searchable
body of knowledge that really unearthed some really
interesting and new insights and correlations in that data.
- Right the advantage of the Azure Search platform is
it allows you to bring in data of lots of different formats,
indexing it so that it's referenceable and searchable
using AI.
- Right and one of the things with the JFK files,
it's not really a niche example it's more relevant
than a lot of people might think.
- Exactly.
For most organizations that have been in business for awhile
we estimate that at least 80% of their data is unstructured.
And this includes documents of different types, images,
and even text that contains valuable information.
Since it's unstructured it's hard to discover and
manipulate the information contained inside.
- Right, now for people that are new to Azure Search,
just to clarify this is very different to a
content management service like SharePoint.
Can you explain what it is and really how things change?
- Azure Search is a Platform as a Service, or PaaS solution,
that allows you to build sophisticated search capabilities
within your applications on your data.
For example it's easy to integrate search within
your line of business applications.
The Azure Search platform allows you to quickly ingest,
enrich, and explore your data.
You can automatically pull data from Azure data store
such as blob storage, Azure sequel, Cosmos DB, and more.
Optionally you can push data directly into the search index
from other cloud storage locations
using the Azure push API.
You can bring in different file formats such as
PDF, Word documents, images, JSON files, and more
and literally crack them open to extract information.
Next you can further enrich your information
in order to build a rich body of searchable metadata
within the prebuilt cognitive skills that
are integrated into the search platform.
These prebuilt skills are AI models across
vision, language, and speech that let you do things
like extract key phrases, tag images, detect language,
and identify key places and organizations.
You can even chain these skills based on the task
you need to accomplish.
For instance you can use OCR images from a document
to extract text, combine it with the rest of the text from
the document, and then run organization and location
detection across all of that content.
- [Jeremy] And I can see here that we have a
skill for language detection.
Does that mean that you can actually go across
different languages and find search results?
- That's right you can!
This really breaks down those types of traditional barriers
giving you universal search parameters.
Our language detection can detect the key phrases
that you care about across languages
and return them in your search results.
- Right now you mentioned location and also
language detection,
how long is the list of skills that I can bring in?
- It's actually unlimited.
Beyond what we give you, you can build your own
custom skills and integrate your own
custom machine learning models.
For example, if you are in manufacturing,
you might be interested in extracting materials
or part numbers from documents
so that you could later filter your technical documents
based on the part numbers you're most interested in.
All of these capabilities combined give you
a powerful search experience.
- And all this sounds really great from
a feature point of view but I'm curious about what it takes
really to get all this built up and powered by Azure Search.
Can you make this real for us?
- Let me start with a real example.
Our customer Howden is a manufacturer of
large industrial equipment.
They build very complex machinery used across industries
such as manufacturing, mining,
oil and gas refining and more.
They need to increase the speed and efficiency
of their bidding process.
To set some context, a typical customer bid can span
thousands of pages and there's limited time to respond.
Their skilled engineering team has to scour through
detailed customer requirements to identify
key areas of design and specialized components
in order to produce accurate bids.
If they miss out on these types of specifics,
they can bid too low and lose money
or bid too high and lose the customer opportunity.
Their process has been very manual, time consuming,
and sometimes prone to human error.
- Okay so they have loads of data they need to review
and really generate proposals with.
And it's clear they need a way to speed up this process
and reduce the chance of missing important design features
and maybe customer requirements.
Now how are they using Azure Search to do this?
- Let me show you Howden's application.
They use Azure search to develop
a collaboration and search application.
I'm gonna search for documents related to
exchangers in this project.
As documents are ingested into the platform,
they are processed, categorized,
and grouped via the search index.
Out of the box cognitive skills like
key phrase extraction and named entity recognition
generate a rich corpus of metadata.
On the left hand side of the screen
is an example of faceted search,
which provides self-directed drill down navigation
that can be used to further guide engineers
in their review process.
Standard features such as these mean that
engineers no longer have to spend enormous amounts of time
doing that work manually and are less likely
to miss important elements of each document.
On top of the out of the box skills,
Howden has customized the cognitive search pipeline
to extract and enrich metadata specific to their business.
They've created a custom classifier that buckets documents
by international engineering regulations
and they've also added their own custom skills
to flag design requirements that are known
to create risk in the cost estimation process.
For instance, stainless steel tagged here may be a risk to
system design when requested as part of the specification.
Let's look at this document a little more deeply.
- Right now if I'm reading this right
I can see that it's a really long document its got 71 pages,
that's going to be something that takes
a lot of time to comb through.
- Not only is it very long but it's also very technical.
So you can imagine it might be easy to miss
important details in the document when
preparing the proposal response.
Looking at the risks tab I'll see the autogenerated risks
that were populated from the
custom skill classifier I mentioned earlier.
In certain cases changes to standard or additional
requirements like these may come in and engineers
will need to escalate to peers in order to analyze
and re-estimate the cost.
They can do that here in the tool,
noting the issue and requesting that it be reviewed.
The human collaboration is actually now building on
their body of knowledge originally provided by Azure Search.
These new comments and new risks are
searchable as well once they're added.
- Okay so it's really interesting,
often when I think of search I think it of just
maybe a simple search bar or maybe something in an app
or in a webpage to help find content
but here you've actually built a whole workflow
around the information discovery process for the business.
Now how would I go about setting something like this up?
- Let me show you how you can set something up
similar through the Azure portal.
- [Jeremy] Okay.
- [Cindy] Now this process can be done programatically
but for the sake of time
we'll focus on the portal experience today.
I'll start by clicking on the import data
in my Azure Search service.
The Azure Search service is well integrated
with Azure databases.
But I'm gonna go ahead and use a source that I have today.
- [Jeremy] Okay.
- [Cindy] Next I'm gonna click on 'Add Cognitive Skills.'
Azure Search is now automatically crawling through my data
to validate it for the next step.
- Right so can you tell me more about
the data that's actually being loaded right now?
- I have a combination of structured and
unstructured content in here.
I've got in some Word documents, some PDF's,
some Excel files, and even
some images that I've pulled down.
Next I'm going to enrich my content
with the Cognitive Search feature.
Since I know I'm wanting to use
cognitive skills on this data set,
I'll attach to the Cognitive Services in my
Azure subscription and now
I'm gonna go ahead and add my enrichments.
I need to give myself a unique skill set name,
I'm gonna enable OCR,
next I'll define that skill set that uses the combination
of cognitive skills that are useful for me.
The capabilities that we discussed before
like OCR, key phrase extraction,
and named entity recognition are really as simple as
checking off a box.
In my case I'm gonna extract everything in my options list.
- [Jeremy] So these were all thing that we saw earlier
with the faceted search at Howden.
- [Cindy] Exactly.
And now I can move on to configuring my target index.
This in your chance to set the parameters that are defined
based on the end user experience
that you're looking to create
such as making different fields retrievable,
filterable, sortable, and so on.
My final step is to click 'Create Indexer.'
Here I can set a schedule for how regularly I want
the Azure search indexers to recrawl my data source.
Since information is often added, deleted,
or changed from the source location,
this is a simple way to make sure that
the search experience stays up to date.
From here, I just hit submit.
- Right so it seems really easy then to hook up some data
with some built in skills,
but everyone's data is unique and often I may need to have
maybe unique logic or
something that is unique to my business.
What are some of the options that I have
to build custom skills?
- As I mentioned earlier, Cognitive Search allows you to
extend the out of the box functionality with custom skills.
We've made customization flexible and extensible.
As long as you provide a secure endpoint
and follow the defined input and output schema,
you can use any framework and any language
to build your custom skill.
This extensibility enables a broad range of features
such as creating custom filters,
classifying documents,
extracting custom entities and more.
For example Howden was able to create a custom classifier
based on standard engineering regulations
that we looked at earlier.
- Right and this is something that developers are gonna love
and it really gives the flexibility for people to write
custom logic in whatever tool or language that you're using.
And as long they can respond back to Azure Search,
it's gonna understand and answer everything
and it just works.
- Exactly.
This gives you full flexibility and control
for your customizations.
- So what are the most common ways that
we can build those custom skills out?
- There are two really common ways for
creating these custom skills.
The first one is to use Azure Functions.
If you don't need an ML model,
deploying an Azure function is the quickest and easiest way
to create a custom skill.
Creating tags for domain specific terms like
stainless steel or classifiers for
specific industry regulations are great examples of
ways to use Azure function.
So I have here a template we've put together based on
a simple Azure function.
For something like the Howden example
where I wanted to create a custom skill that
identified specific metals or fabrications.
I simply updated this sample right here
with a few lines of code.
Once I'm satisfied this skill set is working as I want,
I can then publish this skill set and programmatically
connect the new skill to my document cracking pipeline.
- Very cool.
So you mentioned earlier that Azure functions
are a great example when you don't need an ML model,
but what if I've got something that's a bit more custom
that requires machine learning.
- You can build ML models to enhance your
cognitive search pipeline with highly-tailored skills
such as industry specific entity recognition models,
to extract data such as
mechanical parts or industrial chemicals.
If you wanna learn more about how to create
ML-driven custom skills,
you can visit the link shown.
- Okay so now we've been able to process this data with
some built in and some custom skills,
but we need to surface this data somehow to users.
So how would I go about integrating the search index
to maybe an app or a website.
- You can make your search index explorable
in a variety of ways such as integrating into a website,
chatbot, or even a mobile app.
I have a sample web interface in visual studio right here.
All I need to do is update the app settings JSON file
to connect the web app to my storage account,
search index, and app insides account.
Then it's just as simple as going in
and publishing the project.
Now we have our webpage up and
let me just show you that we've got some really nice
search results very simply and easily.
- Very cool.
So now the app is actually configured,
it's running, it's got by default all the different facets
and tags and fields and all the key phrases right here.
Really cool stuff, Cindy.
Thanks for joining us and also the fantastic
hands-on introduction to Azure Search
with AI for knowledge mining.
Of course my next question is:
where should people go to learn more?
- You can find detailed information and walkthroughs
on Azure Search at the link shown.
We've got some really nice code samples and documentations
of this type of work available on GitHub as well.
Where you take your experience from there
is entirely up to you but for a quick proof of concept,
this can be a great place to start.
- Thanks Cindy.
Of course keep watching Microsoft Mechanics
and subscribe if you haven't already.
That's all the time we have for today's show,
thanks for watching and we'll see you next time.
(techno music)