>> You're not going to want to miss this episode of the AI show.
We're going to talk about the latest from
Azure Form Recognizer with my friend Vinod.
Make sure you tune in.
[MUSIC]
Hello and welcome to this episode of the AI show.
We're talking about the latest from
Azure Form Recognizer with our friend Vinod Kurpad.
How are you doing, my friend? Why don't you introduce yourself?
>> Hey, thanks Seth. It's great to be on the show.
I'm Vinod Kurpad, I'm
a Product Manager of Azure Form Recognizer,
and really excited to be here today to
talk to you about all the new things that
we're releasing with Form Recognizer with the new 3.0 GA API.
>> Fantastic. Let's just set a level playing field here.
For those that don't know about Form Recognizer,
can you give us a 30-second elevator pitch?
>> Definitely. Form Recognizer
has a number of different capabilities.
Think about it as starting with OCR,
you can extract all the content that
you want from any of the document types that you're dealing with.
We even support a few different office type documents.
For example, Word and PowerPoint,
though there's still preview features,
but those are available for you
to use within Form Recognizer today.
In the GA version of the API,
we support PDF documents or images,
and so you can extract text and content
from each of those different types
of document that you're dealing with.
As you go up the stack of Form Recognizer capabilities,
you can extract a number of
different semantic elements of the document as well.
Things like layout, tables, selection marks,
paragraphs, paragraph rows,
those are all capabilities that we have as part of the layout API.
If you go further up, you can even use the general document API,
which is think about it as
a document agnostic model
that extracts key-value pairs of a document.
You have a number of these common types of forms.
For example, you fill out a form, a doctor's office.
There's a number of different questions which
is like your name, age,
date of birth, past symptoms and things like that.
It's able to process some of these documents without
really needing you to build any custom model.
Because extract all the different key value
pairs that you need from each of
these different document types that you
can use in your automation workflows.
Then moving on, we have a few really specific document,
specific models, which are things like
an invoice model or receipt model.
We even have released now
a W2 model as well as health insurance cards.
We have a number of these different pre-built model types.
Which are really targeted at extracting the specific key value
pairs that you expect to see in each of
these different document types.
Finally, if none of these addressed any of your needs,
we have a set of
custom capabilities and we have
two different types of custom models that you can use.
One is called a template model,
which is essentially intended for documents that have a
very fixed and defined visual template,
and then we have a new neural model
now which is also part of our GA release.
Newer models allow you to extract
data from unstructured documents as well.
You can extract, for example, tables,
layout information, as well
as key-value pairs from unstructured documents.
Form Recognizer is intended to
serve all of your document automation,
document processing needs starting from simple text extraction all
the way to really specific document type
some of the scenarios that you might want to deal with.
>> This is really cool, because OCR is at the base,
it's not just OCR.
Then you have structure and
then you have specific structure and then
you have generic structure to
whatever it is that you want to do. Am I getting this right?
>> Yeah. It's perfect.
Because essentially what we have is,
I like to describe it as three sets of capabilities.
So you have OCR,
which essentially extracting text from a document.
Then you have these document agnostic models,
which are really intended to extract
structure as well as key value pairs from a document.
But they don't really care whether it's a document,
whether it's a receded invoice,
or a W2, or a contract.
It's all the same from a document agnostic model perspective.
Finally, we have these document specific model packs,
which are either pre-built document model types,
which are essentially a model that
Microsoft train list of data we've acquired,
which are things like the invoice,
W2, and things like that.
Or you could go and bring your own data, label it,
and use a custom model to extract
those key value pairs or
fields that you're looking for these documents.
>> I love it. What's the latest?
I know there's got to be some announcement that you're making.
What's the latest since we've talked last?
>> Yeah. I think what I'd like to do is
actually walk you through a quick update
of what's contained in the new 3.0 generally available API.
Because this API is now generally available
and it's got a lot of functionality
that we've added in
essentially a number of
different things that we've done over the past year.
Let me share my screen and
walk you through what we're talking about here.
>> Let's do it. Your screen is
shared and we're ready to go. Let's take a look.
>> Perfect. I'm going to start off by
talking primarily about the studio first.
The studio is essentially
the best experience that you can have with Form Recognizer today.
We've taken a lot of the feedback that you had about
the sample labeling tool and things like that in the past,
and really revamped the experience of the studio.
The studio allows you to do three specific things.
First is you can explore each of the different model types.
As a simple example, I can go ahead and click on "Read
here," the slope and the read API,
I can go ahead and click "Analyze" and now
I have a few sample documents here that I can choose from,
or I could bring my own document in,
and upload a specific document that I want to try this out
with and get the outputs from each of these models.
You can see here that in addition
to sample document that I've got,
I've got the output now and you
can see you've got each of these paragraphs identified,
and I can take each of the content elements that I want from here.
Studio, for example, is a great way for
you to go try out each of these models.
Let's go take a look at layout,
because layout is another model that got
updated as part of our V3 API.
Layout, if you remember from prior years,
if you've used layer in the past,
it usually extracts text,
selection marks as well as tables.
With this new update,
you'll notice that layout now does something lot more specific.
If you think about all the challenges you have with
document segmentation and finer grain detail within a document,
you can get those answers to some of
the questions that you have about documents you are dealing with.
We tell you what the paragraphs are,
but we also give you a specific role for each of these paragraphs.
You can see in this case,
the paragraph for the news today is a title type of paragraph.
You're normally getting all of the paragraphs identified,
but you're also getting each of the roles
associated with those paragraphs.
In addition to that, obviously layout also does
all the things that it's been doing for the last few years,
which is extracting selection marks as well as
tables and we're continuing to improve that experience as well.
For example, you can get tables as an example of data,
it just get extracted out of
the document with no real training required.
This is just a layout model doing its thing at this point.
>> This is amazing.
Before you go, basically this is
a place where someone can go and just try it out
on any document to see what the base stuff
does and it does analysis of
unstructured and structured documents, correct?
>> Exactly. Seth,
you hit the nail on the head.
The studio is essentially think of it as playground.
You can go and select a particular model that you want to try out.
Analyze a specific document like I said,
you could use one of the sample documents that are in the studio,
or you could go ahead and bring in your own document,
analyze it, validate that,
the results that the model is able to produce meets
your expectations and that's what you'd want to use.
Not just that, the studio takes it one step further
because it actually shows you what the result is in JSON as well,
and gives you a sample example of how you might want to
use one of the language specific SDK elements in this case,
this is a Python sample that
walks you through how you might want to
use this particular model using the SDK,
anything that you're building as an application for yourself.
>> Love it.
>> The general document example here that I have on the screen
walks you through the capabilities of a general document model,
which is essentially extracting
key-value pairs from each of these different documents.
Like I mentioned earlier,
this is a pre-trained model.
You don't need to train anything.
It just understands documents and is able to extract
all the different key value pairs that it's
able to find for within each of these documents.
There's a few other examples,
you're welcome to go ahead and give it a try,
but that's essentially the Form Recognizer Studio in a nutshell.
It gives you all of these different capabilities but it brings out
each of these things in a way that's
really easy for you to try out,
validate that it works,
and then take it and productionalize it
with it within the applications that you're building.
>> Awesome. Now,
this is now GAID so anyone can use this. Is this correct?
>> That's correct. There's two real components
that I want to talk about.
The studio itself is how you interact with
the Form Recognizer resource or the Form Recognizer APIs.
Then there's the underlying APIs itself.
Obviously the studio's GA,
so are the underlying APIs
and so we just talked a little bit about the studio,
let's dive into each of
the different capability areas maybe spend
a couple minutes walking through some of the updates.
Because there's a lot to unpack here in terms of like
all the different updates that
are available with Form Recognizer today.
>> Let's do it. Let's dive in.
>> We just looked at the general document model,
think of this as a really good start
to analyze any specific document type.
You can bring in any document that has
key value pairs or form fills in it.
Click "Analyze" and now you're able to get an output that
describes each of the different key value pairs
that are extracted as part of this document.
You'll see here that, for example,
as I hover over some of these key-value pairs,
you'd see the proposed start date,
as well as the specific value and so forth.
Each of these elements is now extracted
for you without you having to do anything more
than essentially just submit this document
as an API requests to the API.
That's key-value pairs or general document model.
This model continues to improve,
it can today extract all the things that layout can,
as well as these key-value pairs.
For example, you'll see that some of these things like tables that
are extracted as part of the layout are
also available as part of the output.
But in addition to the output from layout,
you're getting the key-value pairs as well.
You also have selection marks and so you can
see whether each of these selection marks are selected or
unselected and that's something
that is just native to the model that it's just able to
extract all this information from
any of these document types that is processing.
>> In addition to this,
the key area that I wanted to really
highlight in terms of what's new
with the new [inaudible] or API is,
we've now added a new model type called the custom neural model.
Custom neural models are different from
traditional custom template models
that you've trained in the past.
Because they really enable you to
use form recognizer with unstructured documents.
As an example, I'll walk you through
a quick labeling example of what
we typically do is
you need to come into the phone recognized studio,
set up a new project.
Go ahead and label
a few documents for what you're trying to extract.
The labeling process is super simple, it's intuitive.
All you need to do is go select the text that
you want and go ahead and label the document.
Let me show you what that experience looks like.
I'm going to zoom in just a little bit
so we get a better view of what's going on here.
You'll see that I have a few different labels.
I can create a new label in here.
For example, if I want to create
a new field that might be something like payment terms.
I can go ahead and create a new
field called PaymentTerms and you'll
see probably added to the bottom of the list here.
You can also reorder
this list and move things around if you wanted to.
I can go ahead and do that.
I've moved payment terms up here.
I don't know if there's any payment terms
in this particular document,
but let's assume that there was something that I needed to label.
It's just a question of
essentially highlighting the texts that I really want to
label and then selecting
the specific field that I want to leave it at that.
You'll see that that field
got labeled with the value that I'm looking for.
Essentially it's a question of doing this
for as few as five documents,
and that should then enable you to train a custom neural model.
In this case, what I'm dealing with is a few different contracts
and you can see that each of
these contexts are different because they're
all with a completely different structure,
different style written by
a probably different entities and as
a result, are significantly different.
But I'm still extending.
I'm still using the exact same model to
extract the key value pairs that I'm really interested in.
In this case, that what are the different parties,
and I'm also trying to do some things around
class identification as well to really extract some classes.
With this, I can go ahead and label a doc,
level as few as five documents.
Go ahead and train a model,
and let's go look at what the experience looks like
when I have a model train.
Now I'm going to try to analyze a particular document,
I'm going to go find the document here that I can use to analyze.
This is a document that I use with the same training dataset,
but it's not something that the model is ever seen before,
it's not a document that the model has ever looked at.
When I go ahead and use that document and analyze it,
I will get a similar set of results like what we
saw when we were using the general document model for example.
You'll see that I can get all the different key value pairs that
I was able to identify as part of the labeled dataset.
But I also get all of the things that were part of the model,
for example, from layout,
as well as the read options.
It's essentially, think of it as a aggregation of models.
What you're looking at is
a combination of the output from both read,
layout, as well as the customer that I'm running here.
>> This is really unstructured data.
That's the coolest part.
>> Exactly. Each of these document type is different.
This particular document, for example,
that we just analyzed is significantly
different from the other documents that we looked at.
But what you're seeing here, for example,
the model is able to extract all of
the different fields that I was looking for.
I've got a party
that I've labeled that I'm looking for as party 1,
I've got the areas for party 1.
I have party 2 that I was looking for as part of the document.
I've also got the areas of arty 2.
The agreement date,
I can also extract clauses.
It's not just about extracting specific fields.
I can go find the specific clause.
In this case, I'm looking for a force majeure clause.
I see that the force majeure clause on page 19 of this document.
In each of the samples force
majeure clause may have been on different pages and
different sequences within each of those documents,
and the model is able to identify what
I'm specifically looking for and extract
the right values depending on what the specific labels
describe without predator life
primarily on the structure of the document.
>> That's cool. Sorry I interrupt,
but just an observation.
As form recognizer has become more mature,
the documents that it's able to process have
become increasingly messier, which is really cool.
>> That is correct. In addition to that,
if you think about like, most organizations is no exception,
unstructured documents is becoming the norm in terms of
how organizations have to deal with this variety of data problem.
With form recognizer and with the new custom models,
this is essentially the future of
how we is your customers are going to want to
extract all of the different key points
that they want to extract from each of
these different document types they have to deal with.
>> This is amazing. Anything else
you want to show us or where can people go to find out more?
>> I'm glad you asked. There's actually a link at
the bottom of the page that you can go look at,
that should be the form recognizer documentation link
that shows you what's new and form recognizer today.
In addition to that, I just
wanted to call out that there's a number of
other enhancements and improvements that if you're interested in,
please go check it out.
For example, I'll talk about things like the pre-built models.
We've got a number of new pre-built models as well in
terms of things that you can try out.
The W2 model is an example of if you
got a tax or income verification scenario,
the W2 model is a great way
for you to read some of those processing.
We also have a new model for health insurance cards.
Each of these models is essentially
intended to help you solve a specific business need,
dealing with the specific document type.
But there's also a wide variety of
updates in terms of languages that we support.
We're now up to 164 different languages with real layout where,
in terms of languages that are supported within customer orders.
Actually languages that are supported with pre-built models.
We've now extended to Latin-based languages like German,
Dutch, French, Italian, Spanish.
We've also got a number of other improvements in
terms of both the SDKs,
APIs, as well as latest improvements.
There's a tangent bag,
there's lots more that I can share,
but I know we're running out of time.
I'll stop there and turn back on to you.
>> That's cool.
Make sure you go to the link,
it should be just below. I don't know.
It feels this is the AI that actually
helps businesses up level that work we're doing.
Instead of taking a little form and then typing it into Excel,
we can just have the computer do it for us so we can
work on what's really important
to our business, which is really cool.
Thanks for being with us, my friend.
>> Oh, thank you Seth. It's been my pleasure.
>> My friends, you've been learning all about the
latest from Azure form recognizer
with my good friend, Vinod Kurpad.
Thank you so much for watching
and hopefully we'll see you next time.
Take care. [MUSIC]