>> You're not going to want to miss this episode of
The AI show where we talk all about
the newest features from Azure Form Recognizer with my friend Lou.
Make sure you tune in.
[MUSIC].
>> Hello and welcome to this episode of The AI show.
We're talking all about the newest features from
Azure Form Recognizer with my good friend Lou.
Lou, why don't you introduce yourself?
Tell us who you are and what you do.
>> Sure. My name is Lou John.
I'm a product manager working on Azure Form Recognizer.
>> We're talking about the newest features
can you tell us all about them?
>> Sure. In the Azure Form Recognizer,
we have a bunch of models that can
help you extract tax key value pairs,
tables and the structures from your documents.
We have already done a lot of models,
including the document analysis,
prebuilt models to understand your domain
specific documents as well as the custom models.
In the latest release,
we have introduced the health insurance cards
and vaccination card as
the tool previewer model that you
can use in the house care scenario.
You can also do the custom models to understand
the unstructured documents like contracts to
help you easily manage your contracts documents.
>> That's fantastic. Basically we're adding new models.
You're adding I didn't do any of the work.
You're adding new models to automatically
recognize things like vaccine cards.
Is that right? How does that work?
>> Yes. Let me show a quick demo.
>> Okay.
>> What I'm going to show is this little app
which you can easily take a picture using,
this scans of COVID 19 vaccination card.
I just take a picture and I don't need to wait for that.
I have already approved,
process this vaccination card and that ID document.
As you can see from the results,
it can quickly extract
your key information from the vaccination card,
including date of birth, first,
last name and when you got
your vaccination for the first and the second dose.
It's very easy and
straightforward just to choose simple API calls,
you can got all these information.
>> Basically looks at the form and extracts all of that data
and the next question you jumped to it because how is this done?
You said two say simple API call, can you shows us how that do.
>> Yes. Exactly. You just need two simple API calls.
As I just mentioned,
we have two new prebuilt model.
One is the vaccination card and the
other is the health care insurance card,
which this is the example on our studio
and the key information we extract including like copays,
group number, ID number, insurer, etc.
As you might know, for
all these health insurance cards is issued by different companies.
There are a lot of templates,
which is not easy that you can
extract should the basic information.
But what we offer today is just the rest API,
which I'm going to show you and there's Jupyter Notebook.
The thing you just need to set our base deploy your form recognize
resource and input your end point and API keys and
specify the version and puts the post URL composed as this
and then use the headers has the key you have already got.
You can also passing through this source URL with
the pointer to the image as I just show using the string file,
use the base 64 format to feed in your file and then comes to
the most exciting part which you just need to do
a HTTP post with the headers and the body of the data.
Once it's analyzed you can then use
this get request to get the result.
The result is returned in a JSON format,
which you can see an easy simple here.
It includes all the taxing formation
with the bounding box and words.
It also includes the key fields which I just show,
the copays, the acts being I exposing information.
I have this function that you can pass
this JSON result and
the output will be something that look like this.
>> Interesting. It's literally just a post and
a get call to get
the stuff out of the form in the way that you want it.
>> Yes, exactly.
>> That's really cool.
Now, you did mention a little bit about custom things.
What did you mean by that?
>> Again, you can go back to the Form Recognizer studio,
which you can click on from the homepage,
the custom model tile,
and then you will enter this page.
We have two options template based models and neural models.
Today I'm going to show is the neural model which can help
you to understand unstructured documents like contract.
As you know contract there is really
this long document and it's unstructured.
It takes a lot of time for
these legal lawyers to analyze the documents,
and reveal them and manage them.
But with the custom neural model,
which you can easily build,
customize the model to train that with the view samples.
What you just need to do is to label
five samples from the agreement which I have already prepared.
You can highlight these labels,
different pattern names,
alias and they're like
that agreement title and effective date, etc.
Once you have finished the labeling,
you can simply click on the train button
and that will trigger the model training process.
We can name it the second iteration of the contract model.
Put some description, then select the
neural one and kick off the training.
This may take a few minutes to run and I have already prepared
one which finished the training
and once you get the model available,
you can just click on the model and click on test.
I can then just upload an image.
This is not like another type of contract.
It's not the same contract which I use for the training,
but it's a co-branding agreement.
But what I can simply do is just click on
the analyze button and it will
automatically extract basic information,
which I wan't you analyze and manage the contract.
>> As it's doing that, because I want to catch up a little bit.
Basically you take any contract that you
have that you want to do form processing on,
you highlight the things that you want to get out of it,
and then you train a model with examples of these things.
Am I getting this right so far?
>> Yes, exactly.
>> Then once you've trained them,
how many documents would you need?
Well, we'll get to the questions there.
You train the model,
and then you can give it new forms
that it can do inference on to get the stuff out.
>> Yes, exactly. If we zoom in a little bit, as you can see,
the training data I used is all like affliate agreement,
which is the one type of the agreement, the contract.
But the testing data which I'm doing
the prediction against is the co branding agreement,
which is another type.
But even though these are two different types of documents,
I can still use one which is available
for my training and to predict another type.
>> Interesting. Basically you're able to go through a ton of
forms and pull out the exact information you
want once you've trained these models. Is that right?
>> Yes. To answer your question,
how many samples do we need?
As you can see, these purple icons,
these are the ones I have already labeled.
It only takes you few minutes to label
five samples at least and you can get a trained model ready.
>> With five samples?
>> Yes.
>> That's pretty amazing.
What do we have to look forward to in the future?
Because I know this is right now.
But can you tell us more about what's coming down the pike?
>> Sure. Besides these models that I have just introduced,
I want to also highlight
the upcoming features we will be supporting in June.
For reads, which is the basic OCR instruction.
We will support small document types,
including all these popular office formats and HTML documents.
We'll also help your detect
page object which including paragraphs, headers,
folders from your document and enhance
advanced table extraction for the layout model.
I have introduced the new prebuilt model with
vaccination card and health insurance card,
but we are also going to do
the language expansion to existing models,
including business card and the invoice.
For custom model,
as I just demoed,
we will continue to support the tables,
especially the tables that cross
multiple pages that you can try using the custom model.
For studio you can also use the API,
end point and key based access,
and we are expanding to a US government end point as well.
>> This is all really amazing.
I love seeing what was what is now,
and then what's coming.
Where can people go to find out more?
>> You can definitely go
to our different lengths, aka.ms/AIShow/AutomateFormRecognizer.
Using this short link you can access all the resources available.
>> Fantastic. I know there's a learn more link.
If you want to learn more, there's a link right there for you.
Then finally, there's some FAQs there.
Thank you so much. Anything else you want to add before we go?
>> No. Thank you-all for watching.
>> Thank you so much, Lou,
for being with us.
We've been learning all about the newest features from
Azure Form Recognizer with my friend Lou.
Thank you so much for watching and hopefully
we'll see you next time. Take care.
[MUSIC]