>> You're not going to want to miss this episode of the AI show.
We talk all about elevating your document management with
Azure AI Document Intelligence
with my friend Bema, make sure you tune in.
[MUSIC] Hello and welcome to this episode of the AI show.
We're talking all about elevating your document management
with Azure AI Document Intelligence with my friend Bema,
how you doing, my friend?
>> I'm doing great, how are you?
>> Great, so tell us who you are and what you do.
>> Yes, I am a Product Manager on
the Azure AI Document Intelligence team and I'm
super excited to show you what we've
brought to our new public preview.
>> Azure AI Document Intelligence sounds new to me.
It's not a product I've heard about, can you fill me in?
>> Yes you are exactly correct.
The first thing I'd actually like to discuss is the rename.
We are now the product formerly known as, Form Recognizer.
We now go by Azure AI Document Intelligence,
and we didn't just change the name
because it looks better on paper.
We made this change because we
believe that the new name more accurately
represents what our product
does now and what we plan to do in the future.
We don't just deal with forms,
we deal with all types of styles of documents,
both structured and unstructured.
Our goal is to empower customers to be able to do more with
their documents and enable
powerful document understanding use cases.
With our most recent public preview,
we have taken another big step in continuing to reach this goal.
You will notice that in this new studio,
this change has been already reflected,
and that's both in the naming in the studio and also in our URL,
which is now documentintelligence.ai.azure.com/studio.
>> Lovely.
>> If you're ready,
let's just hop in and look at what's new in this new preview.
>> Let's do it. This is cool.
It used to be called Form Recognizer,
but now it's Azure AI Document Intelligence Studio
has a new URI and everything.
What are some of the new things,
because we've done a lot of Form Recognizer,
but I bet there's also new stuff in here as well.
>> There's a lot of new stuff.
Let's start with layout, actually.
Awesome, we have made many changes to
the layout model to make
working with LLMs easier for our customers.
Layout now supports office document types such as
PowerPoint and Word documents,
as well as different document types it already supported,
including digital and scanned PDFs and all types of image files.
You now have more options than ever when uploading
documents and can parse all this data with a single API call.
>> That's cool.
>> The Azure AI Document Intelligence Layout Model offers
a comprehensive solution by providing
advanced content extraction and
document structure analysis capabilities.
With this model, you can easily extract paragraphs,
tables, titles, section headings, selection marks,
font style, key value pairs, math formulas,
QR codes, barcodes, reading order, so many things.
>> That is quite the list.
>> Quite the list indeed,
and all of this in 309 printed languages
and 12 handwritten languages.
>> You could basically extract anything?
>> Exactly.
>> Any language? You said 309 languages,
you said that so fast, 309 languages.
>> Three hundred and nine printed languages and
12 handwritten languages and we're constantly adding to this list.
>> That is cool, so what are you looking at?
>> What you see here is some new analyze options for Layout.
What you'll notice here is that
Markdown is a new output format style.
Markdown is extremely popular with web applications and
LLM tooling and now if I select this and I run the analysis,
we will see that the output is now in Markdown,
which is extremely easy to integrate
in other applications that use LLM technology.
>> Not to interrupt you here,
but this is really cool because I'm
able to do this as an intermediate step.
I'll put Markdown and put it directly into
an LLM call for example, which is really cool.
>> Exactly. This helps to enable
really powerful scenarios with semantic chunking or rag scenarios.
In fact, I want to jump to a little application that
we've made to actually show some of
the power behind what this can enable.
What you'll see here is a little application
that we spun up very quickly,
but basically behind the scenes,
we're using Layout from Azure AI Document Intelligence,
and we've used that to create semantic chunks that
make it easier for the model to understand what it's looking at.
Now the model has better insights into table structure,
it understands sections better.
As you can see, this is a press release for the fiscal year 24.
It understands a lot of context a lot better
than if we were not running this with layout behind,
so now because of this
it enables me to ask more specific questions.
For example, let's ask this model can you
summarize what constant currency is?
>> As this is executing,
is this literally calling Azure AI Document Intelligence,
pulling in the page,
it's breaking it up too,
and then it's putting that in?
Is that what it's doing? Or maybe explain what it's doing?
That way I can get a sense.
>> If you look at
the structure which has been
pulled out from this document, for example,
we'll see that it's able to recognize the different paragraphs,
the different section headings,
break them into different semantic chunks, and label them.
What this does, when this data is given to a large language model,
it's able to use this semantic chunking in order to
gain better understanding of exactly what is in the document.
This helps because it also does things
like maintains reading order.
All of these extra benefits allow the model to
have a better understanding of exactly what
it is looking at and therefore give better results.
>> That's what we're seeing in this application here that you're
showing where you're typing in
questions and it's referring to the document.
>> Exactly. Because it has
such a deeper understanding of what is going on in the document,
it's able to provide much better answers.
>> It's amazing. These questions,
like I said, this is cool stuff.
Another question I have is can you do a little bit more?
Because there's other LLM technologies
that people have been using;
can this integrate with that?
Or is this application
available or how can I make something like this?
>> I'm so glad that you mentioned that.
If you are familiar with a lot of LLM technology,
you've probably heard of LangChain.
LangChain is one of
the most popular LLM orchestration tools
used to create LLM applications.
We are soon going to have native support for
Azure Document Intelligence as a document loader.
One of the really cool things I wanted to show off was,
all of this will be available soon,
I promise you, in the next week or so,
but you'll be able to quickly,
with a few lines of code, go in,
set up the LangChain,
and then you will be able to load and semantically split
different chunks and ask questions using rag pattern.
We're really excited to
see how our customers are able to integrate
LangChain and Azure Open AI in order to do this process.
>> That's cool. Can we scroll up a lot because I thought I
saw Azure AI search as well, is that true?
>> Yes. Azure AI search is
part of the packages that are included in this demo,
but there are many different ways to do this.
This is just one way to quickly get off the ground with using
LangChain and Azure Open AI to create these powerful work flows.
>> This is cool because if you're building LLM stuff,
it's usually hard to figure out how to
do the chunking window because it's like,
do I take the first n characters or how do I overlap?
But what Azure AI document search is doing is it's
semantically chunking the data into,
for example, reasonable paragraphs,
which I think is really cool. Am I getting this right?
>> What we're doing is we're providing
a very good basis to be used in semantic chunking algorithms.
The output of layout can be
fed into these semantic chunking algorithms to provide very easy,
basically good data to be used to create strong chunks.
We all know in AI, if you put garbage in,
you're going to get garbage out,
so basically what layout does is it makes sure that we provide
high quality data to
these semantic chunking algorithms
so that when you are using them,
you're getting good quality data
out when you're implementing these rag patterns.
Cool, that's the layout stuff and we're super
excited to see how people implement that into their workflows.
>> What else you got, my friend?
>> One of my favorite things,
one of the features I'm really excited about,
I really want to show off, is query fields.
Let's jump into the studio so I can show you how that works.
We're going to look at the invoice model here.
One of the biggest complaints that we've
gotten about our prebuilt models is
that sometimes they are
not sufficient enough for some customer use cases.
Now, don't get me wrong, the prebuilt models are amazing,
they require no training,
they require no labeling,
and customers is able to use them out of the box,
but sometimes we have customers with
specific use cases that need a little bit more than we offer.
Let's take this UK invoice, for example.
If I'm to run this, we're going to see all the fields that
our customers are used to seeing in the invoice prebuilt model,
we have things like customer address,
customer address, recipient, customer ID, all that good stuff.
But let's say for example in this specific use case,
we have a customer that needs more.
They need the campaign number,
they need the sort code,
and let's say the account number.
These fields are not fields that we extract out of
the box because most of
our customers don't need this information,
but in the case that they do,
we now have an answer for them, and that's query fields.
>> I see. Generally, the model is very
good at a preset like general stuff or invoices,
but there may be specific things for invoices,
for your organization, for example,
that you want to extract and this is what this does.
>> Exactly. We're never going to be able
to cover all of those use cases out of the box,
but this creates a very easy way
for customers to add this to their solution.
You'll notice this new pretty button
right here called query field,
so if I select this,
basically what we're going to do here is where you're going to use
camel case style naming
to name the new fields that we want to extract.
For example, we said campaign was one that we wanted to extract.
I'm going to select that,
and then also sort code
again just using very simple camel style naming.
Then let's just add one last one.
Let's say account number. Cool.
>> I just have to click that.
I acknowledge that I am using Query Fields. Save that.
Now when I run this analysis,
again what we should see is,
boom, campaign is extracted.
We have sort points, and we have account number.
All this is beautifully integrated
directly into the model to common names.
Customers can now use these fields in their solution.
>> That's cool.
There's got to be bound to be stuff that you want to pull
out that you can totally just put that in there.
Is that something you can do programmatically as well?
You have to show us.
But is that something you can do like in the post request as well?
>> Exactly. That is exactly the case.
>> Those Query Field were awesome because it enables folks to
actually pull stuff exactly the stuff out that they want.
What else you got to show us, my friend?
>> Yeah, so let's not jump into the custom classification model.
Custom classification models basically allow customers to
run a group of documents through a model
and be able to instantly tell which documents
are in there based off
of classifications that they themselves have made.
>> Oh, that's cool. Because then you can
chain these things to pull out the right forms.
Now right forms, and then do other things.
For example, if one form has a custom field,
then you would want to be able to know
that's the right form, then you could do the other form.
Then the other question I had about
this because I heard about this before.
If there's a multi page document that
has multiple different kinds of forms in there,
does that work as well?
>> Yes, so you are able
to classify whatever it is that you want to do.
Let's say that we have a form that
has three different types of forms in it,
forms A, B and C, but they're all one collective.
If you wanted to split them
into their respective parts, you could do this as well.
You just have to train the classification model
to represent document A,
B, and C and then it would work just like that.
>> Can you show us how to do that?
>> Yeah, so the custom classification model
has actually been expanded to different languages,
and one of these is French.
I've set up a demo here just to show how this could work.
I've actually set this up just for the sake of time,
but it's extremely easy.
All you have to do is upload a folder which has two different sub
folders one holding document type A
in this case which is medical healthcare type documents,
and then folder B which has professional service type documents.
Then the service will actually split these
for you automatically and use the folders as labels,
which is why we have this nice ratification without
having to do this manually.
>> That's cool.
>> Yeah, so
once you're done with this, you're going to hit train.
Again, I've done this just for the sake of time,
because it does take a little bit of time.
What this is going to do is it's just going
to generate labels and everything,
so that it understands the document structure.
Once you have a trained model, you can go ahead and test it.
That's what we're going to do now. We're going to go to test.
I'm going to browse for a file.
Let's choose this sample file and run the analysis.
What we should see here is the model being
able to tell which bucket this belongs to and we do.
We have this being recognized as a professional service document.
>> That's super cool because now you're able to
have any number of different kinds of documents.
You're able to have
document intelligence tell you what kind it is.
Maybe even break apart a MultiPage document
and then you can process
those things with custom fields for each kind of document,
so you really got like a really powerful end to end solution.
>> I know I said that this was last thing,
but we actually have one more thing that I want to go over it.
>> one more thing.
>> A couple of small updates that
we've made to the service, if that's okay.
Awesome, so the first thing that I
want to point out is that with the read API,
we've actually added support for
handwritten extraction for the languages
of Russian, Arabic, and Thai.
We're really excited about this.
We have customers all over the world and so
these are some really highly requested languages.
We're excited to announce that those are now
available for you to try in the studio today.
We also have new pre built models.
US tax documents is an area that we've been
investing a lot of resources into,
and we're happy to announce that we now have available to
US tax 1099 forms and all of these variations as well.
Again, these empower very powerful scenarios.
These 1099 forms are things that people use for taxes every year,
so we're really excited to see how our customers can
use to make their document intelligence processing even easier.
>> Just in time for tax season too.
>> Exactly.
>> Cool.
>> That was not a mistake.
Along with this in the invoice model actually,
we've added support for KVK number,
which is a critical field in the Dutch locale.
We've also added support for BPAY information,
which is critical in the Australian locale.
With these changes now more fully
support these locales and we hope that this will allow
customers in those locales to have more robust solutions
without having to rely on things like custom models.
>> That's cool.
>> Then the last update that we have
here is in health insurance cards.
We now have support for Medicare and
Medicaid numbers built into the health insurance card model.
Again, we just are very
excited to see how this empowers customers to have
more robust solutions that allow
them to do more with their document intelligence needs.
>> Yeah, and this is cool because it could even be a picture.
You could build applications where
like let's just say you're a medical place and it's just
like here just pulled your card under our camera and then
Azure AI Document Intelligence can make that
sing if I get it right, am I getting this right?
>> You are getting this right.
That is one of my favorite things about
the service is how robust it is.
I even remember when I first joined the team,
I drew out an invoice with my hand on
plain white paper and it was able to
extract all the information correctly.
It's amazing how far the technology has come.
>> That is amazing.
This has been awesome.
Where can people go to find out more?
>> Yes, so the first place to go would
be the Azure Document Intelligence Studio.
Here you can find
resources that will help you get up and started very quickly.
Allow you to test out all of
our services right here in the studio.
Another good place to go would be our documentation.
We keep all the information
up to date with what's coming, what's new,
and there'd be another great place for you
to check out what we have coming through the pipeline.
>> Well, this has been amazing.
My friend, thank you so much for spending some time with us.
>> Thank you so much for having me. It's been a blast.
>> Also thank you so much for watching and
learning all about elevating
your document management with
Azure AI Document Intelligence here on the AI show.
Thank you so much for watching and
hopefully we'll see you next time, take care.
[MUSIC]