[MUSIC PLAYING]
RAND WROBEL: Hi, I'm Rand Wrobel.
I'm the lead technical solution consultant with the Document AI
incubator team.
Well, we're talking about Document AI and Generative AI.
There are several approaches to now pull information out
of documents and to go further and generate
insights and answers and letters from them.
All right.
Well, we're using Document AI with automation on Vertex
with Generative AI to get more information out
of documents quicker and easier, cut your development costs,
and be able to customize what you want
and generate answers and information and insights
that we really couldn't have done a few months ago.
So the portfolio here has really three parts.
There's Document AI, which is kind of the classic extraction
now powered with Generative AI.
We have Vertex Search, which will
go across your entire set of documents.
And you can kind of limit or control
what we call faceted search, various parameters
of that search, certain document types, certain users, things
of that sort.
And then with Vertex AI being multimodal,
we're coming out with a document modality.
And this means that Gemini understands that it's not just
an image, it's a document.
And that gives more power to do these sort of document
processing and understanding of tasks.
So going on into it, the first demo
is end-to-end automation of ingesting documents
and answering questions and going
into not just extraction, but really document understanding.
What can you tell me about the documents?
So for here, we've imported about 8,500 prospectuses.
We've imported user profiles.
And we've created an app to make all that available to you.
So we've done the processing part by importing and storing
the data into tables, and now we're going into understanding
and provide even conversational-- kind
of a smart chat bot.
So this is the overall architecture
where we're populating the documents into an ERP and CRM
systems and various databases that have
the information for retrieval.
And we've done all the embedding that's needed for Gen AI.
Then, over on the user side, we have a user interface
that will take each input from the user, create the prompt,
use the context, and come out with the result.
So in this end-to-end processing,
we can go into this demo.
And here we're using simple investments
as a Generative AI example.
And for example, you can just do a search here
on those prospectuses.
For example, I want to find investments
that deal with high inflation and hedge against it happening.
So we can just do a keyword search
sort of thing straight out of the product OOB,
and we get certain results that are not the ideal results.
But if you turn on the semantic search,
you get much better results that actually
understand what you're after.
So then if you're looking for prospects,
you've loaded the user profiles.
So if you're after a young, aggressive investor,
you can go ahead and find all of those investors.
But perhaps you want to make sure that they're pretty young.
They're not old because the style
of investment you're after is more long-term in older ones.
So you can control the search with parameters.
This is the faceted search.
And it will filter your results according to what you've put in.
Next, you can just start asking questions against all this data
that you've accumulated.
So, for example, here's an email from an investor
who inherited some money.
And they're asking, well, what should I invest in?
And just with that information, you can generate a result here
that's kind of general.
But you can also go into that same question
but turn on some advanced features
that take into account--
you can tune the role.
You can tune the mission.
You can tune the output instructions.
But moreover, you can tune it to the user
and what their preferences are.
So here we know the user ID, and we'll use their history.
And now we get very much different result.
It's tuned to what the user's parameters are.
Lastly, you can go into research on these prospectuses
and do some queries about, for example,
how a particular investment would be used.
And it generates the answer on how
would the proceeds from the investment
be used in the company.
That gives you one answer.
And you can do the same thing in an embedded widget
that you can put anywhere in your user interfaces.
So here we can say, "How will the proceeds be used?"
Well, it got it.
And we can pick a particular one here in this case.
Right.
There we go.
OK.
All right.
So that's the solution of using custom aspects of this
to build really a complete system
to ingest certain types of documents,
populate databases, and then use that for document understanding.
And we also have an approach for just doing
document processing and analytics
where we use base products.
And in this case, it's called Custom Document Extractor--
Custom Extractor.
And without really knowing any prompt engineering
or any technical aspects, you can just
give an example document and tell it what you want.
I just briefly wanted to show you this new document modality
that's coming out on Gemini.
And in this case, we're just using
a simple prompt-- there's no training involved--
on an invoice.
And it will do all the extraction of that information
from it.
Again, no training.
This is just prompt engineering.
And it's extracted the results, the information
from the invoice.
I have to say, this is so crazy.
It's like science fiction come true from my perspective.
We're doing things we couldn't do six months ago.
And in another six months, I have
no idea what's going to happen.
It's like the hockey stick inflection point
in the technology.
It's just crazy good.
I'm very excited.
[MUSIC PLAYING]