[MUSIC PLAYING]
DEBI CABRERA: We're here at Cloud Next.
I'm with Dimitris.
Hello, Dimitris.
Thank you for being here with us.
DIMITRIS MERETAKIS: Hi.
Nice to be here.
DEBI CABRERA: And he's going to talk
to us about building apps faster and better with Vertex AI,
right?
DIMITRIS MERETAKIS: Exactly.
DEBI CABRERA: Great.
So before we get into that, can you
give a brief description of yourself
and what you do here at Google?
And then give us a little synopsis of your talk.
DIMITRIS MERETAKIS: Yeah.
So my name is Dimitris Meretakis.
I am a product manager within Cloud AI.
And I'm focusing mostly on Search and Document AI.
So the talk is about the following.
We see a lot of developers building generative applications
for enterprises.
And one of the key aspects there is grounding.
In other words, how do you make those applications reliably
access the right enterprise data in order
to produce responses which are accurate and consistent?
We also see that in trying to build those applications,
there is a number of technical challenges
that developers have to face.
And they come up again and again.
So what we are talking about in this talk
is we're launching a number of new APIs and some improvements
to existing services to effectively solve
those technical challenges.
And we hope that developers who adopt those APIs
can use them in order to lighten the load there and focus really
building what is really unique for their own use cases.
DEBI CABRERA: As a developer, I love
to hear that, when you can help me make my job a little simpler
and let me do the stuff that I really like.
DIMITRIS MERETAKIS: That's why we're here.
DEBI CABRERA: Wonderful.
OK, so walk me through some of these new Vertex AI APIs.
DIMITRIS MERETAKIS: Yeah, there are quite many, actually.
There are six things that we are talking about today.
So the first one is a document understanding API.
So most of those applications, they
have to process large amounts of documents
that come on all sorts of weird, complicated formats.
There are sections and subsections and tables
and complicated graphs.
And typically, we see that such documents
create problems for retrieval and answer generation
components.
So this first API is using our know-how from DocAI
to understand the documents, understand the structure,
and help then improve the quality of those applications.
The second one is a significant improvement
in our embedding API, our gecko model has existed some time.
We are launching some improvements
that make those models some of the most
performant in the market.
They are now some of the leaders in
the corresponding leaderboards.
The third one is around vector search.
Vector search is our, I think, one of the most performant
and scalable and cost-efficient applications
for doing embedding retrieval.
So we are enhancing this with this launch
to enable hybrid search, which gives extra tools for developers
to improve the quality of their applications.
The fourth launch is about a ranking API.
It's a very elegant concept, where when you have a question,
and you retrieve a bunch of results from your search,
this API is, effectively, checking how each of the results
is, How good is it in answering that question?
And this helps you basically bubble up
the most relevant information in order
to improve the quality of the answers
that your LLM model will produce at the end.
The fifth one is a grounded generation API.
It's using the power of Gemini.
And it's a fine-tuned model, which is specialized exactly
in taking a question, taking a bunch of evidence,
and producing well-grounded answers with citations
to the reference information.
And the last one is an API that we call check grounding.
And this is used to, basically, fact
check a statement against a number of facts.
So basically, you have a statement.
It can produce by a human or even a language model.
And this API will check every sentence
of that statement against all the evidence
that you might provide to it to say, this statement is supported
by result number three or, in an upcoming launch,
to say that actually, I do find some contradicting evidence.
This statement is not only irrelevant.
It's actually negative.
So that's all, just six things.
DEBI CABRERA: That's all, just six new Vertex AI APIs--
amazing, wow.
Each one of those sounds really wonderful
and something I'd like to try.
DIMITRIS MERETAKIS: Imagine if you use all together.
DEBI CABRERA: Yes, exactly.
So in terms of all of these APIs,
what are some of the standout features that set them apart?
DIMITRIS MERETAKIS: Yeah, good question.
I think there are two, two main things.
The first is quality.
For these things to exist, they need
to produce the right results.
The second thing is those APIs embed
a lot of our Google know-how.
So we try to select problems that are popular enough,
big enough, coming frequently enough from developers,
but also, where we can actually add
some of our special knowledge to produce
solutions that solve the problem really, really well.
And you can see that, for example,
if you take the document understanding API.
It's based on years of investment on Document AI.
And also, in our deep understanding of,
What kind of processing do you need
to do to documents in order to have a high-performing search?
Another example is the Vector Search.
Vector Search, we are proud about how the efficiency
and the scale it can achieve.
And this is because it's, effectively,
using the technology that some of our planet scale applications
use for Google.
YouTube is using this technology.
Google Ads is using this technology.
So we bring really something unique to the market
with this solution.
And then if you take the ranking API or the ground generation
API, they are implementing concepts
that we have already available in our Google websites,
the consumer product, since years for producing good results
and lately for producing generative responses
for results.
DEBI CABRERA: Wonderful.
So they're high quality, and they're
unique and really focused on problems
that users face significantly.
DIMITRIS MERETAKIS: We hope we sprinkle a little bit of Google
magic on those as well.
DEBI CABRERA: Yes, a little Google magic
sprinkled on there, love that.
Great.
OK.
So in terms of using these APIs, how
can developers seamlessly integrate these APIs
into their workflow?
DIMITRIS MERETAKIS: Yes.
We design those APIs as primitives,
very simple, standalone, stateless
APIs with clear interfaces, so that a developer can easily
try them, understand them, apply them to the problem
and see if it works.
We hope that it does work and then use them.
We are also investing in integrating them
in popular frameworks, like LangChain or llama index,
so that developers can prototype with them,
but also, combine them with other APIs
from third parties, open source, as well,
to build together the solutions they want to build.
DEBI CABRERA: Amazing.
Well, thank you so much for being
here with us today, Dimitris, and telling us
all about these new APIs.
And have a great rest of your Next.
DIMITRIS MERETAKIS: Thank you very much.
[MUSIC PLAYING]