JOE FERNANDEZ: In this video, we'll
explore how you can build an AI-powered conversational
search solution for your content.
Let's go.
[MUSIC PLAYING]
Hey folks.
Welcome to "Build with Google AI,"
where we explore how you can build practical solutions
with Google AI technology.
Searching for information is one of the most common uses
of artificial intelligence language models.
Ask a specific question, get a specific answer.
It's a pretty compelling experience,
even if the answer isn't always correct.
But can you build that kind of experience
for your content and your users?
Yes, you definitely can.
Docs Agent is an open source project
built by one of Google's technical writing teams.
It interacts with an AI language model,
like the AI model behind Google Bard,
and uses a clever system to build custom prompts for the AI
to answer developer questions in a conversational format.
Here's a quick demo of the project.
For this demo, we're using the Flutter developer
documentation.
As you can see, Flutter has a lot of great developer content.
However, I have a really specific question.
I just want to know how to use Flutter
to display really long lists because I'm pulling data
from a database.
The Docs Agent provides a web-based interface
where you can ask a conversational AI
model a question about your content and get an answer.
So I just type in my question and click the Ask button.
Now I just have to wait a little bit while it
talks to the language model.
And bingo, I've got an answer.
You can see here that it's suggesting
I use the ListView.builder constructor, which
sounds like a good idea.
The thing I really like about this response
is that it gives me context.
It tells me what information it used to answer my question,
and it includes links to relevant documentation sections
so I can follow up and get more details, which is really handy.
But wait, how did they do that?
Did the team build an AI language model
and train it on Flutter content?
No, they didn't.
They actually used a simpler technique,
and you can do the same thing.
Let's talk to the project team and get
them to explain how it works.
All right.
So I'm joined today by a few of the key contributors
to the Docs Agent project, Kyo Lee, Nick Van der Auwermeulen,
and Meggin Kearney.
Thanks for being here.
NICK VAN DER AUWERMEULEN: Thanks for having us.
MEGGIN KEARNEY: Thanks for having us, Joe.
KYO LEE: Happy to be here.
JOE FERNANDEZ: All right.
So you all are technical writers at Google,
and what you all have built is really cool and useful.
But I have to ask you, why go to all this trouble?
Doesn't Search work just fine?
MEGGIN KEARNEY: Search is great.
But if you notice, it has a lot of links,
a lot of pages, a lot of reading to get
to single sources of information.
And the promise of conversational AI
is to help you have a conversation with almost
like a human to get the kinds of answers
that you want for yourself.
And so, as tech writers, we're always
trying to give our audience the simplest, most useful
explanation for technology.
So we were very curious about this new platform
to see if it could work better for our end users.
And we're also curious about it as a tool for us
to see if it could improve our own documentation production
life cycle.
So anyway, we thought it was worth a shot.
JOE FERNANDEZ: OK, so how does this conversational AI
system actually work?
NICK VAN DER AUWERMEULEN: Yeah, so one
of the important things to know about this project
is we actually didn't train or fine-tune the actual AI model.
When we started this, we didn't really know what we were doing.
We're kind of exploring what to do with this.
So one of the things we came across that we could do
is we could give additional context to the AI model.
So what we did is we basically took small chunks of text--
this could be headings and paragraphs, for example.
We turn those into numbers, like the numbers are not actually
a digital fingerprint or anything like that.
It's an actual representation of the data.
We store all of that data in a vector database.
So when you ask a question-- we use the same text
embedding to also convert that question into numbers as well.
And then when you ask your question,
it actually searches against the numbers
and it gets the closest context that you
can return from the documentation set we gave it.
And then it just answers your question
and also gives you relative links as well.
So that way, you can actually check the content
because sometimes there's a lot of ambiguity--
did it actually answer my question-- so you can actually
fact-check and make sure it was accurate for you.
JOE FERNANDEZ: All right.
So how can I extend this project to work with my own content?
KYO LEE: So with the Docs Agent project,
the main thing you have to do is to populate a vector database
with your own content.
First, you've got to break up the content
into small plain text chunks and create embeddings
for those text chunks and store them in the vector database.
JOE FERNANDEZ: OK.
KYO LEE: But important assumption here
is that you have your source content in a text format.
In our project, we use markdown because it's
relatively a simple text format, which is widely used
and it still has meaningful formatting information
like headings.
So when we first started the project,
we initially divided up the content
at random, which was using the 1,500 characters--
splitting the content by 1,500 characters.
But this ended up producing less relevant context
for AI model which consequently generated a less
helpful response for users.
So we switched it to a different approach
where we split the content by headings.
So now every text chunk contains headings and the paragraphs
under that heading.
This approach worked a lot better in terms of the answers
that the AI model was able to generate.
So the key takeaway here is that when you process content
into a vector database, you need to split up
the content in meaningful chunks.
In other words, be careful not to split up
the content at random.
JOE FERNANDEZ: OK, great.
So I don't actually have to build my own AI
model to make this work.
But wait a minute-- isn't this kind of cheating?
MEGGIN KEARNEY: Well, AI is about to be
one of the most transformational platforms
that we've seen in a long time.
And, for sure, fine-tuning and training models
is something that requires a lot of knowledge and experience.
And there are a lot of people with PhDs working on that.
But our little band of Docs Agents people,
we wanted to see if it was possible to participate
in the AI space in a way that was
more inclusive and a little bit more accessible.
And so that's kind of why we started experimenting with it.
And we learned really fast that it is, first of all,
not that hard.
And this is a way for people to start
to use AI that's very approachable,
but also, it's really fun.
And every week, we meet on a Friday and we hack together.
And we've been able to expand upon all of the things
that we're learning and share with a small community so far.
But we're hoping that community gets a lot bigger, so.
JOE FERNANDEZ: OK, well, great.
Thank you so much for coming.
I appreciate you talking about the project with me today.
NICK VAN DER AUWERMEULEN: Yeah, thanks for having us.
KYO LEE: Thank you for having us.
MEGGIN KEARNEY: And thanks for asking us to do this.
It's been really fun for us.
It's really cool to be here.
[MUSIC PLAYING]
JOE FERNANDEZ: It's coding time.
Let's quickly go over the technical details
of extending this project for use with your own content.
Don't worry about taking notes.
Link to a detailed tutorial is in the description.
After you clone the code from the Git repository,
go through the installation process to complete the setup.
Remember, you need to get a Google PaLM API key
and set it as an environment variable.
You also need to have a directory containing
all the content you want the AI agent to search against.
Ideally, this content should be in markdown format
and include titles and headings.
Next, you'll need to prepare your content
for processing and storage in the vector database.
For this step, you'll need to split up the content
into small text files so they can be encoded into vectors.
If your content is in markdown format,
you can use this markdown to plain text script,
which divides up your content based on titles and headings.
Once that's done, use the populate_vector_database script
to generate vectors and split up the text
and add them to the vector database.
This process uses the Google PaLM API
to generate vectors using an AI text embedding function.
The text embeddings are numeric representations
of the text content.
In other words, they're used to approximate the meaning
of the text as a set of numbers.
Having a numeric approximation of the meaning
is really useful for matching a question
with related information.
And that's exactly how this AI agent
finds content related to your question
before it gives you an answer.
The project team discovered that they got better results
when they split the text into meaningful chunks, headings,
and related paragraphs.
So if you have well-structured, well-written documentation,
this embedding process will be more effective
and the search function should perform better.
And now, we'll accelerate this processing
through the magic of editing.
Once you've finished populating the vector
database with your content, the system is ready to test.
Run the chatbot launch script from the root directory
of the project.
And now you can open up the web interface
and start asking questions.
I know I skipped a few steps, but you'll
find detailed instructions in the tutorial, which
is linked below.
Let's wrap this up.
[MUSIC PLAYING]
Using AI technology for content search
can be really game-changing.
Being able to ask specific questions
and get specific answers feels a lot more
like a human interaction, and so it's no wonder
that lots of people are excited about it.
However, just like human beings, AI systems
sometimes get it wrong.
So when you're building a system with this technology,
make sure you put in checks to help
folks verify answers and make sure that they're
on the right track.
And that's all the time I have to tell you
about the Docs Agent project.
Thanks to my guests, Kyo, Nick, and Meggin
for building this project and helping me share it with you.
Links to the code project and a tutorial
are in the description.
If you get something working, let us know in the comments.
I really hope this episode helps you kick off your own project
for building AI-powered search for your content
and helps your users find answers faster.
So keep learning, keep building, make something great.
We'll see you again soon.
[MUSIC PLAYING]