[MUSIC PLAYING]
KRIS OVERHOLT: Hello, and welcome.
My name is Kris Overholt, and I'm
a developer advocate who works with conversational
AI in Google Cloud.
I'm very excited to talk with you today
about how to boost your conversational experiences
with generative AI.
We'll also get into the details of how you can quickly
build out an amazing customer experience with something
called Gen App Builder so let's get started.
First we'll review the current capabilities of chatbots
and how they affect the customer experience today.
Then we'll get into specific aspects of the next generation
of conversational experiences, including
informational transactional and generative capabilities.
We'll start with a customer journey that most of us
have probably been through firsthand,
a frustrating experience with a chatbot or a virtual phone
agent.
Let's say that you want to buy tickets
for a concert in San Francisco on May 5th.
The call starts with a greeting.
Then the virtual agent proceeds to ask one question
after the other about the location,
the performer, the date, the number of tickets,
and lots of confirmations along the way.
After this very long and painful interrogation process,
it turns out there are no tickets
available for the date that you actually wanted anyway.
We can see that this interaction was
very mechanical and robotic.
And the customer experience was similar to filling out
a long and tedious form one field at a time.
And only after you've completed the entire form, in the end
you're disappointed that we can't help you anyway.
This kind of interaction usually ends
with the customer repeatedly yelling
into their phone "customer support, customer support,
customer support," until they're ultimately
transferred to a human agent to handle their issue.
Now instead, let's walk through a more natural and pleasant
customer experience.
In this case, rather than following prompts
from the chatbot, the customer just speaks
in plain natural language and states
all of the important details up front,
just like they would when talking
to another person in real life.
I'd like to buy two tickets for this particular concert in San
Francisco.
Then we move on to available dates and payment.
Here we used natural language understanding and machine
learning models on the back end, and the chatbot
was able to deconstruct, parameterize,
and act much more quickly and efficiently with the customer.
Now even when you have powerful natural language understanding
models on the back end, implementing a virtual agent
is still a very complex process.
It's very common for chatbot developers
to end up with virtual agents that have hundreds or even
thousands of components, since they need to represent all
of the different paths that a customer
might take along the way.
This results in a very large and complex conversation flow
that's difficult to implement and even
more difficult to update, just like the conversation graph
you see on the right side.
As a result, developers often get
lost in the logic and the routing of the virtual agents,
which takes away valuable time and effort that
could have gone into designing a more pleasant customer
experience.
And this is exactly where generative AI comes in
to help us better align our chatbots with conversation
design principles.
If we follow fundamental conversation design
principles and our development tool
handles the implementation details for us,
then we're much more likely to end up
with a simple and intuitive user experience, which
means happier customers, increased
call containment rates, and overall better experience
for customers.
For the rest of this session, we'll
focus on three types of conversational capabilities
that you can combine to create amazing customer experiences.
First, informational capabilities,
which means that customers can ask more open ended questions
with natural language to quickly get the information they need
without chatbot developers having to manually define
thousands of questions and answers. ,
Second transactional capabilities.
This means that chatbots can now handle a wider range of tasks
and actions automatically rather than
needing to escalate to a human agent
early in the conversation.
And third, generative capabilities.
This means combining the first two capabilities
to make the customer experience personalized.
For example, we could generate or rewrite content in real time
based on how the question was phrased, or even based
on a specific customer profile.
Now that we understand the specific challenges
and complexities involved with implementing
conversational experiences, let's
walk through the timeline of large language models
and generative AI capabilities so we can understand
how we got here and how we can use these together to improve
the customer experience.
We'll start by discussing what generative AI is
and what large language models are.
Large language models are built on top
of an understanding of general knowledge,
because they're trained on a very large corpus,
or a set of documents.
Because of this, LLMs can handle much more complex tasks,
like auto completion, information retrieval,
sentiment analysis, and intent detection.
And if you're wondering about the term "large"
in large language models, it means
that these models have on the order of billions
of parameters.
And generative AI is the application
of these kinds of models to produce content,
such as text, images, video and code.
In the case of chatbots, this generated content can actually
help customers solve their problems faster
by pulling from a larger realm of general information.
Now if we look at the timeline in terms of machine learning
and AI developments, Google has always
been at the forefront of AI.
You can see this both in our research and in our products.
Over the last few years, we've made significant investments
into building and training large language models.
Back in 2017, there was a breakthrough research paper
on transformers published by members of the Google Brain
team.
At that time transformers represented a huge leap
in how well these models performed in the real world,
and ever since then Google's released
several other large language models,
like the ones you see here on the timeline.
Alongside all of this research, we've
also been productionizing and making
these technologies usable for our own customers
in Google Cloud.
We're very excited to continue to translate
these technical breakthroughs into products
that help billions of people around the world.
Now you might be wondering, where does all of this
actually fit into Google Cloud?
A couple of months ago, we announced a few new products
related to generative AI, including
foundation models, enterprise search, and conversational AI.
These products help developers build
apps that surface generative AI capabilities in lots
of different ways.
For example, foundation models are available in Vertex AI,
and they include generative functionality
for things like text, code, and images.
Enterprise search complements the foundation models
to provide up to date and targeted information
from both internal data sources and public websites.
Conversational AI, which is the main focus of our talk today,
enables developers to steer customers
towards productive actions using informational capabilities,
and can answer questions like, how much would my phone
bill change if I added international data roaming?
Or even transactional capabilities like,
help me pay this month's phone bill.
Now we're going to focus on conversational AI
and learn how generative AI can enhance customer experiences.
Specifically, we're going to take a deeper look
into the capabilities provided by generative AI App Builder,
or Gen App Builder for short.
With Gen App Builder, we're going
to look at the informational, transactional, and generative
capabilities that we described earlier.
For now you can think of Gen App Builder
as a very fast and easy way for developers
to quickly create apps like chatbots, virtual support
assistants, and custom search engines
with minimal development overhead.
And behind the scenes, the conversational capabilities
in Gen App Builder are powered by Google Cloud's
natural language models, speech models,
and of course large language models.
We've been helping our users deliver more natural chat
and voice based experiences over the last several years,
and we do this by using virtual agents that
support natural language understanding,
multi-turn conversations, and conversational technologies
that are built with dialog flow in Google Cloud,
and the same deep learning technologies that power
the Google Assistant.
Now for the latest generation of virtual agents
that we're talking about today, we're
really just getting started with LLM
powered conversational experiences and generative AI.
And it's going to drastically change the way
that we develop virtual agents, as well as how
customers interact with them.
In this section, we'll talk about a few different ways
that Gen App Builder can help you
build amazing conversational experiences,
both in terms of faster chatbot development and directly
improving customer experience.
In the past, if you needed to provide an informational chat
experience on your website that could answer hundreds
of different questions, you would actually
need to manually create a very large amount of intents
and responses in your chatbot.
Then you would need to specify additional variants that
represent different ways that a customer might
ask their question.
But with Gen App Builder, you can
skip over all of that tedious step by step process
and easily create an FAQ based informational bot that
uses existing content from your support knowledge bases or even
your internal documentation.
On the back end, Gen App Builder uses advanced natural language
models to parse and tokenize your website and FAQ content,
organize it into question and answer pairs, and then
route users to the most appropriate answers
at the right time.
If you use this approach, you can quickly
set up a conversational app that can
be embedded on your website or mobile app,
and it's instantly ready to help your customers find
the right answers.
Another capability that we'll discuss
is called generative fallback responses,
which also focuses on customer-facing conversational
functionality.
In the past, if a user said something
that was not represented in the logical design of a chatbot,
the virtual agent would fall back to a default intent
and respond with some variation of "I'm sorry,
I didn't get that.
Can you rephrase?"
And as we saw in the example of a frustrating customer
experience, this is a very common point
of irritation for customers when it
happens over and over during a call or a chat session.
Instead of trying to design for hundreds of potential edge
cases and potentially overengineering our chatbot,
we can instead use generative AI and large language models
to create responses for users in real time.
For example, we might have a customer
who has some questions about travel policies
before they actually purchase a flight.
Or we might have another customer
who wants to know about regional health care
information in a particular country before they travel.
Instead of leaving these customers
in the dark with inactionable responses,
we can actually use generative responses in our conversation
to help us get customers back on the happy path
in the conversation flow.
Now we'll move on to more development-focused
conversation tasks.
The next capability we'll discuss
helps developers build conversational experiences
faster using natural language prompts.
Earlier we talked about how developers can manually
create intents, routes, pages, and flows within chatbots.
They can do this either using the Visual Flow Designer,
or by using client libraries in Python, Java,
Node.js, and other languages.
But if we go one step further, what
if we could just describe the task that we want
the chatbot to perform and then our development tool
handled all of the details and gave us back a fully
implemented conversation flow?
Again, that's where Gen App Builder comes in.
As you can see here, we don't have
to deal with flows, intents, and other low level objects,
but instead we describe what our virtual agent should
do to help the user, including the task definition, the end
goal, parameters, and even API endpoints
and validation checks that should happen along the way.
Gen App Builder then uses a large language model
on the back end to parse all of these instructions
and quickly build out a virtual agent that's ready for us
to use.
You can think of this capability as a way for chatbot developers
to focus more on the conversational design
and less on the implementation details, all thanks
to the power of LLMs and generative AI.
The final capability that we'll talk about also
focuses on speeding up the development
time of virtual agents.
But this time we're going to talk
about handling complex graphs and state
transitions more efficiently in conversation flows.
Essentially, you describe the conversation flow
at a very high level, and then an LLM-based intelligence model
generates and auto completes a detailed conversation graph
and helps you quickly build out a more comprehensive customer
experience.
Of course, you can still customize every aspect
of the conversation flow and dig deeper
into specific points of the customer journey
when you need to.
The traditional conversational development console
is still there underneath Gen App Builder
if you need more control, or if you
want to preserve backward compatibility
with your existing virtual agents.
Looking back at the session, we covered a lot of ground
in terms of generative AI capabilities
and new approaches for LLM-powered chatbots.
But what does this actually mean for the future
of customer experiences?
It means new generative conversational capabilities,
easier chatbot development, and the best of both worlds
when it comes to combining domain specific information
with general knowledge.
I'll wrap up our session today with a question
that we get asked about a lot in recent times.
In a world where there are large language models
and generative AI, why do we even need chatbots at all?
Why not just drop a large language model
on my company website and move on
to other interesting problems?
Well, the answer has a lot to do with the scope
of a large language model versus the scope of a virtual agent.
So if you remember from earlier, a large language model
is trained on a massive corpus, which
results in a model that acts like a generalist across lots
of different topics.
And it's really designed to handle
open ended conversational dialogues or content
generation.
On the other hand, a virtual agent
is a curated, custom built conversational bot
that can guide customers on an optimal path
to a specific solution.
So if you take one thing away from our discussion today,
just know you don't have to choose between generative
AI or chatbots.
With Gen App Builder, you actually
have access to more options to provide a great customer
experience that captures the best of both worlds--
rich AI-generated content, along with goal
oriented conversation steering.
Now I want to leave you with a few different resources
you can visit for more information.
First up, if you want to learn more about generative AI
and Google Cloud, including everything from foundation
models to enterprise grade generative AI solutions,
you can visit the first link.
If you actually want to try out and give feedback
on emerging technologies like Gen App Builder,
you can visit the second link to join the wait list
for our Trusted Tester program.
And finally, as you're exploring and trying things out yourself,
we'd love to hear what your experience is like
and how we can help you build an amazing experience
for your own customers.
You can get in touch by using that third link.
As you can tell, we are very excited
about the new conversational capabilities powered
by generative AI and the positive impact that
will have on improving customer experiences everywhere.
Thanks so much for your time, and for attending the session.
I hope you learned something that
will be useful in your own journey
to build amazing customer experiences.
And I hope that you enjoy the rest of your time at Google
I/O. Thank you.
[MUSIC PLAYING]