[MUSIC PLAYING]
[APPLAUSE]
SAMEER SAMAT: Hi, everyone.
It's great to be back at Google I/O. Today you've
seen how AI is transforming our products across Gemini, Search,
Workspace, and more.
We're bringing all these innovations
right onto your Android phone.
And we're going even further to make Android the best
place to experience Google AI.
This new era of AI is a profound opportunity
to make smartphones truly smart.
Our phones have come a long way in a short time.
But if you think about it, it's been years
since the user experience has fundamentally transformed.
This is a once in a generation moment
to reinvent what phones can do.
So we've embarked on a multi-year journey
to reimagine Android with AI at the core.
And it starts with three breakthroughs
you'll see this year.
First, we're putting AI-powered search right at your fingertips,
creating entirely new ways to get the answers you need.
Second, Gemini is becoming your new AI assistant on Android,
there to help you anytime.
And third, we're harnessing on-device AI
to unlock new experiences that work
as fast as you do while keeping your sensitive data private.
Let's start with AI-powered search.
Earlier this year, we took an important first step
at Samsung Unpacked by introducing Circle to Search.
It brings the best of Search directly into the user
experience, so you can go deeper on anything
you see on your phone without switching apps.
Fashionistas are finding the perfect shoes.
Home chefs are discovering new ingredients.
And with our latest update, it's never been easier to translate
whatever's on your screen, like a social post
in another language.
And there are even more ways Circle to Search can help.
One thing we've heard from students
is that they're doing more of their schoolwork
directly on their phones and tablets.
So we thought, could Circle to Search
be your perfect study buddy?
Let's say my son needs help with a tricky physics word problem,
like this one.
My first thought is, oh, boy, it's
been awhile since I've thought about kinematics.
If he's stumped on this question,
instead of putting me on the spot,
he can circle the exact part he's stuck on
and get step-by-step instructions right where
he's already doing the work.
Ah, of course.
Final velocity equals initial velocity plus acceleration times
elapsed time.
Right.
I was just about to say that.
[LAUGHTER]
Seriously, though, I love how it shows
how to solve the problem, not just the answer.
This new capability is available today.
And later this year, Circle to Search
will be able to tackle more complex problems involving
symbolic formulas, diagrams, graphs, and more.
Circle to Search is only on Android.
It's available on more than 100 million devices today,
and we're on track to double that by the end of the year.
[APPLAUSE]
You've already heard from Sissie about the incredible updates
coming to the Gemini app.
On Android, Gemini is so much more.
It's becoming a foundational part of the Android experience.
Here's Dave to share more.
[MUSIC PLAYING AND APPLAUSE]
DAVE BURKE: Hey, everyone.
A couple of months ago, we launched Gemini on Android.
And like Circle to Search, Gemini
works at the system level.
So instead of going to a separate app,
I can bring Gemini right to what I'm doing.
Now, we're making Gemini context aware
so it can anticipate what you're trying to do and provide
more helpful suggestions in the moment--
in other words, to be a more helpful assistant.
So let me show you how this works.
And I have my shiny, new Pixel 8a here to help me.
[APPLAUSE]
So my friend Pete is asking if I want to play pickleball
this weekend.
And I know how to play tennis, sort of--
I have to say that for the demo--
but I'm new to this pickleball thing.
So I'm going to reply and try to be funny.
And I'll say, is that like tennis but with pickles?
[LAUGHTER]
And this will be actually a lot funnier with a meme.
So let me bring up Gemini to help with that.
And I'll say, "Create image of tennis with pickles."
[LAUGHS] Now, one new thing you'll notice
is that the Gemini window now hovers in place above the app
so that I stay on the flow.
OK, so that generates some pretty good images.
What's nice is I can then drag and drop
any of these directly into the messages app below.
So, like so.
Cool.
Let me send that.
[APPLAUSE]
All right, so Pete was typing.
And he says-- he sent me a video on how to play pickleball.
All right, thanks, Pete.
Let's tap on that.
That launches YouTube.
But, you know, I only have one or two burning questions
about the game.
And I can bring up Gemini to help with that.
And because it's context aware, Gemini knows I'm looking
at a video, so it proactively shows me an "Ask this video"
chip.
So let me tap on that.
And now I can ask specific questions about the video.
So for example, "What is the two-bounce rule?"
Because that's something that I've heard about
but don't quite understand in the game.
By the way, this uses signals like YouTube's captions,
which means you can use it on billions of videos.
So give it a moment.
And there, I get a nice, succinct answer.
The ball must bounce once on each side of the court
after a serve.
OK, cool.
Let me go back to messages.
And Pete's followed up.
And he says, "You're an engineer,
so here's the official rulebook for pickleball."
OK, thanks, Pete.
Pete's very helpful, by the way.
OK, so we tap on that.
It launches a PDF.
That's an 84-page PDF.
I don't know how much time Pete thinks I have.
Anyway, us engineers, as you all know,
like to work smarter, not harder.
So instead of trolling through this entire document,
I can pull up Gemini to help.
And again, Gemini anticipates what I need and offers me
an "Ask this PDF" option.
So if I tap on that, Gemini now ingests all of the rules
to become a pickleball expert.
And that means I can ask very esoteric questions,
like for example, "Are spin serves allowed?"
And let's hit that, because I've heard that rule may be changing.
Now, because I'm a Gemini advanced user,
this works on any PDF and takes full advantage
of the long context window.
And there's just lots of times when that's useful.
For example, let's say you're looking for a quick answer
in an appliance user manual.
And there you have it.
It turns out, no, spin serves are not allowed.
So Gemini not only gives me a clear answer to my question,
it also shows me exactly where on the PDF to learn more.
Awesome.
OK.
[APPLAUSE]
So that's a few of the ways that we're
enhancing Gemini to be more context aware and helpful
in the moment.
And what you've seen here are the first
of really many new ways that Gemini
will unlock new experiences at the system level.
And they're only available on Android.
You'll see these and more coming to hundreds
of millions of devices over the next couple of months.
Now, building Google AI directly into the OS
elevates the entire smartphone experience.
And Android is the first mobile operating
system to include a built-in, on-device foundation model.
This lets us bring Gemini goodness from the data center
right into your pocket, so the experience is faster while also
protecting your privacy.
Starting with Pixel later this year,
we'll be expanding what's possible with our latest model,
Gemini Nano with Multimodality.
This means your phone can understand the world the way
you understand it, so not just through text input but also
through sights, sounds, and spoken language.
Let me give you an example.
2.2 billion people experience blindness or low vision.
So several years ago, we developed
TalkBack, an accessibility feature
that helps people navigate their phone through touch
and spoken feedback.
Helping with images is especially important.
In fact, my colleague Carol, who uses TalkBack,
will typically come across 90 unlabeled images per day.
Thankfully, TalkBack makes them accessible.
And now we're taking that to the next level
with the Multimodal capabilities of Gemini Nano.
So when someone sends Carol a photo,
she'll get a richer and clearer description of what's happening.
Or let's say Carol is shopping online for an outfit.
Now she can get a crystal clear description of the style
and cut to find the perfect look.
Running Gemini Nano on device helps minimize the latency.
And the model even works when there's no network connection.
These improvements to TalkBack are coming later this year.
Let me show you another example of what on device AI can unlock.
People lost more than $1 trillion to fraud last year.
And as scams continue to evolve across texts, phone calls,
and even videos, Android can help protect you
from the bad guys, no matter how they try to reach you.
So let's say I get rudely interrupted by an unknown caller
right in the middle of my presentation.
[PHONE RINGING]
Hello?
SPEAKER: Hey, I'm calling from Save More Bank security
department.
Am I speaking to Dave?
DAVE BURKE: Yeah, this is Dave-- kind
of in the middle of something.
SPEAKER: We detected some suspicious activity
on your account.
It appears someone is trying to make unauthorized charges.
DAVE BURKE: Yeah?
What kind of charges?
SPEAKER: I can't give you specifics over the phone,
but to protect your account, I'm going
to help you transfer your money to a secure account we've set up
for you.
[LAUGHTER]
DAVE BURKE: And look at this.
My phone gives me a warning that this call might be a scam.
[APPLAUSE]
Gemini Nano alerts me the second it
detects suspicious activity, like a bank asking me to move
my money to keep it safe.
And everything happens right on my phone.
So the audio processing stays completely
private to me and on my device.
We're currently testing this feature,
and we'll have more updates to share later this summer.
And we're really just scratching the surface
of the kinds of fast, private experiences
that on-device AI unlocks.
Later this year, Gemini will be able to more deeply understand
the content of the screen without any information leaving
your phone, thanks to the on-device model.
So remember that pickleball example earlier?
Gemini and Android will be able to automatically understand
the conversation and provide relevant suggestions, like where
to find pickleball clubs near me.
And this is a powerful concept that
will work across many apps on your phone.
In fact, later today at the developer keynote,
you'll hear about how we're empowering our developer
community with our latest AI models
and tools like Gemini Nano and Gemini in Android Studio.
Also, stay tuned tomorrow for our upcoming
Android 15 updates, which we can't wait to share with you.
As we said at the outset, we're reimagining Android with Gemini
at the core.
From your favorite apps to the OS itself,
we're bringing the power of AI to every aspect
of the smartphone experience.
And with that, let me hand over to Josh
to share more on our news for developers.
Thank you.
[APPLAUSE]