Brian: Let's talk about how Tensor is making typing much faster on Pixel.
Obviously, speaking is the fastest and most natural way to go,
but today's voice inputs systems just haven't been up to the task.
First, understanding the nuance of human speech
is really a hard problem with challenges that are unique to each person.
It means understanding syntax, intent, and the context of your request;
knowing the terms and names that are important to you
but potentially uncommon to others;
Understanding your accent and dialect,
isolating your voice when there's background noise;
and even hearing you correctly if your mouth is full.
Google has led the industry for years
with our natural language processing.
It's why Search queries are so accurate, and Google Translate is so good.
Thanks to some outstanding work from our speech team,
Pixel 6 has an on-device speech recognition model
that can transcribe speech with incredible accuracy.
And thanks to Tensor,
it does it using half as much power as previously possible!
It's the most advanced Speech Recognition model
ever released by Google.
With this new capability, we re-thought the voice typing experience
based on some key insights about how people write.
Let me show you how it works:
Hey, Rani, are you still up for dinner tomorrow with our friends?
You can see how accurate the transcription of my friend's name is.
Pixel is smart enough to take the hint from my contacts list,
and the mic stays open while I tap to insert the clarification
about who's coming.
Pixel even automatically adds the right punctuation,
so I don't need to specify where to put commas and question marks.
Let's keep going.
I hope you can come. Katherine is hosting this week.
Of course, Pixel won't always know which Catherine I'm talking about,
and the one I have in mind spells her name with a K.
The language model also helps with transcription suggestions,
so corrections make sense based on what you're saying.
Historically, transcription suggestions have been the same ones
designed for typing, based on keystroke proximity.
But that doesn't make any sense if you aren't typing with keystrokes!
Now Pixel's model is phonetically based,
meaning it suggests corrections that sound similar.
So Catherine with a C becomes Katherine with a K,
instead of "catharsis" or "cathedral"
or whatever else you might have ended up with.
And to make voice typing as fast as possible,
I should be able to do more with my voice,
including commands like "clear" or "send."
Pixel is smart enough to understand
that I don't want those words in my message.
I'm sure it's gonna be great. Clear. I'm looking forward to it.
Katherine has a really nice new place.
Send.
See how fast that was?
Also, notice that your phone remembered it's Katherine with a K,
so you won't have to make the same edit twice.
The model adapts to your usage.
And watch this:
The model also unlocks the accuracy of emoji transcriptions,
which is so much faster than searching through little pages of icons.
I think we're making homemade pasta emoji and ice cream emoji.
Send.
With the new speech model,
typing with your voice is faster and easier
than just typing with your fingers.
Thanks to Tensor and these new breakthroughs,
we can do all this on-device without connecting to a server,
so it's super fast and reliable.
We really think it's gonna change how you use your phone.
Google's natural language understanding allows us to rethink phone calls too.
Call Screen is great for shielding you from spammers.
Unknown callers have to say why they're calling
before you decide to pick up or hang up.
It's a useful feature, and the Tensor speech models
are making Call Screen even more accurate in Pixel 6.
The next problem we wanted to solve is calling a business.
It's been the same bad experience for decades, with hold music,
the same long automated system saying "your call is very important."
It's the worst, but Pixel 6 makes it so much better for you.
Now, before you even place your call,
Pixel shows you the current and projected wait times,
so you can call when it works for you.
And when you do call and encounter that endless list of options
like "press 1 for branch hours and locations",
you actually don't need to listen carefully
and remember all the automated menu options.
Google Assistant listens for the details
and shows them on-screen for you to tap.
Of course, if you still find yourself in the hold queue, don't worry.
Google Assistant will hang out on the line
and listen to the hold music for you.
It understands the difference between a recorded message
and an actual representative on the line,
and lets you know when a real live human is ready to talk.
Tensor also dramatically improves translation features in Pixel
so you can interact naturally with people
who don't speak the same language that you do.
Pixel can help with smart, simple translation
that's there wherever you need it.
Just like with voice typing, speed and accuracy are best-in-class.
Until now, these kinds of breakthroughs haven't been able to run on a phone.
They require intensive computation on highly specialized hardware--
this is exactly what Tensor is good at--
it improves Pixel's translation quality by 18%,
a level of improvement that typically takes multiple years of research
from our world-class research team.
AND the improved model uses less than half of the power
when running on Tensor.
We all have friends or family members
whose native language isn't the same as ours,
and I've always wished I can speak with them
in the language they're most comfortable in.
With Pixel 6, I can finally do that.
My sister-in-law is Japanese, and here's a message from her.
It says, "I have a question about the latest Pixel."
Now I can reply in English, and it will show up in Japanese for her.
"Sure, now I can finally tell you about it!"
It all happens right on the device in the Messages app,
so I don't have to deal with cutting and pasting text into Google Translate.
In-app translation is also available in WhatsApp, Google Chat,
Snapchat, Instagram, Twitter, Line, and a growing list of other chat apps.
Now, if we combine our real time translation models
with our speech recognition models,
we can translate audio from any source, in real time,
right from the device itself.
It opens up a world of content for people.
Watch YouTube videos in your own language.
Stream a sports broadcast from another country.
Watch Instagram live videos from around the world.
All translated and captioned in real time for you,
right on your Pixel 6.
Translation is even integrated into the Pixel Camera,
so you can read signs, product labels,
documents, and even menus in your own language.
Across all the speech and translation features in Pixel 6,
you can see a few of the ways Tensor unlocks new capabilities.
Google's ML research enables things like helpful voice typing,
calling assistance, and translation on Pixel.
And we've got one last translation feature to show you today.
The improved Interpreter Mode in Pixel 6 is more fluent,
faster than ever, and available in 48 languages.
Here's my colleague Shenaz with Marie Kondo to try it out.
Shenaz: Marie Kondo is famous for changing people's lives
through her KonMari method of organization.
She has a television show, has written books,
and is a huge star in the United States.
Thank you very much for being our first Live Translate interviewee.
I love seeing what sparks joy in other peoples' lives.
What sparks joy for Marie?
AI voice: For me, it is a pleasure for as many people as possible
to finish cleaning up and lead a thrilling life.
I'm surprised that it's translated so perfectly!
Shenaz: What is it like working in a place
where people don't speak the language?
AI voice: Speaking English
is certainly one of the difficult things for me.
But it's just fun to be able to join people with different values,
different languages, and cultures.
Shenaz: It was so nice to talk and connect with you in this way.
Thank you.
AI voice: To be honest, I'm really surprised that my words
have been translated and conveyed so well.
Shenaz: We are so happy to hear your excitement
because we worked very hard on this.
AI voice: I think that the language barrier
is getting smaller now.