[cheerful music]
[cheers and applause]
Prabhakar: Thanks, Sundar.
People's quest for knowledge often starts with a search.
And over time, we worked hard to understand the trillions of questions
that people ask us every year
and to deliver the most helpful information possible.
But the way people search for information shouldn't be constrained
to typing keywords in a search box.
It's human nature to gain knowledge through multiple senses and inputs
as we go about our day.
For instance, if I hear a bird chirping outside the window,
I might point to it and ask my wife,
what kind of bird is that?
And while it's a huge challenge for computers
to understand information the way we do,
advances in technology are helping bridge the gap.
With AI technologies like natural language understanding and computer vision,
we're transforming Search to be far more natural and helpful than ever before.
Imagine the future where you can search any way and anywhere
and find helpful information about what you see, hear, and experience
in whatever way is most intuitive to you.
This is our vision for the future of Search.
And it's one we've already taken steps towards.
When Google started, our breakthrough was in understanding text-based queries.
Over time, people have asked us more complex and nuanced questions.
And that investment in natural language understanding
has significantly improved our ability to answer these,
even when the query feels like a brain teaser.
[laughter] I don't know this one either.
It's "Hachi: A Dog's Story." That's the answer.
But for many questions, it can be easier to speak than type,
which is why, over a decade ago, we introduced Voice Search.
We now get hundreds of millions of voice queries every day,
and it often is even higher amongst new Internet users.
In India for example, nearly 30% of Hindi queries are spoken.
But often, seeing is understanding.
So we reimagined Google Search yet again
with Google Lens to help you search what you see
using your camera right from your search bar.
Lens is now used over eight billion times a month,
which is nearly triple last year.
Now we're redefining Google Search yet again.
We're combining our understanding of information across multiple modes
to help you express your needs more naturally than ever before.
Just last month, we launched Multisearch,
one of our most significant updates to Search.
With the Google app, you can now search by taking a photo
and asking a question at the same time.
You can snap a pic of a spill-proof water bottle
and ask for one with rainbows on it to brighten your kid's day.
Or in my case, I was able to take a photo of my leaky faucet
and order the part to fix it.
The funny this is, I still don't know what the part is called.
[laughter]
But this is just the beginning of what we can do with Multisearch.
Later this year, we'll add a new way to search for local information
with Multisearch Near Me.
Just take a picture, or long press one you see online, and add "near me"
to find what you need from the millions of local businesses we serve on Google.
Near Me will work for a Multisearch for everything from apparel to home goods
to my personal favorite, food and local restaurants.
So let's say I spot a tasty-looking dish online.
I don't know what's in it or what it's called, but it's making me hungry.
With this new capability, I can quickly identify that it's japchae,
a Korean dish, find nearby restaurants that serve it,
and enjoy it in no time.
[applause]
While this all seems simple enough, here's what's happening under the hood.
Google's multi-modal understanding recognizes the visual intricacies
of the dish and combines it with an understanding of my intent,
that I'm looking for local restaurants that serve japchae.
It then scans millions of images and reviews posted on web pages
and from our active community of Maps contributors
to find results about nearby spots.
Multisearch Near Me will be available globally later this year in English
and will come to more languages over time.
[applause]
Today--today, this technology recognizes objects captured within a single frame.
But sometimes you might want information about a whole scene in front of you.
In the future, with an advancement we're calling Scene Exploration,
you'll be able to use Multisearch to pan your camera and ask a question
and instantly glean insights about multiple objects in a wider scene.
Let me give you an example.
Let's say you're trying to pick out the perfect candy bar for your friend
who's a bit of a chocolate connoisseur.
You know they like dark chocolate and have an aversion to nuts,
and of course you want to get them something good.
If you went to the store today to find the best nut-free dark chocolate,
you'd be standing in the aisle for a while.
You'd look at each bar, figure out which type of chocolate it is,
whether it's nut-free, compare and contrast the options,
and maybe even look up reviews online.
But thanks to Scene Exploration, you'll be able to scan the entire shelf
with your camera and see helpful insights overlaid in front of you.
[cheers and applause]
Yeah, insights overlaid so you can find precisely what you're looking for.
Try doing that with just keywords.
Here's how it works.
Scene Exploration uses computer vision to instantly collect
the multiple frames that make up the scene and identify all the objects within it.
Simultaneously, it taps into the richness
of the web and Google's Knowledge Graph
to surface the most helpful results,
in this case, which bars are nut-free, dark chocolate, and highly rated.
Scene Exploration is a powerful breakthrough in our device's ability
to understand the world the way we do.
And it gives us a superpower-- the ability to see relevant information
overlaid in the context of the world around us.
You could imagine the use in this in a pharmacy to find a scent-free moisturizer,
or at your local corner store to find a Black-owned wine label to support.
[cheers and applause]
This is like having a supercharged Control-F for the world around you.
[laughter]
Looking further out, this technology could be used beyond everyday needs
to help address societal challenges, like supporting conservationists
in identifying plant species that need protection
or helping disaster relief workers quickly sort through donations in times of need.
From Multisearch Near Me to Scene Exploration,
the advancements we've talked about today are in service of our broader vision
to make Search even more natural and helpful,
so you can search your whole world asking questions any way and anywhere.
To deliver on this promise, Google needs to serve
billions of people's diverse information needs,
and it's critical that they see themselves reflected in our products.
Building for everyone is a core value at Google.
You might recall at I/O last year, we announced Real Tone,
a multi-year initiative to build more equitable camera
and imagery experiences on Pixel 6.
[cheers and applause]
We are now expanding our commitment to skin tone equity across Google products.
We partnered with Howard professor, Dr. Ellis Monk, who spent the past decade
researching the impact of race, ethnicity, and skin tone on social inequality.
And his research and expertise are shaping
how we approach inclusivity in our products.
Before I--
[applause]
Before I hand it over to my colleague, Annie,
to tell you more about the work we're doing in this space,
let's hear from Dr. Monk himself.
person: Oftentimes, when it comes down to representation,
I'm amongst the darkest that you would find in media,
when I know that there are much darker folks, even of different races.
Ellis: Color biases and colorism are really a global phenomenon.
The reality is, life chances,
opportunities, all of these things were
very much tied to your phenotypical makeup.
person: It's facts. I feel like, ever since I was a kid,
if you were dark, you were not considered pretty.
Ellis: From a really young age, I did feel a really strong sense of responsibility
to dig more deeply into issues of colorism.
So for the last ten years, I've been working on issues of racial inequality,
and I developed a ten-point skin tone scale with the goal of making sure
that everyone across the skin tone continuum feels represented.
person: Do you feel like right now it's easy to find your skin tone?
person: When we type "beauty tutorial," if we only see white women,
it just ties into the bigger picture of not being represented.
Ellis: We can weed out these biases in our technology
from a really early stage and make sure that the technologies
that we have work equally well across all skin tones.
I think this is just a huge step forward.
person: Being represented is a right.
If the Internet truly is for everyone, then everyone should be represented.
Ellis: There is great potential for the tech industry to adopt the scale,
because the world is a better place
when technology works equally well for everyone.
[soft music]
[cheers and applause]
Annie: At Google, we build products for the world.
Billions of people with varied backgrounds and experiences rely
on our products every day.
As Dr. Monk mentioned, skin tone is one of many important dimensions
that shapes people's identity and experiences.
And we all deserve to feel seen and validated.
Today, we're excited to share how we're starting to use the Monk Skin Tone Scale
to build more inclusive products across Google.
Developed by Dr. Monk, the scale adds a critical step
to our product development and testing
to ensure the technology powering our features works well for everyone.
Our research has shown that more people in the US find the Monk Scale
to more accurately reflect their skin tone compared to the current industry standard.
This was especially true for people with darker skin tones.
We're testing the scale globally and in different product settings
and will continue to improve it to reflect people everywhere.
At Google, we've started using the Monk Scale to help improve
how we understand and represent skin tone in products like Photos and Search.
Every day, millions of people search the web expecting to see images
that match the diversity of the world around them.
We've started to roll out improvements to images in Google Search
to show a range of skin tone diversity so that people
from all kinds of backgrounds can find more relevant results.
And for makeup queries, like "everyday eye shadow"
and "bridal makeup looks," users will have a new way to filter
by relevant skin tones to find more helpful results.
I can't tell you the number of times that complimentary shades--
[cheers and applause]
I can't tell you the number of times complimentary shades
for my skin tone haven't been available.
And so I'm personally thrilled to be able to easily filter
for images that I can relate to.
Building on our work with Real Tone, we're also using the Monk Scale
to improve imagery experiences and products like Google Photos.
And later this month, we'll be launching new Real Tone filters
that were designed to work well across skin tones
and evaluated using the Monk Scale.
These filters were crafted by a diverse range of renowned image makers
who are celebrated for beautiful and accurate depictions of their subjects.
With Real Tone filters, you can apply the beauty and authenticity
of professional editing to your own photos with just a few taps.
[cheers and applause]
Building more inclusive experiences is a long-term commitment,
one that involves close collaboration with brands, publishers, and researchers.
And that's why we're pleased to announce that today,
we are open-sourcing the Monk Skin Tone Scale
so anyone can use it as a more representative skin tone guide...
[cheers and applause]
In research and in product development.
By open-sourcing the scale, our goal is to improve it over time
in partnership with the industry.
In the coming months, we'll also be developing a standardized way
to label content for images on Google Search.
Creators, brands, and publishers will be able to use an inclusive schema
to label images with attributes like skin tone, hair color, and hair texture.
All of this is part of our ongoing commitment
to ensuring that the web is as representative as our world.