[MUSIC PLAYING]
DALE MARKOWITZ: Hi.
Welcome to From Research to Ready,
where we're going to talk about the connection between Google
research and Google Cloud products.
I'm Dale Markowitz.
I'm an applied AI engineer here at Google Cloud.
And I'm joined by Nikita.
NIKITA NAMJOSHI: Hi, everyone.
I'm Nikita Namjoshi.
And I'm a developer advocate at Google Cloud.
DALE MARKOWITZ: So Google research powers
all different parts of Google--
Google products like Gmail, Google Search, Lens, Translate.
And it also powers tons of different Google Cloud tools.
Like for example, you might be familiar with BigQuery,
Google's super fast data store.
Well, BigQuery actually started as a result from research
in an internal version called Dremel,
or you might be familiar with TensorFlow,
which is one of the most popular open source tools
for building machine learning models.
Well, TensorFlow also started out of Google research,
and then it became an open source tool,
and now is used in all sorts of Google Cloud machine learning
pipelines.
Today we're going to talk about two recent Google Cloud
offerings that come directly out of research.
So the first one has to do with vector search.
Now nowadays in machine learning,
the technique of vector search is really popular.
In fact, if you saw the demo at the beginning of our Applied AI
Summit, you saw Nikita and Zack walk you
through a demo that used vector search
to do question answering.
So the idea works like this.
Let's say that you've got a database of a million news
articles, or maybe like, in the case of the demo,
Q&A answers, or maybe even Google web pages.
And you want to be able to search through all those text
documents and supply a result that's relevant.
Well, a simple way to do this is with keyword matching.
But keyword matching is a bit simplistic.
It's not exactly what you want.
What you really want to build for text search
is something called semantic search.
So the idea of semantic search is to take a piece of text
and find other similar pieces of text
based not on overlapping words but
on similar semantic meaning.
By the way, this is the problem more or less of Google Search.
When you go into google.com and you
type into a text box a question, we
try to reply with a semantically relevant web page as a search
result. And the same thing when you, for example, go on YouTube
and you type something, what's a semantically relevant video.
For many, many years this type of problem
was extremely difficult for computers to solve.
But in the past decade, it's become solvable thanks
to a technique called vector embeddings.
Actually, one of the first groups
to really work on this problem in terms of text embedding
was Google research back in 2013,
we published a result called word2vec,
which was a way of embedding individual words as vectors.
If you do any work in NLP, you probably
have encountered word2vec at some point.
So this is what word2vec looks like.
The idea is that we take words and we map them
to points in space so that semantically
similar, synonymous words are mapped closely
to the same areas in space.
And word2vec enabled tons of different NLP apps.
But nowadays we can do even better
than embedding into the words.
We can actually embed whole sentences
or paragraphs or blocks of text or articles
with something called a universal sentence encoder.
So this is another machine learning model also published
by Google Research in 2018.
And it's made lots of text search applications way easier.
In fact, this is one of the most popular TensorFlow models
that you can download.
So for example, if you wanted to build this,
well, here's one news article and I
want to find another one that's like it,
you can embed the text using the universal sentence encoder
module.
Now often when we build awesome models here in Google research,
we make them publicly available in open source
through something like TensorFlow Hub.
So you can take this universal-sentence-encoder,
download it from hub, and use it in your own apps.
So the way that we do text similarity search
is with text embeddings but nowadays
we're embedding all sorts of different types of data.
For example, with image embeddings,
we can do something like reverse image search.
So maybe I take a picture of my shoes
and then I say find me a shoe that looks similar.
We can also embed users and customers.
For example, maybe we want to find a cluster of customers
with similar buying habits or maybe we
want to do a multimodal embedding
where we embed like an image and a text
and we see if they're describing the same thing.
Now oftentimes we think a lot about these cool embedding
models because they're very exciting,
but we often don't pay attention to the more practical aspect
of if we're going to do finding similar vectors,
how do we actually run the calculation
to find what vectors are similar?
Usually, we use a distance metric, something
like cosine similarity or dot product,
but these are mathematical operations
that can be really computationally
costly, especially if you have to do them on a really, really
large vector data set.
So the question is, given a query vector,
how do we find its nearest neighbors as quickly
and efficiently as possible?
We solve this with vector similarity search techniques
like k nearest neighbors or approximate nearest neighbor
search.
And it's one of the most critical steps
in neural deep retrieval architectures.
OK.
So I want to talk about Matching Engine, which
is a new product out of Google Cloud
that's designed to allow you to do this efficiently.
So it allows you to do vector search
with low latency and high scalable and really,
really quickly.
So the way that this is possible is
thanks to a result out of Google research.
In 2020, we made significant headway
in solving this problem of doing vector search really
quickly thanks to a new algorithm called Scan.
So again, the idea is to speed up
the calculation of the distance between vectors.
And whenever we want to do this at scale for millions
or even billions of vectors, we have
to be able to approximate that calculation in some way.
And so one of the most popular [INAUDIBLE] to do this
is called quantization where we take millions of vectors.
Then we map them to a smaller vector data set
that we compute distances on.
So the Scan algorithm that came out of Google research
found a more efficient way of creating this smaller
approximation set so that when you do a vector
search on this quantized set, you
get both more accurate results and results more efficiently.
So we used this new algorithm to build an internal version
of Matching Engine that actually powers tons of different Google
products like Google Search and YouTube.
And now through Matching Engine, we've made it available to you.
So when you want to be able to search through billions
of vectors, you can also do it quickly, efficiently
using this algorithm.
So that's Matching Engine.
We're very proud of that.
And now I'm going to hand it over
to Nikita who's going to talk about another Google
product that was built in strong collaboration with research.
Take it away, Nikita.
NIKITA NAMJOSHI: Thanks so much, Dale.
So when you start working on a new ML problem,
it's not uncommon to use the latest
architectures from research as your starting point.
But have you ever used models like NasNet, or AmoebaNet,
or SpyNet?
It might surprise you to learn that these state of the art
model architectures weren't hand designed.
They were actually created by a technique called
neural architecture search.
Neural architecture search or NAS
is a technique for automating the design of neural networks.
In a typical NAS setup, you have a controller
which proposes models that are sampled from a search
space, which is the range of architectures
that can be represented.
Then it trains and evaluates the models.
And it iterates thousands of times
to find the model that best meets the objective.
How did this all start?
Well, in 2016, Google researchers
published a paper called "Neural architecture search
with reinforcement learning."
In this paper, they showed how to use a recurrent network
to generate model descriptions of neural networks.
They trained this RNN with reinforcement learning
to maximize the expected accuracy
of the generated architecture's autovalidation set.
Using this method, they were able to design a novel
architecture for the CIFAR-10 data set that rivaled the best
human-invented architecture in terms of tests that accuracy.
Since that paper, the field has continued to grow,
including nonreinforcement learning-based techniques
and the introduction of PyGlove, which
is the symbolic language that can
be used to help convert your ML code into a NAS program.
In addition to research, we started
to see adoption of neural architecture search
across Google and other Alphabet teams.
In 2018, the team at Waymo used NAS to help discover models
with significantly lower latency but results
of the same quality.
And many of the architectures found during the search
showed creative combinations of convolutions, pooling,
and deconvolution operations in ways
that might have gone undiscovered
with hand tuning alone.
The team behind Pixel 6 used NAS to automate
the process of designing models, incentivizing the search
algorithms to discover networks that
achieved higher quality while meeting latency and power
requirements.
The latency and power requirements
were really crucial because these models were
being run on the Pixel phones.
For face detection, they discovered a model
that achieved the same accuracy as an optimized baseline model
but consuming less than 70% of the energy.
And NAS doesn't just work for image use cases.
When it comes to NLP, the Pixel 6 team
used NAS to discover language models with up to two times
higher hardware utilization, allowing
for larger and more accurate models
compared to their baseline.
So what does all this have to do with you as a Google Cloud
user?
Well, Vertex AI NAS is a managed service
for performing neural architecture search.
What started out as an experimental research project
is now a GA product.
Vertex NAS works in two stages.
The first stage uses a smaller representation of full training
called a proxy task.
In stage two, a full training occurs using the top scoring
models from the first stage.
You can use the Cloud Console UI to monitor and manage
jobs and kickstart the search from a notebook.
With Vertex AI NAS, you can optimize models
for accuracy, latency, and memory all in the same trial.
And what I think is really exciting is that now instead
of waiting for the next state of the art architecture
to be published in a paper, you can actually
be the one to discover that architecture for yourself.
Today we talked about the critical role research
has played in both Vertex AI Matching Engine and Vertex AI
Neural Architecture Search.
We hope this gives you an idea of how Google research has
deep product impact and how you can
take advantage of this cutting edge research as a Vertex AI
user.
Thank you so much for joining us.
DALE MARKOWITZ: Thanks so much, Nikita.
And enjoy the rest of the Applied ML Summit.
[MUSIC PLAYING]