[MUSIC PLAYING]
SPEAKER: The rapid growth of unstructured data
like text, images, and video offers hidden insights.
However, its sheer volume and complexity pose a challenge
in extracting its value.
Traditional keyword searches struggle,
leaving important information and connections undiscovered.
This is where vector search offers a breakthrough.
Vector search uses embeddings, which
are numerical representations of your data.
Machine learning models transform text, images,
and even video into embeddings, allowing
you to compare and search them based
on their underlying meaning, not just literal keywords.
This is because semantically similar text, images, and video
have similar numerical representations in embeddings.
Imagine finding images similar to a product photo,
identifying video clips related to a customer review,
or finding text documents that all express a similar sentiment.
Vector search makes this possible.
And what if you could do all of this
without leaving your data warehouse?
Well, BigQuery makes it a reality.
In this video, we'll explore BigQuery's functionality
that enables you to generate multimodal embeddings
and perform vector search.
Let's start by understanding how BigQuery handles
not only structured data but all types of data, including
text, images, audio, and video.
BigQuery is a powerful and flexible tool
for bringing together all of your data assets.
Using object tables, BigQuery lets you directly
connect to and analyze unstructured objects
in cloud object stores.
You can combine insights from your unstructured data
with your structured tables and unlock new analysis
opportunities.
To understand complex data like freeform text or images,
you need a standard way to represent them,
and embeddings make this possible.
Machine learning models take your unstructured inputs,
like images, and transform them into numerical vectors.
These vector embeddings capture the underlying meaning
of your data, making it possible to compare and search
based on semantic relationships, not just surface-level keywords.
BigQuery machine learning lets you generate embeddings
for your unstructured data directly
from your data warehouse using familiar SQL syntax.
It also supports multiple embedding models
to suit your use case.
You might choose a text embedding
model for all of your textual data
or a multimodal embedding model for semantic search
across modalities, like text, image, and video.
You simply invoke the ml.generate_embedding function
over an object table, and embeddings are returned directly
to BigQuery.
With your embeddings stored in BigQuery,
you can index them and use the vector search function
to find data based on semantic similarity.
This means that regardless of the input modality,
you return relevant results based on the actual meaning.
BigQuery vector search is optimized for analytical use
cases and can efficiently process large quantities of data
and handles the infrastructure management for you.
Previously, these types of analyzes
required complex pipelines and external tools,
which created silos and friction.
BigQuery brings it all together, embedding generation, vector
search, and your existing data, enhancing your analysis
experience.
Let's head over to the cloud console and see how this works.
Imagine I work at an e-commerce company in the fashion
retail space.
I'm thinking of introducing a new sweater to our collection.
But, first, I want to check whether we have produced
or sold anything similar.
I have sales data in BigQuery tables,
and I also have a vast catalog of product images
in cloud storage.
But, unfortunately, none of these images
are tagged or indexed.
We can see a sample image from our catalog
here in cloud storage.
I'm going to then navigate over to BigQuery.
I begin by creating a cloud resource connection.
This allows BigQuery to access our images
in cloud storage and the embedding models in Vertex AI.
To add a connection, I'll click Add Connection
to external source, connection type as Vertex AI remote model,
and then I'll give it the name my_conn,
and click Create Connection.
I can then navigate to the cloud resource connection
and copy the service account ID.
I'll need to give the service account a few more permissions.
So I'll open up IAM in another tab.
I'll assign the service account two roles, storage object
viewer and Vertex AI user.
Then I'll move back to BigQuery.
With permissions all taken care of,
I'll now create a remote model so
that I can access the embedding model through BigQuery.
The Create Model statement takes a few inputs--
a model name, a remote connection, which I just
defined, and an endpoint.
In this case, I'll use both text and images as inputs,
so I'll specify the multimodal embedding model.
This remote model setup is one time for the embedding model,
and I can continue to use it in this project.
The last setup step to generate embeddings
is to create an object table.
This will give me a structured interface
to my objects in cloud storage.
For this example, the object table
will display metadata around the product catalog images.
Object tables require a few inputs.
These include a table name, a cloud resource connection,
and the Google Cloud Storage URI where the product catalog
images reside.
Note that object tables only store metadata,
and that there is no movement from cloud
storage to another location.
With our setup in place, I can now use
the ml.generate_embedding function.
The function takes two inputs.
The first is a remote embedding model
that I defined that takes multimodal inputs.
The second is an object table that references
objects in cloud storage.
Then I'll run the query.
Behind the scenes, BigQuery is submitting
all of the images in the object table
to the multimodal embeddings API hosted in Vertex AI.
It's then returning the results and storing them
in a BigQuery table.
Once the query finishes running, I can check out the new table.
Here we see a number of fields.
The first is a field containing a vector embedding.
The next field displays the status in case of an error.
And the function also returns all
of the fields in the object table associated
with each image.
So instead of spending time outside of BigQuery
to generate these embeddings and load them in later,
I've instead generated everything
without leaving our data warehouse.
With embeddings in a BigQuery table and ready to use,
let's now turn to vector search.
I'll begin by creating a vector index on top of my table.
This will enable faster and more scalable
semantic search by using an approximate nearest neighbor
search algorithm.
I need to give an index name, specify the BigQuery
table it refers to, and define the embeddings column
within the table.
Lastly, I can input a number of options.
In this example, I'll specify that the index uses
cosine distance, and you can find other options
available in the documentation.
BigQuery will create a vector index for me,
and I can immediately run vector searches.
But full indexing may take some time
depending on the size of the embeddings table.
I can check that progress using the information schema views.
Note that vector indexes are fully managed by BigQuery
and are automatically refreshed when the indexed table changes.
Now I can run that semantic search.
The following query does two things.
It first takes a text input and uses the ml.degenerate_embedding
function to generate an embedding using our remote
multimodal model.
It then takes the embedding and compares it
to all of the indexed embeddings.
I've specified one optional argument so that I only
return the top five results.
Again, I'll be searching my product image catalog
of clothing images to see whether we've ever sold
similar products in the past.
I'll input the string, comfy rainbow sweater,
and check out the results.
I return the top five matches and the cloud storage URIs.
Now we'll run the same query using a BigQuery Studio
notebook and a helper function that renders the images.
And it looks like we've sold similar sweaters in the past.
For my next steps, I might want to look
at the sales data for these sweaters
to see which were the best sellers,
or maybe look at how many were returned
to us and for what reason.
I might also consider summarizing the customer reviews
for these items to get some inputs for our new product.
There's a wide array of possibilities
on how to proceed with further analysis.
Traditionally, working with embeddings and vector search
required deep knowledge of specialized libraries, managing
complex pipelines, and overseeing
additional infrastructure.
BigQuery dramatically simplifies this process.
With embedding generation and vector search built
in, and with the familiar power of SQL,
you can unlock new insights without needing
to leave your data warehouse.
With embedding generation and vector search in BigQuery,
you can expand the scope of your analysis.
Examples include building content-based recommendation
systems, content classification and outlier detection,
and even retrieval augmented generation
with BigQuery's LangChain integration.
[MUSIC PLAYING]
Give it a try.
Check out the description below for a step-by-step guide
and resources to get you started.