Let's now talk about filtering.
As I mentioned before, adding filtering function in vector databases is actually quite hard. And
different vector databases also implement this differently as well. There are largely
three categories of filtering strategies: you can either do filtering post-query, in-query,
or pre-query. But there are also vector databases, which implement their own proprietary filtering
algorithms, grounded in one of these as well. So take the example of this query, I can start out
by searching for only shoes, which gives me the broader selection choices possible. And then I can
make my queries more and more precise by adding adjectives like "black", "black shoes" in the
query. But maybe I finally decide I want to buy only Nike shoes. So we can view this brand "Nike"
as a brand-specific callout in the metadata. And in fact, for many online retail websites,
we can indeed filter by this metadata, for example, by brands, by styles, by price and
etc. So a question for you is, would you implement the brand filter prior to the query? What if you
don't actually have any Nike shoes in your store? Should you then return no shoes in your results?
Let's go through the nuances of each filtering type in the following slides.
The first is post-query filtering.
Say that now you are trying to find the best Pixar movie that's similar to Frozen. After we identify
the top-K nearest neighbors ordered results, we can then apply the Pixar Studio filter. The upside
here is that we get to leverage the speed of ANN but the downside is that the number of results
is highly unpredictable and maybe there's no such movie that meets your requirements.
The second type of filtering is in-query. This is quite interesting because the algorithm does both
ANN and filtering at the same time. For instance, when you search the movie again that is similar
to Frozen but produced by Pixar, all the movie data will have to be converted into vectors.
But in the meantime, the studio information is also stored in the system as a scalar field.
So during search, both vector similarity and metadata information will need to be computed.
This can put a rather high demand on the system memory because it needs to load both the vector
data and the scalar data for filtering and vector search. So perhaps when you shop online and when
you have used a lot of filters at once, you might realize that when you add more filters,
the website sometimes may take more time to return you the results. So if the memory of the system is
limited, as you increase the number of filters, you might actually hit out-of-memory issues,
abbreviated as OOM, in this slide. But this approach is actually quite suitable for row-based
data because, in a row storage format, you need to read in all columns in a row at once; as opposed
to for columnar storage format that allows you to read in subsets of columns of data at a time.
The last type is pre-query filtering. This limits similarity search within a certain scope.
What vectors can I actually even consider, you know, after I apply the filter? The downside
of this approach is that it doesn't leverage the speed of ANN and all data will have to be filtered
in a brute-force manner. So this is often not as performant as the post-query method or the
in-query filtering method because the other two methods could easily leverage the speed of ANN.
So this wraps up the discussion about filtering strategies. In the next segment,
we'll talk a little bit more about vector stores.