Hey everyone, I'm Jeff Hollan, Product Manager here at Snowflake.
I am thrilled to show you an amazing demo
of using an LLM directly in your Snowflake account powered by
NVIDIA Accelerated Snowpark Container Services.
So to do this, we're going to start by
finding an LLM directly from the Snowflake Marketplace to help with quality control.
So quality control can be difficult because we have a variety of different quality issues
that might arise, and I'm going to need an app that can reason to actually interact with our
quality supervisors. This is a great fit for a large language model, or LLM. So using the
Snowflake Native App Framework and Snowpark Container Services accelerated by NVIDIA AI,
third parties like Reka can now package their leading LLMs into a Snowflake Native App.
This lets me install it in my Snowflake account in a way that is secure for both of us. So for Reka,
when I install this, their LLM weights and proprietary IP aren't exposed to me, the app consumer.
And because the LLM is running in my account, my governed enterprise data used for
fine tuning or interacting with the LLM, is never exposed back to Reka. I have full control over
what data I use with the LLM. It's a win-win for both of us. Now, all I need to do to get access
from the Snowflake Marketplace, is go ahead and install this application. And once it's installed,
I can go ahead and use the interface that comes bundled with this app, built with Streamlit
in Snowflake. This will help product quality supervisors to interact with Reka's Multi-modal
AI Assistant that has the ability to generate responses based on both visual or text inputs.
So let's try this LLM without any fine-tuning and see what questions I can ask out of the box.
So to start, I've gone ahead and given secure access to a stage filled with images in my Snowflake account.
I've gone ahead and selected one of those images here, and let's ask some information without
this being fine-tuned. So first off, what is this an image of?
Great, and you can see this accurately identified
this was in fact an image of snow goggles, which is actually tricky,
because you could have thought that that might have been a virtual reality headset.
But we'll go ahead now and try something else.
Are there any quality problems with these goggles?
Yeah and it's identified here from the image.
There are some scratches on the lens. But let's try something
even more specific. Can you give me the model name or number? Now you'll see here it's responded it
doesn't have that information, which makes sense because it's not familiar with the nomenclature
that we use for our products. But I can enhance the experience by fine-tuning the model with my
data in Snowflake. So I'm going to go ahead now and switch to a version of this model which was
fine-tuned and try it again with the same image and let's see what information we can now get.
So we'll start again let's ask it about the damage on the goggles and you'll see here it's identified
scratches in multiple locations. But now I want to get some of that specific information. So I'm
going to give it to access more data that it has about these goggles. Awesome, you'll see in this
case now it actually knows about the specific model of these goggles. And this is really
useful for the quality manager to quickly follow up with the appropriate production line among the
hundreds of products they manufacture.
Alright, let's recap some of the stuff that we just saw here.
I was able to find and install a third-party LLM to run entirely within my Snowflake account
powered by the Snowflake Native Apps Framework and the Snowflake Marketplace. I could then fine-tune
that data and access it powered by the NVIDIA Accelerated Snowpark Container Services. Now if
you are interested to learn anything more about some of the amazing tech that we saw here today,
be sure to check out the links in the description of this video and subscribe to this channel
to stay up to date with the latest news and announcements from Snowflake.
Thank you so much.