We've all encountered powerful copilots like Bing Chat, Office 365
and GitHub built on Azure.
These are some of the world's most complex workloads.
Today I'll demonstrate how you can build your own retail
copilot safely using Azure AI Studio.
The first thing that I will do as a part of building my
application is creating a project.
Through this, I create an Azure AI resource.
The Azure AI resource helps me connect to create and later, as
you'll see, manage all the AI related assets that go into my
project.
These include resources like Azure Machine Learning, Azure
Open AI, Azure Speech, and Azure Search.
This minimizes set of challenges, enabling users to
start building quickly and seamlessly.
I've created a project.
In the Settings tab, you see all of the different connections
within that project, the project members that have access to it,
how resources are configured, including Azure Open AI, and
within the Deployments tab, we have all of the different models
that are deployed within your Azure Open AI resource.
They're ready for me to use.
If I need to add additional deployments, I can create those
here and use them within my project.
I'm going to go to the project playground now.
This is a great place to get started.
As you can see, I'm using a GPT 35 Turbo model.
Initially, when I asked a specific question about my data,
the model doesn't know because it hasn't been grounded in my
data yet.
As expected, it says that I do not have access to the pricing
information.
So let's go ahead and fix this.
Now I'm going to ground this model with my data using Azure
AI Search.
By ingesting our data from 1 Lake and using a vector index
through Azure Search, we've solved this problem.
As you can see, this time when I asked the question it shows the
answer.
It also references which product document it's retrieving that
information from.
Not only have we grounded this model, but we've also enabled
multilingual responses.
One link is supported in the runtime and therefore it's fully
compatible across AI Studio.
We can use it in the playground as you see here, as a source for
fine tuning and even within my orchestration flow as you'll see
later.
Now that I understand how the model works, let's take a look
at some of the other tools that we have within the playground.
As we can see here, my system message or prompt is basic.
The quality and relevance of the response generated by the LLM is
heavily dependent on the quality of the prompt, so prompts play a
critical role in customizing LLMS to ensure that the
responses of the model meet the requirements of a custom use
case, I can use the prompt catalogue to select from prompts
across different industries.
So far in the playground, I've been having a one-on-one
conversation with this model for me to understand more about the
behaviour of my application grounded with my data.
I want to be able to ask several questions and compare the
results.
For that we can use our manual evaluation tool.
Both the system message and my vector index have been carried
over.
Let's change this to the GPT 35 turbo model.
I can input questions directly in the tool.
Optionally, I can add expected responses to compare with the
output of the LLM.
What could be quicker is if you import an entire data set.
As you'll see, I've already gone ahead and done that.
I had a data set that had around 10 questions along with expected
responses that I know to be accurate.
I'm going to run all of them as a manual evaluation.
After the test runs, which is pretty fast, I can quickly
review all of the outputs in conjunction with my data set and
it's expected results.
Now let's analyse these responses.
Many of them appear to be highly accurate.
For those that meet the mark, I can express my preference, but
if a response falls short, I have the option to flag it for
further attention.
The effort invested in this process doesn't go to waste.
I can easily export these results for sharing purposes.
Additionally, I can preserve this data set, utilizing it
later in my project for a more comprehensive evaluation as we
will see later.
Now that you have an idea of how the playground works, I want to
go back and talk a little bit more about Azure AI Search.
This is what we use to ground our model with.
If you recall, under the covers, the playground runs a
sophisticated ingestion pipeline.
This pipeline cracks open the documents.
It runs a chunking strategy.
It then embeds the data using an LLM and inserts that, embedding
into Azure AI Search.
Next, we're going to look at the different search types and the
impact that they will have on your application.
Now that you have an understanding of how we grounded
the model, let's talk a little bit about the different search
types that are available through Azure AI Search.
First, we have Vector Search.
Vector Search is great at understanding the relationships
between words.
For example, it can understand the relationship between tent
and camping.
However, it doesn't do a good job with exact words.
For example, if you're using phone numbers, e-mail addresses
or product names.
For that, good old keyword search works much better and
it's faster.
With hybrid search you have the ability to use both keyword when
users want specific information and vector search when you want
to better understand user intent.
Last, we have semantic search.
This is the search that rules them all.
It builds on top of hybrid by adding a ranker.
This ranker makes sure that the best information gets surfaced
to the top of our search.
So let's go ahead and try the hybrid plus semantic option.
I've selected it.
Now my model is grounded with my data.
I'm going to ask this a question about which hiking shoes should
I get?
Initially.
It's telling me that I cannot provide a specific
recommendation.
However, since we had the Hybrid Plus Semantic Ranker enabled it,
it does pull up the Trail Walker Hiking shoes at the top of the
list.
And not only does it do that, it also gives me a summary of why
those shoes could be a good fit for me.
If I hadn't enabled that, I would have stopped here and not
been able to provide my user recommendation for the Trail
Walker hiking shoes and why they would be a good fit.
Now that you've seen an example of grounding a model with your
data and the different search capabilities, the reality is a
lot more complex.
Customers want a way to be able to add many data sources.
They want to use other models along Azure Open AI models.
They want to test different variations of meta prompts and
models together.
AI Studio offers you direct access to the orchestration
engine that fuels this UI.
With prompt flow, the data sources and APIs created by the
wizard are represented as nodes within a flow.
The flow easily translates to code that developers can
seamlessly integrate with.
If I ask the same question I will get the same answer.
Developers face challenges with LLMS due to their non
deterministic nature.
Minor changes in meta prompts can yield very different
results.
To tackle this, developers require evaluation tools.
I'll be testing 2 meta prompts.
One is simple and the other one has instructions to promote
safer behaviour.
I'm going to use the same data set that we created earlier.
It includes a set of questions and answers that I want to
evaluate my flow against.
AI Studio gives me a comprehensive list of metrics
that help me evaluate the impact of changes on my application.
The platform provides A diverse set of pre built traditional
machine learning and AI assisted metrics for measuring answer
quality and safety.
These can include relevance, coherence, fluency, similarity,
and groundedness.
With a few clicks I've generated metrics to evaluate each variant
of the meta prompt.
Variant zero was the basic meta prompt and if you recall variant
one was the safer meta prompt.
In general it looks like the safer meta prompt has slightly
better metrics.
If I wanted it deeper into the metrics for this variant, we can
do that too through the instance view.
Here I can see a set of metrics along with the information that
I need to pinpoint where the gaps are in my application for
the safer variant.
If I need more control over how these metrics are calculated,
the evaluation process powered by LLMS operates as a prompt
flow.
This not only gives me transparency, but also control
over defining my own metrics.
Let's look a bit more into deployment.
With a few clicks, I can deploy this to an endpoint and
integrate it into my application.
I can also monitor my application's performance and
safety using the same metrics as I did while building my
application.
Through Azure AI Content Safety, I can apply content filters to
my deployed application.
This enables me to create custom thresholds to filter out harmful
content from both the user input as well as the output of my
application.
This gives me Peace of Mind when my application is deployed to my
customers.
Let's say you want to use other models within your application.
Imagine having the flexibility to choose from thousands of open
source models within your application.
For example, you can use meta 70 billion parameter Llama 2 model.
However, if you don't know which model to pick, we've got you
covered.
With our benchmarking tool, you can choose your task with the
set of models that you want to compare, and you'll be able to
benchmark different models against the standard data set in
the future.
You'll also be able to replace that data set with your own data
set and benchmark those models.
Now that you know how to choose a model, let's talk about
customizing that model for your scenario.
With AI Studio we're introducing the Model as a Service.
You can easily fine tune and deploy Lama and Azure Open AI
models with your training data and an API call while we manage
the quota for you.
Tailor made AI solutions have never been this accessible.
Having seen a complete text in text out scenario, let's take a
look at what the future of generative AI unfolds, which is
multi Modality.
Multi Modality enables me to craft dynamic applications
incorporating images, text, speech and even videos.
We're going to look at several scenarios.
In the first one, I can generate descriptions for my products.
I can then build on top of that by also generating images and
various outdoor conditions for my hiking shoes.
This is where it gets really exciting.
Now let's go hiking.
Thanks to the cutting edge GPT 4 remodel, we've taken
personalization to new heights, seamlessly integrating video
content.
This technology empowers my application to analyse and
curate personalized product suggestions and create outdoor
itineraries.
Witness the magic as an enhanced GPT 4 way model precisely
pinpoints the ideal hiking trail at Yellowstone National Park.
It then crafts a tailored itinerary along with a
comprehensive patent list for your journey.
And that's not all.
With the power of Azure Speech, I can enable Speech to text
functionality in the playground, enhancing your experience
further.
There's more to see here.
With integration of custom URL voice into AI Studio, you can
create a custom voice solution that can adapt across languages
and speaking styles.
Our AI Studio tour has ended, but your journey is just
starting.
We're thrilled to equip you with the tools to safely build
generative AI applications using Azure AI Studio.