(gentle upbeat music)
- Up next, we take an inside look at Azure's OpenAI Service
that lets you leverage large scale generative AI models
based in Azure that have a deep understanding
of language and code as you build new applications.
Now we'll unpack the core concepts for interacting
with models, using prompts,
and demonstrate how you can use Azure's OpenAI Studio
to experiment with and test your models
before bringing them into your code
to deliver differentiated app experiences,
all with Azure's enterprise-grade security
for your app's foundation.
And joining me today
for an in-depth look at Azure's OpenAI Service
is Pablo Castro, distinguished engineer on Azure's AI team.
Welcome back to Mechanics.
- It's great to be back, and in the studio this time.
- Thanks so much for joining us back on Mechanics.
So OpenAI, it's really started to gain momentum
over the past couple of years, you know,
provides this really large, extremely rich set
of base AI models that can accelerate the process
of constructing natural language powered experiences,
that can, for example, greatly improve how you interact
with your apps and extract knowledge from data.
Now, this is an exciting area that Microsoft
has invested a lot in.
So what are we solving for then
with Azure's OpenAI service?
- There's definitely a ton of potential and we see lots
of applications powered by Azure OpenAI today.
From a developer's perspective
the thing that excites me the most
is how it helps build new experiences
with several cutting edge models like GPT-3,
a large language model that generates content based
on natural language input, Codex,
which can translate natural language instructions
directly into code, and DALL-E 2, a new model
that generates realistic images
and art from natural language descriptions.
And when we combine OpenAI
with Azure into the Azure OpenAI Service,
you get the core building blocks you need
for production-grade applications,
from how we host OpenAI at scale for you in Azure
to enterprise-level security.
We're also giving you a great environment
to try things out and iterate.
The Azure OpenAI Studio lets you experiment
and test your ideas with OpenAI
before bringing them into your code.
And when you're done experimenting
and you know what you want, you can call the service
from your code just like any other REST API.
And security is built in
with capabilities spanning strong authentication,
role-based access control and the ability
to configure virtual networks, private endpoints
as you would for any Azure service.
We've also incorporated tools for supporting responsible AI
with content filtering in the service.
For example, if your input includes inappropriate content,
the Azure OpenAI Service would catch it.
And this works for any generated output too.
You can use this to detect
and mitigate harmful use following responsible AI principles
which you can find more about at aka.ms/AIprinciples.
- So can you give a few examples
of how we're using OpenAI in apps and services today?
- Sure, OpenAI actually powers a number of experiences
at Microsoft today.
For example, co-pilot in the Power Platform enables users
to author Power Fx commands or even reach automation flows.
Another great example is the Designer app
we recently announced, which uses the DALL-E 2 model
to interpret your text descriptions
to generate images and even artwork.
Let me show you. I'm here in the Designer preview.
You can find this at designer.microsoft.com.
I'm going to add a design headline, say DALL-E 2
and here I can generate an image from natural language.
I'll type grass field with trees
and you'll see Designer gives me some nice recommendations
to augment it.
But I'll keep things simple and just add cows.
Now if I want to change the look,
I can add as a cartoon
and that's starting to look closer to what I want
but I'll change the cartoon to a blocky video game
and that's exactly what I wanted.
Now I just need to choose a design.
I'll pick this one and you'll see our progression of images.
And just to see a little bit more detail,
I'll select this second one,
so I can zoom into the middle of the screen
and now you can see more of a closeup view.
These were all generated on demand.
Designer interpreted what I typed
and it created new images each time I iterated.
So these apps aren't just putting an app frame
around the API.
OpenAI models like this can help you build
more dynamic, interactive and differentiated experiences.
- And that's really an important point
because we're not training models
in the traditional AI machine learning sense,
we're just interacting with them using text.
So can you explain a bit more behind that concept?
- Yes, these types of models use text for interaction.
They have been trained on huge amounts of written text
and can use their accumulated knowledge
to perform a number of tasks directly.
We call the text input to a model a prompt.
The simplest form of this interaction pattern is
to give a model a string of text or a prompt
and ask it to complete it.
It could be a partial statement, a question or anything
you want the model to add to.
In cases where the model has enough context
to generate good outputs, that might be all you need.
This is sometimes called zero shot
because the model responds
with no additional input or training.
That said, sometimes you do want
to guide the model a bit more.
You can do this by giving it examples
of how you want it to behave,
such as a question-answers pair, and finishing the prompt
with the one you wanted to complete.
This still requires no actual updates
of the weight in the model.
The examples are learned on the spot.
So this is often referred to as few-shot learning.
- So how easy is it then to integrate OpenAI
as part of everyday apps?
- Oh, it's super straightforward.
OpenAI can be used to create many new experiences
that were impossible before.
My favorite example is how you can transform
the entire surface area of an app
into something you can converse with using natural language.
Let's use Minecraft, the game as an example.
And if you've played Minecraft, you might be familiar
with the text commands you can make during gameplay
to manipulate the world around you.
The challenge is that there is a lot of commands.
I'll run /help to see all the possible commands.
Here you can see there are dozens of different commands
that you would need to memorize
and this is in addition
to the more than a thousand different materials
and lots of different entities.
One command is slash /summon, which is used
to spawn a new entity into the world from cows to ship
and then /field to build structures using blocks and so on.
The nice thing about Minecraft is that it's extensible,
so you can write custom mods.
What I'm going to show you is
how we created a mod called conjure, which acts as a plugin
so that when you write a natural language message,
it'll pass that command to the OpenAI Service in Azure
using the Codex model. This interprets the intent
and finally passes back the generated command
that Minecraft can natively understand.
So let's try this out for real in Minecraft.
I'll ask it to make a cow and there's our cow.
It converted my request to create one.
That was pretty easy though
because it just converted my command to summon cow.
Now let's be less precise
and type, "Make an animal that goes moo."
And OpenAI knows about the world
and it was able to translate my intent
using the sound a cow makes
to tell Minecraft to summon a cow.
And beyond the summon command,
it can also interpret other requests.
I'll ask it to make a transparent wall and there it is.
Even though transparent isn't part of a block name,
Codex knows that glass is transparent
and it connected the dots for us.
It executed the fill command and as you can see,
it even came up with relative coordinates.
So what we just saw is how we can use OpenAI not only
to let you interact using natural language
with an application, but also how the models knowledge
of the world in general can really enrich the experience.
- This is really a great example of the possibilities
for interacting with apps using natural language.
Now aside from building the mod itself,
was there anything else you had to do?
- Remember that we discussed before
the importance of prompts?
So for this I needed a very simple prompt.
This asks Codex to translate natural language
into Minecraft commands.
At the top, I made a statement in English about my intent
and then I gave it a few examples
such as building walls using relative coordinates
and summoning animals.
This is like a few-shot scenario.
You'll see that even though I defined that "Make a fish,"
in my case translates to /summon tropical fish.
It could take that tiny piece of information and generalize
to all similar cases, even if the details vary a lot.
So these prompts apply to all the animals in Minecraft,
even the sounds they make
or other indirect ways you might use to refer to them.
This really shows the power of few-shot learning
with the OpenAI Service.
This whole thing was just a few hours of work with my kids.
- And it really shows how just a little bit
of guidance actually gets you the outcomes
that you want using your prompt.
- Exactly and these models are already in use out there.
For example, Codex powers the GitHub co-pilot capability
that helps developers write code faster.
Let me show you.
Here's a portion of the code behind my Minecraft mod
in Visual Studio Code
and I have the GitHub co-pilot extension enabled.
The mod is written in Java and this is the part
that takes the text that the player wrote
and sends it to OpenAI.
So here I have the HTTP request
that calls the OpenAI Service in Azure.
What's missing is actually sending the request
and then processing the response.
But I can use co-pilot to write that code.
Here Co-pilot is predicting this entire line of code.
I just need to hit tab and I can keep going
and just by hitting the tab key a few times,
co-pilot wrote all the code we needed,
from login to parsing the JSON response
and producing a result.
- And I got to say that's super impressive
just using the tab key in that case.
So can you show us how you might put all this to work
with an app that you might build from scratch?
- If you're building a custom app,
you just need a few lines of code
to call the OpenAI Service.
Let's take the example of a green energy company.
We need to keep up to date
on the latest trends and insights,
which requires us to build an understanding
across lots of unstructured data sitting in documents
from various sources.
I'll explain how we've architected our app first
and then I'll show you how it works and what we did.
In this example, the user can ask the app a question
using natural language.
The app then calls Azure Cognitive Search
to discover text-based documents related to that question.
This ranks the documents and returns the top candidates
but to answer some types of questions,
we'll need data across multiple of these top documents.
So it passes the top candidates to the Azure OpenAI Service,
which reads and understands these top documents,
leveraging the GPT-3 model to generate an answer,
which is then passed back to our application
using information spread across this relevant documents.
So let me show you this in action.
I'll start in the browser
and you're looking at the simple app,
but the real beauty is in the power of information
it can bring to you.
I'll do a quick search
for renewable sources expected and actual for 2022.
And it finds a few relevant documents
each containing parts of the answer.
And at the top you'll see the generated answer
that grabs elements from multiple documents,
like 22% from the first doc
and first half of 2022 from the second doc.
Notice there is a third unrelated doc
that was not included in the summary
because GPT-3 realized it was not related
to the question.
Now I'll show you how easy it is to call OpenAI
and generate the cross-document answer which you saw.
I'm in Visual Studio and here are a few lines of code
that do all of the work.
First, I take the user query
and send it to Azure Cognitive Search.
Then I take the results from search,
which are multiple documents and I construct a prompt
from them using a template.
Finally, I send this prompt to OpenAI
to generate the summary answer.
So with very little code we could leverage OpenAI.
And this same process could be used
to very quickly solve other problems
like, entity extraction or summarization
that otherwise would've required you to pick, train
and deploy custom machine learning models.
Here you more or less just build a prompt and you're done.
- Right, I can really see how useful this would be
across any number of research-based activities.
That said, though, in this case,
you actually knew what to do.
So if someone's watching and needed to start from scratch,
where would you even begin?
- The Azure OpenAI Studio is a great place to start.
It lets you experiment with prompts
before you build them into your apps.
To use the OpenAI Studio,
first you'll need an Azure subscription.
You'll need to apply for access
to Azure OpenAI using this form
at aka.ms/oai/access.
After you create an OpenAI resource,
it'll link you directly to the OpenAI Studio.
In the studio, you'll start by creating a deployment.
You can think of a deployment as your own service endpoint
for OpenAI that your application calls.
In my case I'll use an existing deployment,
so we'll skip that.
You can see I already have two deployments,
one using davinci2, for natural language generation
and another one using Codex for code generation.
Going back to the home screen,
you'll see the studio offers a few examples to get started.
Summarize Text, Classify Text, Natural Language to SQL
and generating new product names.
In the studio, we also have the playground.
So let me head over there.
First I'll choose one of the deployments I created.
I'll choose davinci2.
Then I'll select an example from the dropdown menu
and I'll start with this example
for generating a new product name.
You can see the example uses a few-shot prompt,
with a product description and seed words
to help it understand and generate results.
In this example, we're looking to name shoes
that can fit any foot size,
with seed words like adaptable, fit and omni-fit.
When I generate the results, it comes up
with omni-fit shoes, perfect-fit shoes and all-fit shoes.
And so you can see how this can be used
to help people with their creative process.
- How would you then go from these examples
that you just showed to maybe the OpenAI search app
that you showed before?
- Well, the playground helps you test
and iterate on your prompts
before you incorporate them into your custom application.
Let's use the summarized text example as a starting point.
This example prompt is about neutron stars.
By looking at the structure of the prompt,
I can see the pattern they follow and learn from it.
We're using the Davinci model,
which is the largest of the GPT-3 models
and optimized for text understanding.
Now you can see this long paragraph of text as input.
The example adds Tl;dr at the end to suggest to the model
that this should be completed with a short summary.
Let's run it.
And here, you see the generated output about a neutron star.
So I'm giving OpenAI instructions
for what to do in plain English.
Coming back to the cross-document summarization example,
let's guide the model to read a set of statements
so it can answer a question based
on those statements, just like a human would.
If I go to Visual Studio
and grab my prompt and paste it here,
you'll see our prompt, including the placeholders,
our code replaced that runtime in the app.
This is real content,
so it's pretty long and hard to follow.
So instead of running that one,
let me just paste a shorter version
that has the same structure,
so we can more easily see what's going on.
I'll ask it to complete the task,
which is to answer the question
about the percentage of homes with solar.
And you'll see a detailed answer
with a few regions called out.
The answer was accurate
but I'd like to see a shorter version of the answer.
And this is where I can provide an example answer
to further refine the results
for the style that I'm looking for.
So first, I'll undo the last run to clear the answer.
Now I'll paste in the same statement from before
but you'll see that this is a different question
with a very concise answer.
From there, I can generate a new result
for the first question and you'll see that it learned
from our example and returned a very concise answer.
So the playground in the Azure OpenAI Studio
helps you experiment and iterate.
You can test your prompts for accuracy, refine them,
and then bring them into your custom applications.
And if that isn't precise enough,
one thing that I didn't have time to demonstrate today
is Azure OpenAI's support for fine-tuning
where you can specialize one of the base models
with additional data that you provide.
So you have the full spectrum,
from zero-shot to creating your own fine-tuned models.
- And all of what you've shown today to be clear,
is completely grounded in security and responsible AI.
So now that you've got hands on with Azure's OpenAI Service,
can you explain for everyone watching
how they might get started and learn more?
- The best way to get started is by trying it out.
We've made it easy to sign up for the service
at at aka.ms/oai/access.
And from there you can start using Azure's OpenAI Studio.
- Thanks so much, Pablo, for joining us today
and showing us some highlights of what Azure OpenAI can do.
Of course, keep checking back to "Microsoft Mechanics"
for all the latest updates.
Subscribe to our channel if you haven't already.
And as always, thank you for watching.
(gentle music)