>> You're not going to want to miss this episode of the IA Show.
We'll talk all about Azure Video Indexer,
with ChatGPT integration and Edge expansion.
Make sure you tune in. Hello and
welcome to this episode of the AI Show.
We're talking all about Azure Video Indexer with
ChatGPT integration and Edge expansion.
How you doing, my friend?
>> I'm doing great. Thanks for having me.
>> Fantastic. Why don't you introduce
yourself, tell us who you are and what you do.
>> My name Inbal Sagiv,
I am Principal Product Manager of Azure Video Indexer.
>> Fantastic. For those that don't
know what Azure Video Indexer is,
it saved my bacon a lot of times incidentally, tell us what it is.
>> In one line that's the product of Azure video AI.
What it does is you upload a video and then we extract
more than 30 different AIs for
the video and for the audio part of it.
>> You're saying that basically I just upload
a video and I get a bunch of goodness.
I mean, you said it really fast.
I feel you're not giving it its due.
It extracts the text.
But what did you say? Insights. What does that mean?
>> Let me walk you through the story of Azure Video Indexer.
Look at the set of customers that we have here.
All of them coming from industries,
starting education through media, through public safety.
And all of them has multiple challenges
of large media archive where they would
like to extract the information
of that video and be able, for example,
to search for faces, for locations,
for key words, for topics,
for objects to understand
the transcript of that video for multiple purposes.
Whether it's around search,
whether it's to make their video much more accessible,
whether it's even degenerate ads and place those ads
in the right moments in those videos to create new content,
for example, for social media,
as well as to moderate some of the contents.
We run bunch of AIs, really,
really long set of AIs on top of cognitive services.
Customers have the freedom to choose whether they would like
to use our portal offering,
a web offering, or to take some pieces,
we call them widgets,
web components and embed those components in your offering.
Or to go and use the APIs.
>> Hold on. Can you go back one?
This is the part
that drives me crazy because we're like, it does some AI.
It does a lot of different models on the thing,
down to recognizing who's
in the video or that it's switched people.
Am I getting this right?
>> Definitely, with a single API endpoint,
you get more than 40 different AIs.
Some of the output of those AI
is used as an input for a different AI.
We actually work on both your audio track
and your video track and are able to do a multi-channel,
merge these information to truly
understand what's the content of that video,
in a single API call.
>> That's the cool part, I said,
it's saved my bacon a number
of times just with getting transcripts,
knowing when things have switched, its really cool.
Sorry to interrupt you. Keep telling us more about Video Indexer.
>> Let's just focus on what's new.
Remember that I mentioned those customers and their challenges.
We always look on the scenario of
the deep search and the accessibility of those video libraries,
rethinking how can we make those scenarios better?
Everybody knows Azure OpenAI and ChatGPT.
It's super trending these days.
We're thinking about a solution that will integrate these two and
actually would allow to do
a better search and accessible scenarios.
I don't want to speak too much just to walk you
through the demo that will explain exactly what we've done here.
>> Before you go to the demo,
now we're talking about the new stuff.
You somehow integrated ChatGPT and Edge
computation because this is
a lot of videos are pretty big and moving things back and forth.
Those are the two things you're going to
start to show us now. Is that right?
>> I'm going to start to talk only about the integration of
Azure OpenAI and ChatGPT to
improve the search experience, that's one.
Then I'll show you another demo where we basically enhance
our Cloud native offering and also expand it to the Edge.
But let's focus first on the first announcement that
we're going to show you a code sample that
everybody actually can download and try it yourself to be able
to integrate with Video Indexer,
GPT and Azure OpenAI together to improve search and accessibility.
Later on I'll move to the announcement around
the private preview of
the edge offering that we have our Video Indexer.
These are separate things.
In this demo we are going to look how we can integrate
Video Indexer with Azure OpenAI
to make the content more accessible.
Here you can see Video Indexer account where we
have our video files with a transcript.
We push those transcripts into the Azure OpenAI model to
generate embeddings and keep those embeddings in Redis database.
Finally, there are the web page
where you can use to search through
the content using Azure OpenAI search model.
Now here you can see
my Video Indexer account using the portal of Video Indexer.
You can see I've already indexed several videos for this demo.
These videos all contain a transcript.
After I deploy the solution,
this is the front end where you
can use a search through the transcript.
Now, let's just imagine I work for LinkedIn,
where we have indexed the large archive using Video Indexer.
For this demo, we know that I've indexed
several videos and I know that it is all talking about OpenAI.
Let's start with a simple question.
How is Microsoft planning to use OpenAI and
click on elaborates to get more information.
I click on elaborate, I get more information.
A cool feature is that I can also change the language,
let's say I want it this time on Dutch,
and you get the response in Dutch.
Let's give it one more try.
Now we want to learn,
what percentages of the students are actually using ChatGPT?
I get immediately an answer, five percentages.
I can click on elaborate to get more information about it.
The nice thing here is that I can also link to the moment
in that video where you can
see the information coming from the OCR,
which was detected on the screen.
That based on it,
GPT was able to generate the answer for this.
That's a code sample which is available for everybody to use.
If you just want to learn more,
you can click on this link.
There is a blog post with all the information there,
just download the code and try it yourself.
>> This is cool. Basically,
what you've done is you've used Video Indexer
to extract a bunch of
information and then you've done a pseudo RAG pattern,
retrieval augmented generation, where you're embedding results
from people's queries then send it
directly to the right video place.
Am I getting this right or am I off here?
>> Exactly. That's exactly what we do.
Currently, the demo that I
showed you just focus on the transcript,
but imagine the whole bunch of AIs which I've presented to you.
This is also a multi-modal approach
where we are currently working on another code sample.
Maybe the next AI show I will be able to show you this as well.
>> That's cool because when you're using OpenAI,
you could basically search
all of the goodness that's
extracted from videos and you can embed it.
You might be able to say, hey,
when did Seth say foo in this video?
You'd be able to find it because
it's in the transcript if you add,
who said what etc.
There's a lot of potential here for this.
>> You got me right.
You have these code sample,
just download it and try it yourself.
Everybody can do it in his own video library.
>> Cool bin. So what do you show us next?
>> The next thing is,
remember those customers with a large media archive?
>> Yes.
>> Sometimes some of them which has the eligibility.
>> They have the API of running face detection,
and in this case, this is Satya. I'm telling you a secret.
But these customers may want to redact some of the detected faces.
For this purpose, this is another feature
that we've added in private preview.
We've already created another private preview release for you.
That's again, my video library.
I would like now to choose one of the videos
where I would like again to redact the faces.
Out of that video library,
let's pick one of the videos that I have here.
In this video, we can see it says Satya Nadella.
Now, what I would like to do,
I would like to generate with an API call,
and you video, exactly the same one
just when Satya faces will be redacted.
For that purpose, I will use a postman API.
You see that we have basically, here,
three parameters that I would like to control.
The low blur,
is the one that I've selected in terms
of the different learners that we have.
The second one is the ID of Satya.
You see it as 1145,
that's the ID of Satya Nadella's faces that were detected.
It says, include,
because I want to include only that particular identified face,
and redact only Satya face.
Now, if I run the API,
it will give me a response.
After that response, I will be able to see
a new generated video here in my library,
and as part of this library,
I will be able to play that video,
the new video that was generated,
it should be exactly the same one.
I'll just expect the DBS time.
Satya faces, as you see here, will be blurred.
>> This is amazing. How do people get access to this,
if they want to try this?
>> To be able to get access,
just reach out to us through this e-mail,
and we will enable these on your subscription,
and you can give us feedback
>> Fantastic. You said you had two things,
but you may have three things.
I remember something about the Edge.
Can you tells us a little about that.
>> Sure. Here's what we hear from our customers.
Customers may have these nice native Cloud solution
with all the different set of AI's.
But now the third announcement is
around a private preview where we basically
expanded also to Edge types of use cases through the Arc platform.
What does it actually means?
Let's take a look on this example.
This is a real customer scenario.
They have a lot of regions,
some of them running on Azure,
some of them are running on AWS and GCP.
Some regions are actually have a Kubernetes clusters,
and those regions are restricted.
But they don't want to use different AI solutions from Microsoft.
They say, "Hey, we have Video Indexer running on the Cloud."
Just give us the same offering.
Also on the Edge for these two restricted regions or to run it
through content that simply sit on a different IS.
For that purpose, we basically connected to our Arc,
which is a single platform that will allow us to connect
those different regions and have a single Video Indexer running.
It doesn't matter if it's Cloud or Edge.
If you have some restricted regions where
you have to run the AI's close to your data centers,
then now it's possible with
the new private preview of Video Indexer.
Today we have six different buckets of AI's.
I show you the whole diagram,
but these are actually the way
that customers are looking on our product.
The one which is highlighted here,
if the frost private preview where we took transcription,
translation, and captions, and actually,
run those AI's on the Edge.
Now we're really looking to get feedback from the customers,
and there is a link to the registration form.
How you can actually reach out register,
and try it yourself.
It's exactly the same Video Indexer
under a single product offering.
Just now also enabled for Edge type of
scenarios on top of Kubernetes clusters.
Now, how does it look?
>> That's cool.
>> Yeah, it is.
We've worked hard on it,
and we were really listening to customer that says that they
haven't hybrid landscape as I presented before.
Now let's give a quick look how the flow looks like.
I don't want to talk a lot about Kubernetes and how it looks like.
But the demo starts basically with creating a Kubernetes cluster.
To simplify the process,
we've created an installation scripts that
you need simply to put here your subscription ID,
with a bunch of other parameters,
and simply get everything deployed on your account.
Here, I'm just running behind the scene.
That scripts that creates for me the Kubernetes cluster,
the Video Indexer account,
and all the information that I need.
At the end it said that it was running successfully.
I actually get a link to a Swagger API
where I can either run the APIs or
to the same Video Indexer look, it says hybrid.
Now it has both Cloud and Edge extensions of Video Indexer.
Like from VI can upload a video these time by taking
the video from my Edge extension.
I can give it a name like,
tests for Arc, for example,
I can choose here the language to optimize the AI's.
I can click on "Upload".
This is actually running in the background. It's really unique.
It's a combination of a Cloud and
Edge offering under the same experience.
Now you can see that the AI is actually processing,
and you can also track their progress.
Everything is done,
close to the data center of the customers who are
not leaving any data to the Cloud.
Now, after it's finished,
you'll see that the transcripts,
and captions, and the translation,
if you remember, this is the very first thing that we
do for our Edge customers are actually available,
so I can now switch the language.
In one click I get all the information in
a different language, does gated span,
and I can turn on the closed captions,
and the closed caption's immediately will be added.
The streaming is done from the Edge,
as well as the AI processing was
done on the datacenter of the customer.
That's actually something which our customers,
we're really looking as they have for a long time
to have the same solution from Microsoft, from first-party AI's.
Now not only as a Cloud-native offering,
but also running on the Edge.
If you want to join the arc
experienced Azure Video Indexer private preview,
please reach out to us through this registration form.
>> I mean, this is cool. The AI processes
are running, I'm guessing,
on containers inside of the Kubernetes clusters,
but they're accessible through
the Azure portal or the Azure Video Indexer portal as well.
It's like you have the same functionality but it's
on your clusters. Is this right?
>> Exactly. The control plane is done from Azure.
From Azure you create your Video Indexer account and you trigger
the creation of the Arc-enabled Video Indexer extension,
which is actually running and deployed on a Kubernetes cluster.
The AI processing is happened close to where the data resides.
But then from the Cloud portal,
from that web app that I've showed you,
you're able to see the combination of
the content that you actually prefer to run to the Cloud,
because you may have no restrictions and no local regulations.
But also you can go to any VI extension and run the
AI's on the Edge close to where the data resides.
>> Which is cool. I know we keep saying that,
but I feel like videos are big,
and being able to run it on the Edge is just absolutely amazing.
Where can people go to find out more?
What if they want to try this stuff out, where should they go?
>> Basically, there is one click.
Go to Video Indexer trial account,
you can see all those goodies
that I've presented in the trial account,
specifically for the private previews
that I've mentioned, you need to reach out.
Let me just make a quick recap.
The code sample is available for everybody.
There's these blog that I've put here.
Then the face reduction or the
Arc-enablement Video Indexer, private preview,
both you need to reach out to
these e-mail address or to
submit a form with all the information,
and then we will enable your subscription,
and you can work with us,
and share your feedback.
>> That's amazing. Also for those that are wondering there's
some good documentation here that you can take a look at it.
Then the sample that you mentioned before, that's available.
The ChatGPT integration is available here. Is that right?
>> Absolutely correct.
>> Fantastic. Well, thank you
so much for being with us, my friend.
>> Thanks you for having me.
>> We've been learning all about
Azure Video Indexer, which ChatGPT integration,
and Edge expansion, and there was a face blur in there to boot.
Thank you so much for watching,
and hopefully we'll see you next time.
Take care [MUSIC]