[MUSIC]
>> Welcome to another stream.
Today's stream is a little bit special.
But we're going to be trying
to play with cognitive search today,
which should be pretty cool.
I'm excited for the project that we're building.
The plan for today is we're going to be following a tutorial
which, I picked out a tutorial that I found on the internet
which sounds similar to the project that I want to do.
It should give us a good, you know,
set of groundwork to implement.
So we're going to run through the tutorial first
and then our stretch goal
is to get it actually working.
Okay. Before the stream, I set up an Azure free account
using my email address.
So this is a blank, empty Azure account.
We're going to be using this today
to build out our solutions.
But the tutorial that we're going over is this one here.
It's relatively recent so I'm hoping that it still works.
But the basic idea is we're going to be using
the Azure search and speech to text API
to get knowledge from audio.
We're actually going to get it from video,
hopefully in the end, by converting video to audio.
But we'll see, we'll see how that goes.
We're probably gonna skip parts of the lab
cause I don't actually care so much
about the rest of the detection.
We're just concerned with speech to text.
Oh, they also say video files.
Cool, blah, blah, blah.
Hey! Workflow, create the environment,
clone the GitHub repo, that's easy.
We can do that real quick.
CD workspace git clone.
All right. So we've got a bunch of stuff in here.
We will look at the code later.
We're probably going to change the code, guessing.
Okay. Create the environment.
Azure search service, okay.
So we're going to be creating a bunch of services.
Oh, look at this fancy thing.
Before you start, blah, open a free account.
Okay, cool.
Azure cognitive search.
Yeah. So the way Azure is organized is in these resources,
and so you create a bunch of resources
and then you stitch them together.
Okay. So create a resource.
Azure cognitive search, create.
Let's see, where were we in this tutorial?
Zoom this in.
So we are going to create a new resource group.
I'm just going to call this asottilecognitive.
Name the service.
So we're going to give this a name,
asottilecognitivesearch, and select that.
Great, review and create.
Okay. I think that's our first step, create.
If I recall correctly, most of the resources
take a little bit of time to create
so we can actually go on to the next step here.
Speech to text service.
This is what's going to let us take our audio
and convert it into text and create a resource.
And we're going to be creating a speech resource.
Perfect. So create, asottilespeech, create.
Okay. So that's the speech resource.
Storage counts, I think I'm just going to click add,
see what happens, right?
Create storage account.
Resource group, this.
Okay. I think that's probably good.
Just click the button, I guess.
I assume the defaults are fine so let's create that.
That get built.
Blob container.
Follow this process to create a container
within the storage account you just created.
Storage accounts, blob service section.
Okay. We're going to go here, I guess.
Oh my goodness, there's so many things here.
Lot of service containers.
And we are going to add a container,
asottilecognitivecontainer, create.
Cool, we have a container.
I don't know what that means,
but we will get there eventually.
Cognitive services resource.
Okay. My guess is we're going
to click create resource, right?
Blah, blah, blah, blah, blah.
So let's try adding just a resource
and see if it's a resource or if it's a subpart.
Cognitive services, seems like a thing.
Let's do create.
All right, all right, chat.
Are we the police? [laughing]
We're not the police, right?
We can check this box.
Well, that was weird.
I think that's all we need to do.
We can just do review and create.
That was easy.
[laughing] Great, great name here.
All right, done that.
Clone the repo.
In the Cortana search bar type git bash.
Now we're going to use [inaudible] for this.
Okay. So we're not going to be passing these
as command line arguments.
So we're going to adjust the way this script works.
We're just min maxing, fixing their script with ShellCheck.
I don't know what command pattern RE is
but I'm guessing we would need that there, too.
Anyway, we're not going to do this.
We're gonna do this, we'll do that some other day.
Okay. So West US two and subscription key.
This is the key of your cognitive service
speech to text API.
Use the image below to help you find it.
Okay. They've blacked stuff out.
So my guess here is [laughing]
we're going to have to black stuff out, too.
Okay. So this is the speech to text thing.
So let's go here.
Let's just type speech here and see what happens.
Asottilespeech, okay, it does exist.
Also, it looks like they do a really cool thing
and they have two separate keys here.
Only one key is necessary to make an API call.
Oh, this is cool.
So it gives you like a built-in way
to do atomic key rotation.
This is actually really nice, I really like this.
But I'm actually going to copy this
and we're going to open this M file
which is currently empty.
And then I'm going to hide this and paste.
You guys got all excited that I'm going to leave my keys
but that's not happening.
That should be good.
We're now going to run this script and see what happens.
Code, we're going to get rid of the echo here.
And now we run it and pray that I don't leak the tokens.
Okay. We're gonna run the scripts.
We're gonna see what happens.
It's doing something.
Oh, this is going to take forever
because of my bandwidth limiter, but that's actually good
cause we're not dropping frames.
That was the goal.
But anyway, I think it's completed now.
So in theory, we have a bunch of JSON files now.
And let's look at one of those and see what it looks like.
It looks like it was able to detect some text,
so that's cool.
I wonder what this sounds like.
But anyway you can download the audio samples yourselves
from the tutorial link, which is in Bang today.
Okay. So we've generated these JSONs.
I think the next step is uploading these.
I'm just going to manually upload stuff into this.
JSON to my share, those are all in Windows.
We'll go here, upload, and we'll select some files from here
and then we'll do that next and we'll just call it that.
And all right.
So now we should be indexing our search, in theory.
This is how a programming streamer died choking on water.
Okay. So it says that we created our index.
So now we should be able to search it
using the search explorer.
Let's try the word likelihood, see if that shows up.
Yeah, there we go.
That's kinda cool.
So we can search this.
That's pretty neat, I like it.
That's kind of the basics
of what I wanted to get out of this.
Okay, so now we got the demo.
So we're basically done with the demo part of this.
I want to see if we can get a YouTube video
ingested into this.
I am going to copy one YouTube video.
Let's copy the PyPI video.
Two minutes, that should be probably fine.
I need to copy it.
Okay, cool, copied there.
And then we're going to try and use FFmpeg
to extract a wav file from it.
FFmpeg, mov to wav.
All right. Let's try that.
File ep022.wav.
Acodec, we probably need to pick a particular codec.
All right, I'm just gonna click this,
and you guys won't be able hear it, but I might.
Okay.
>> Video: How to pronounce PYPI.
>> So we can hear that, right?
You guys can hear that
in chat? >> How to pronounce PYPY.
>> But you guys can hear it.
Now in theory, I should be able to up,
I should be able to run the speech recognition on that file.
Oh! Hello and welcome back to another video.
This one is going to be super short
and it's a huge pet peeve of mine,
but hopefully this will be insightful to yuan now.
The, this speech recognition is not perfect,
but it's actually pretty good.
This is actually kinda cool.
But let's copy that and upload it.
Storage bucket, okay.
I'm starting to starting to piece a lot of this together.
Let's just search for somewhere that we know that's in here,
like the word pronounce,
And search for pronounce.
Oh!
What if we search for PyPI?
Okay, cool. So PyPI matches as well.
That's pretty cool.
I'm gonna actually build something out of this
because I've wanted something like this.
It is interesting that it found pi, the symbol,
when I said PyPI.
But it did very carefully figure out
that I said P, Y, P, I.
Oh, that's so good.
I didn't really have much planned.
I didn't think we would get to this point.
Is there anything that you guys would want
to see me try and do with this?
I guess we could try and see if I have any other short ones.
Okay. So let's try this video and it might be too long.
I don't actually know how long it is.
Scripts, get text, the_front_fell_off.wav
See what happens.
Oh, look, that worked pretty well.
Let me put this in a JSON file.
It actually detected way better,
probably because the speaking is more clear than mine.
Yeah. Seems like it did pretty well.
Okay, let's put it into search too
and make sure that that comes up as well.
Upload, containers, this one, upload.
Yeah, it works.
That's so cool.
This is such a cool piece of tech.
I think we got what I wanted to get working
but this has been great.
We got stuff working, I played with the text to speech API.
We got search working, got all the components
that I would need to build an application around this
for searching YouTube videos.
So that's pretty cool, I'm excited about that.
But thank you all for stopping by
and let's go find someone to raid.