[MUSIC]
>> Hey guys, how you doing?
So tonight, I'm gonna be looking into Azure.
I wanted to use a TTS service,
but the problem is it costs money.
And as a private developer,
somebody who is just making this on my own time,
this being GIFBot, I didn't want to take the brunt of that.
And so what I'm gonna do is set up TTS and get it running
using the Azure Cognitive Services and speech.
Cognitive speech service, sorry.
Okay, I've already set up TTS
to kinda function with the exception
of not actually doing the actual TTS portion.
The Azure website has a portal
that you access to add services.
And what I'm gonna do is I'm in my portal,
I'm looking through.
[MUSIC]
So I go to Cognitive Services,
embed the ability to see, hear, speak, et cetera, et cetera.
Create a cognitive service.
It's called Speech.
Let's see here.
Speech, okay.
Will this do text to speech?
Oh yes, convert text to audio nearly in real time.
Okay, so I'm gonna create that.
I'm gonna move that off of the screen.
I named it GIFBotTTS.
This is gonna be my service.
You know, it's my subscription there, I selected that.
I am in Central US, so I'm gonna set it to that region.
Free tier, please and thank you.
And I just created a resource group.
So I'm gonna pop this back over here and create that.
Okay, looks like the deployment is in progress.
So we're gonna let that kinda do its thing.
Configuration wise, this is the tutorial that I'm following.
And it legit has all the stuff that I need in it already.
I just want to stream it to a file.
I don't care about it being in memory
because it needs to be accessible
from the web and loaded from the web.
And so I'm gonna save it to a temporary file
that's gonna be like a random GUID that I create
that goes with the request.
And create that file,
and then, immediately turn around
and delete it when I'm done with it so it won't build up.
Because the request,
they basically end up getting queued up.
So you can't guarantee it have this one file
that you overwrite and write to all of the time.
It's just not gonna work.
You know, I probably need to add the service to this.
Dependencies, Manage my NuGet Packages.
Microsoft.CognitiveServices.
There we go.
I'm going to install that.
Hopefully, this gives me everything that I need.
So we're gonna do SpeechConfig., using FromSubscription.
Now, I temporarily hard-coded my subscription key.
I just threw it in there temporarily.
This is not where it will stay forever.
I don't like to use var.
And the reason I don't like to use var for something is
'cause I don't know what kinda object I'm working with.
So I'm actually going to change that.
This is interesting, using AudioConfig.
AudioConfig equals AudioConfig.FromWAVFileOutput test.WAV.
So we'll just do that.
Using SpeechSynthesizer equal to new SpeechSynthesizer.
And then I pass it that config.
That config plus the AudioConfig.
Await Synthesizer.SpeakTextAsync.
This is just a test of the emergency broadcast system.
I'm gonna temporarily make this public.
[laughs] And I'm gonna go into GIFBot
and I'm gonna hijack a button.
I just don't know what I wanna hijack.
I need to steal...
I'll go into the hub for GIFBot, GIFBotHub,
PlayAnimation, this is the one I usually hijack.
So I will just do ___ Bot.RegurgitatorManager.
There we go, I think that was it.
Yeah, that's the method I made, TTSVolumeSvavaBlount.
Put a break point in here
and just make sure that we can get into this point.
If I can get this configuration with my sub ID
and the region that I created, then we should be good.
Okay, my deployment is complete.
So if I go to my resource,
it looks like it's up and running.
I don't think I need to do anything else with it
because I just pass it my ID and region.
Let's start it.
God, I hope it builds.
No, it doesn't build.
Oh, that's so weird.
I know I refactored it when I did it.
Huh? Okay.
So this is the babyyoda or it was the babyyoda.
Wait, why didn't you play?
Oh, I hijacked that button. [laughs]
I'm so sorry, I hijacked the button.
Okay, so it's connected.
There's the AudioConfig, and it return.
So I'm gonna hit playthrough >> I always sneeze on
my knuckles before a fight to inflict extra poison damage.
>> Wait, what?
Oh, my God!
Wait, the code actually goes through?
No, no, no. Wait a second.
Where is it playing at?
I didn't tell it to play.
Did you guys hear that?
Wait a second, what happened?
It played this, but how?
Because I'm using Google TTS and it doesn't work in...
No, no, I'm just like, wait a second.
So this worked, I don't know how.
So here's the thing is, in Chrome,
there is a way to invoke TTS.
It doesn't always work, especially in Chromium.
So that's why I wanted to use Azure
because they have a more sophisticated system.
Clearly it didn't take much time
for it to come back with this information.
So I probably should grab that result.
So let's do speechsynthesisresult, result is equal to...
Let's Google what this does.
It presents the synthesized audio in the properties
of the result.
The reason, result ID, which is a string.
So we're gonna put a break point right here
and it won't fall out of scope 'cause I'll just step over.
So we'll break here and we're gonna do F10.
And what's in the result?
Canceled, why was it canceled?
Maybe there's something I didn't do in the portal.
These keys are used to access your cognitive service APIs.
Store them securely.
For example, using Azure Key Vault.
We also recommend regenerating these keys regularly.
I'm not sure if the subscription key is the GUID
that's assigned to my subscription account.
Or if it's the key that's on the resource.
Both keys do the same thing, one is a backup.
Use the key with the endpoint.
Here with the endpoint.
Yeah, because the endpoint is a token.
It's issuing a token.
So I have this feeling that's not gonna do
what we want it to do.
Okay, I'm gonna bring the bot over.
So if you guys haven't seen GIFBot,
the TTS option is gonna be for the regurgitator.
But people will need to learn how to create
an Azure account and use that.
So I'm just trying to get this TTS functionality
kind of started
and experimenting with the Azure service tonight.
All right, let's see.
Gonna hit F10 to step over.
It did it.
Okay, so I'm gonna shut this off.
This file...
Let's open it up.
>> This is just a test
of the emergency broadcast system.
>> Fantastic! Okay.
So this worked, I'm so shocked when that happens.
This is our request force, a TTS request.
I don't wanna do these.
What I wanna do instead is to create
a temporary animation data, and that should work.
So we have an entry.
GUID TTSID is equal to GUID.NewGUID.
Okay, so that will be the identifier I use
for the animation.
So AnimationData, TTSAnimationData is equal to new.
TTS will be the name of it.
This is not gonna be stored anywhere.
So I'm just gonna send it up, and play it, and stop.
Let's initialize it this way.
All right, TTSAnimationData.
This is what I wanna do.
Is I wanna create this temporary animation data object
to kinda just hold that information and push it up.
But the thing is
it usually tell it to stop playing an animation,
but it don't tell it to stop playing the audio.
Audio is equal to TTSID.WAV.
Why is it auto, stop auto formatting?
[laughs] I don't want you to do that, volume.
Okay, there's gotta be a way that I can get the length
of the file.
The result, it has how many bites were written,
but it doesn't tell me what the duration is.
If I don't set an accurate duration on the WAV file
that's pumped up to the animation.
Then there's an opportunity for it to get stomped on
if there's another animation queued behind it.
For now, we'll set it to 5,000 milliseconds.
So there's the result.
Instead of forcing a TTS request
we'll do Bot.AnimationManager.ForceQueueAnimation.
And we're going to do animMrSmoofy, pass empty trigger name
'cause I don't care who triggered it.
So what that's gonna do is that's gonna go through,
and it's going to force queue it here,
and add the animation to the...
Yeah, I think that's really all I needed.
I will eventually need to have the time on it.
Let me just make sure this is the right way to do it.
As I push it up to a blazer front-end,
I have that do a JavaScript interrupt to play the sound.
But the thing is there is a queue that is tightly controlled
by the server that's running an ASP.NET Core specifically.
Another animation could get put in behind it
and then play it.
So if the duration of the WAV file is actually
a little closer like 10 seconds,
it'll sit there and play the next one
because the sounds don't auto stop.
So I have to be cognizant of that.
So when I send it up, it has to have accurate timing,
otherwise it overwrites.
Because the browser doesn't have an idea of audio threads.
If you tell JavaScript, "Play this audio,"
and then you turn around and say, "Play this audio too,"
[sound] it'll stop.
Okay, so we send the start of an animation,
the send chat here.
So here's the send for the TTS message.
That starts a timer and this tells it to stop.
That's why there's somewhat of a delay by the way
if there's a bigger visual
because I have to drive it through here
if I trust the client to kill it.
Then if you're switching scenes or something,
you'll get stuck in a state where GIFBot
is constantly running the same animation
because it never got told to stop.
I need to know when it's ready to stop.
In here, when I go to deactivate the animation.
AnimationData animation, animation command.
DeactivateAnimation TryRemove command,
AnimationRequest removed.
Okay, so the command name needs to be a GUID.
So when I put this in, instead of naming it that
I need to name it by the GUID.
I wanna say I had a TTS request,
and it's a request-based audio, audiofile.
Basically, I'm talking to Azure now
and retraining this information to write to a file
that I can then play in the browser real fast.
This is my first time using this.
And so this chunk of code here is probably
gonna go into a helper class at some point.
So I don't have it hard-coded in this one spot.
But I wanted to get something up and running
and working tonight.
So I'm kind of like working in this tiny little area
just to get things going.
ForceQueueAnimation, I send it the data,
but I need a way of tagging it.
Let's do animation.
If animation is not able to know, then AnimationRequest
request is equal to new AnimationRequest,
animation, String.Empty, String.Empty, true, false.
We're gonna create an enum.
Okay, so let's go to AnimationEnums.
Yes, public enum RequestType, animation, and TTS.
The type of request for an animation when it is queued.
It determines how it is treated once it
has finished playing.
Public RequestType,
AnimationEnums.RequestType animation by default.
Okay, so what this is gonna allow me to do
is identify the request so that I know how to treat it
on the returning end.
I'm gonna do ForceQueueTTSRequest,
which is gonna take this animation data.
And it is going to generate this request.
I'm gonna set the RequestType equal to the TTS type.
So I can check against this when it's done playing
and clean up the file I created
because I don't need to do like,
stop, get rid of the animation from the queue,
all that stuff.
I just need to kill it [laughs] with fire.
So what this is gonna do is create the animation.
The other thing I need to look at was where
to shove the file.
'Cause right now, I'm putting it in a place that the web
and that's not gonna be able to find.
Let's look at our controllers
because I have the upload controller.
And that should be putting files where it's supposed to go.
Animation data import.
There we go, that's where it's supposed to go.
Now, we'll queue a TTS requests that goes in here.
Once it's played, we start the animation timer.
It'll try to remove it from the active of animations.
That should be fine because I name it by the GUID.
The command name is a GUID.
That's gonna be, if somebody can guess a GUID, go for it.
If animationRequest.RequestType is equal to TTS.
This was a generated TTS alert.
Delete the TTS audio file.
And that would be animationRequest.AnimationData.Audio,
string audioPath.
AudioPath, File.Delete audioPath.
♪ Pam para pa ♪
♪ Pa pa rapa para ♪
And that will be it because the AnimationRequest
and the AnimationData was all temporary.
It should fall out of scope
because nothing will be referencing it after it does this.
I am so curious to see if this works.
I wish I could check the result a little better.
Otherwise, if it fails it's just gonna start racking up
all these files.
I'm gonna hit this break point here anyway.
So I'll look at that
and I'll see what information we're getting back from it.
It might give me the duration in the results.
Let's just see what happens.
Okay, 10, there's the result.
What do we get back from it?
Here's the properties.
There's absolutely nothing in there that's helpful.
The audio data is just bytes.
I can probably figure it out.
I bet you it's in the properties.
F11, ActiveAnimation, there's only the one.
Interesting, it didn't play it.
Aw, regurgitator data loaded.
Let's try it again.
>> A guy that's really bad at limbo walked into a bar.
>> Now, it works!
[laughs] It works, I just had it paused.
Okay, so I'm gonna go into our dev folder here, server.
And that's where the settings are, Client.
♪ Dubbie dub dub dubba dubba route. ♪
Let's go to our Details view and do Date.
I'll sort it by date.
What I wanna do is I wanna watch this
>> Hard work will never hurt you
as long as you watch it from a safe distance.
>> I'm shutting the bot down.
All right, so it's working. Aah!
Okay, well that's basically it.
So now the real crux of the problem is ensuring
that we get the right duration.
You would think it would pass back
this information to let me know, but I guess not.
Can I synthesize straight to SpeakTextAsync?
Oh, I might be able to do this right here
and not have to put it through the system at all.
Which means it would play through the speakers,
which does understand.
SpeakTextAsync...
Oh, do that, but don't do the AudioConfig.
If I just do this section like that,
just with the configuration file
and don't tell it I wanna write it out,
then it should just speak the results.
But mind you it's from ASP.NET Core.
It might not understand desktop audio,
but we'll see if this works.
So GIFBot is hosted on the user's computer.
As much as I would love to host it on a website somewhere,
I can't because of the nature of the files.
I thought about while I could put GIFBot light up somewhere,
but it would limit people in the size of the animations.
They wouldn't be able to do half of the crazy stuff
that they wanna do with it.
So it just seems silly.
Okay, so we've made it down here.
>> My therapist told me my narcissism
causes me to misread social situations,
but I'm pretty sure she was hitting on me.
>> [laughs] I am in love.
I don't even have to do this.
I can't believe that actually worked.
I love it!
Okay, so that now works.
Wow, I may pick this up again tomorrow.