Hi, my name is Bob Heddle, and I'm here to tell you about the speech platform capabilities
of Windows 10 IoT Core.
Ok - here's what you're going to need:
A dev board running Windows IoT Core plus a USB microphone and speakers.
Also, to see the demo, you'll need a screen; to interact with it, you'll need a mouse,
Right now, as of //Build 2015, the dev board will have to be a MinnowBoard Max
because audio doesn't work yet on the Raspberry Pi 2, but it will soon!
Lets start by using the SpeechAndTTS sample to see what the speech platform can do.
This sample is part of the WIndows 10 SDK, you can find it up on github,
and since it is Universal Windows app it also works on Windows 10 IoT Core.
Ok! Here we are in the speech and TTS sample application.
This is a UWP application, and so it works across desktop, mobile, and also Windows IoT Core.
This first section shows speech synthesis.
Simply click the "Speak" button and it will say the sentence that's in the text box.
"The quick red fox jumped over the lazy brown dog"
Ok, the next sample is Speech Synthesis using SSML,
which is a W3C standard for speech markup and allows you to create much more complicated phrases:
"This sentence has a mark here and another mark here.
"This is an example of how to speak the word
'whatchamacallit.'"
Next come some predefined grammars: a dictation grammer and websearch grammar,
Those don't work on the build release for Windows IoT Core.
They will be coming soon in a later release. So let's move on to Custom List Constraint
which is basically recognition of a set of phrases.
This particular recognition is very simple - simply recognizes "Yes" or "No".
Yes
No
Finally we'll look at the last section which is recognizing custom SRGS grammars.
SRGS is a W3 standard for defining speech grammars,
and speech grammars are basically the set of things that it can recognize, so let's try it out.
Red Border
Green Circle
Cyan Background
Ok, now it's time to define our grammar. Basically we want to say stuff like "Turn
the upstairs lights on" and we might want to also say "Turn on the
upstairs lights". So how do we build that?
First, we create a list of locations: Upstairs, Downstairs, Upstairs Bedroom...
You can put in whatever you want. And then we'll have states: On and Off.
With that, we'll create the constraints for the recognizer.
So for each location we'll create a phrase for turning it on
and another one for turning it off, and we'll also have both of the phrases:
"Turn on the downstairs lights" and "Turn the downstairs lights on".
When we've created each of those states, we'll also add a tag for that.
We'll come back to the tag later. Then we'll add the constraints for the recognizer
and move on to the next room.
Ok - now you see that we've recognized the utterance correctly,
but the important thing is to be able to do something useful with that information!
So remember back when we set the tag? That's what you would use programmatically
to recognize each of these commands. In order to show that we've done that, we
will change the recognition result, the text displayed,
to be something that we've generated - more like a response from the system,
and you can see that for each one you'll see whether it was recognized ON or OFF and which
location it was. And if we did have a control system here,
this is where we'd call the control system with the tag of ON or OFF
and then the tag of Location, and the system would take action accordingly.
Ok, that's great - but really we would want the system to tell us what it's going to do
and make sure that it did the right thing. And we would do that with Speech Synthesis,
So now, instead of just displaying that result, we're going to read it back as well.
In order to do that we're going to make a couple more changes
because Speech Synthesis wasn't part of this particular scenario in the demo.
We're going to add a media part to the XAML, then we're going to add Windows.Media.SpeechSynthesis
in addition to what's already there, Windows.Media.SpeechRecognition -
You can look both of those up on MSDN, by the way.
We're going to define a Synthesizer and then initialize it.
And now let's scroll down to where we recognized all those results.
This is where a successfull result is recognized and we display it out.
After we display it we will create some synthesized text based on that phrase.
That's pretty simple: we simply synthesize text from that text stream,
assign it to the media element, and tell the media element to play.
Ok, we'll go one more time down to our modified sample,
start recording: Turn on the guest bedroom lights.
"OK, I'm turning ON the GUEST BEDROOM lights."
Turn off the barn lights.
"OK, I'm turning OFF the BARN lights."
We just created a voice recognition system to control your home lighting.