>> You're not going to want to miss
this next episode of The AI Show,
where we're talking about enabling
productivity tools using language understanding.
It's very cool.
I get to do the demo because
they teach me how. Make sure you tune in.
[MUSIC]
>> Hello, and welcome to this episode of
The AI Show where we'll learn about enabling
productivity tools using language understanding.
Did I get that right?
I did. Fantastic. It was a mouthful,
but it's pretty cool.
We've got some special guests with us.
Why don't you tell us who you are and what you do,
my friends? We'll start with you, Alan.
>> Yes. I'm Alan Liu.
I am in the Office Machine Learning team,
and what I do is I work
on machine learning models to help improve our Office products
and enable intelligent scenarios
that allow our customers to be more productive.
>> Awesome. Nayer.
>> I'm a part of the Language
Understanding Cognitive Services team,
from the engineering team,
and I work with Customer Success to make sure that
customers using this service
are successful in achieving their goals.
>> Awesome. Well, let's start with you, Alan.
What problem was the Word team trying to solve?
>> We are bringing a next-generation version
of dictation into our Office client,
starting with Word Online as well as on mobile.
Word, you're able to use your microphone to
dictate text onto your documents,
and we wanted to add more powerful features to support
the most common commands that people do as well.
Things like formatting their document and editing.
>> Let me see if I understand because I've used
dictation before and I've loved it.
But anytime I actually wanted to do something with the document,
it was hard, and what you're doing is
you're enabling Dictation plus plus.
Am I understanding that right?
>> Yeah, that's correct. We're looking into how we
can allow you as the customer to be able to
still continue to use your voice or work in
a multi-modal situation where you
have both voice and mouse and keyboard,
and being able to,
in a more natural way,
tell Word what it is that you want to do with your document.
>> Now, is this something that's available now?
Can people go and try it?
>> Yes. We released to the public a few weeks back,
and so if you go into Word Online,
or into Word for Android or iOS,
you'll have the new dictation experience available to you.
>> Instead of a demo, would it be okay
if you could like teach me how to use it because
I got a ton of Word documents I got to get
through and maybe this will help me. What do you think?
>> Yeah, let's give it a shot.
>> I already have opened on my screen right here, a Word doc.
You said it works Online and in Android, you said?
>> Yeah. Android and iOS.
>> Fantastic. So what do I do?
>> On the top ribbon you see near the right-hand side,
there's a Dictate button with the microphone icon.
Yes. If you click on it, and often,
for a first-time user in the browser,
you'll be asked to allow the microphone.
Otherwise, you can just start talking,
and whatever you say will
be put on Canvas until you start doing commands.
>> So all I need to do is literally just start
talking and everything's going to work? Oh my goodness.
This is neat. Everything I say is
literally going to be written down so I can say anything,
and it's going to type it out for me in a really nice way.
What about punctuation?
Do I have to say punctuation?
Because I know I hear people saying comma and period all the time.
>> The floaty that appeared at the bottom of your screen,
which has the gears icon and the mic.
In the gears icon,
there is an option to turn on auto-punctuation,
where our speech to text service,
powered by Azure speech recognition,
will try to intelligently decide what to punctuate.
But yes, you can also just say the punctuation you want.
For example, if you want to say comma or
a semicolon. All those things.
>> I also noticed that I was being quiet,
so it stopped listening. Is that right?
>> That's to balance
the fact that maybe people will forget that they
have it on and it's continuously trying
to record and figure out what you're saying.
I think there's also in the browser, especially,
if you lose focus,
then it automatically disables the microphone.
>> I'm going to turn it on, it's like "Okay, you stopped talking."
This is really cool, period.
I would like to use this feature a lot
more often because sometimes
I am better at speaking than I am at writing, period.
This is cool. Because I know you could do this before,
what is it that's actually new?
>> Some of the new commands that we've added
enhance the capabilities of the previous version of dictation,
which just matched a few key phrases like bold,
and underline, and things like that.
Now, what you can do is you can say things like,
"Bold that" which would bold the last thing you said,
or you could say things like "Bold the last sentence,
or the last word, or the last paragraph."
So it makes the commands more powerful and flexible to use.
>> Let's try that. Period. Bold that.
>> In that case, period was the last thing you said.
>> Oh my goodness.
This is really cool. Bold that.
Yes. If I want to say something even more important,
comma, then I'm going to say this, underline that.
Well, I'm going to stop right here.
It looks like it finds the last punctuation
and it intelligently underlines it as well. Is that right?
>> With the bold that,
it's actually figuring out the last chunk that you said,
because what happens when you've sent
your speech out to Azure speech recognition,
it's constantly deciding what's
a chunk of text that it will produce.
That's why, depending on
your cadence and where you put your pauses,
we also let you just say you want to bold
the last sentence or format the last sentence.
We also have a way to say if you have a particular phrase that
you want to format or you want to delete,
then you could say something like delete,
and then the phrase that you wanted,
or delete from some beginning word
or phrase to or through some beginning word or phrase.
Those are [inaudible].
>> I love you advanced commands,
what are some other commands that you can do?
>> Maybe the most important command
is you can actually just say, "Show help."
I think you could also ask, "What can I say?"
>> What can I say?
>> I think that might have already.
>> It's already over here. It's literally
going to come out of here. Got it.
>> Yeah. What you see is
a brief list of the commands that we officially support.
We're adding new commands continuously,
but we don't want to overwhelm the beginning user these features.
So what you see in the help pane is
collection of the most common basic commands that you could use.
So beyond formatting and deleting,
you can also do things like creating a list.
If you said start list or next bullet, next line,
or create numbered lists,
and then you can also add comments into a document.
With add comment, you can actually just say whatever it
is you want to be in the comment and in one shot,
it will insert it into your document.
>> Well, let me try a couple of these things
because this is really cool.
Next line, start list.
First bullet point.
Next line.
Second bullet point.
Exit list. Add comment.
This is the greatest thing ever.
What?
This is bananas.
This is really cool.
I said bananas because bananas are amazing,
but this is really cool.
Basically, it's not just
saying things into it and having it written out.
It's intelligently being able to dictate to
it as if it was my own personal assistant,
John, that takes my notes, for example.
>> Yeah. That's the experience
that we're working toward to make it something
that doesn't really break the flow
of whatever you're doing as you're authoring in this scenario.
The commands that we want to support are ones
that we want to make natural,
so you stay in the flow of being productive.
>> Let's go to you, Nayer.
How is something like this accomplished?
Because you work on the cognitive services team.
How is something like this accomplished?
>> The language understanding cognitive services allows
you to pick the speech to text and
use AI in order to do the extraction to really understand and
distinguish between what is dictation and what is a command.
Allows you to commit these different variants of commands,
like bold from, to,
or through to that, and extract that for the action to happen in
the office environment with limited set of data.
You don't have to either dictate this as
rules in previous experiences with dictation,
which had a very fixated grammar,
but it gives you the flexibility of trying to
design this with limited set of examples,
and leverage the power of conversation into such an experience.
>> Speaking of the speech service,
it was intelligent enough to know boundaries,
which was really cool,
but the thing that's even more interesting to me is that
there's not fixed words.
You mentioned that there's not a fixed grammar.
How do you accomplish that?
For example, if I say end-list or for example, next bullet,
it knows what I want to do,
but I don't have to stay exactly the right thing.
Is that correct, Nayer?
>> Yeah. You train with different variants and it learns how to
capture these subtle differences in through the training,
and the common servicing,
unlike all AIs, conversational experiences continues to improve.
We went through a process with Alan in
the crew there to make sure that this continues to improve,
captures a variety in a wider spectrum of changes.
There are different ways people express these elements.
It does generalize for them to be able to encompass them.
>> Look, I've done dictation before.
It's been around since well, like forever.
But I've never done a dictation where I could feel
like it understood what I was wanting to do.
I didn't have to memorize some thing or talk like a robot,
because I felt like I had to do that before.
But this felt a little more free flowing, which is really cool.
By the way, this side thing,
if we want to go to my screen here,
this side help thing is pretty
amazing for telling me what I can do.
As I see this, what are some things
that we're thinking about moving forward,
Alan, for this feature?
>> Expanding on what you and Nayer were discussing
about how can we generalize our language understanding,
so that you don't have to say
exactly the thing that we expected you to say.
That's something that's an area of improvement
for us because initially,
we have to figure out the balance
is going to be because if you are
actually intending to dictate something that sounds command-ish,
then it could be
potentially those false negatives could be a little bit jarring.
Imagine if you really wanted to
talk about bold initiative that your company
is undergoing and then suddenly it
decides instead to toggle bold on for your document.
That would be something that would definitely be
an annoyance or where we would fail.
What we want to do is continually improve,
so that we can support
more commands and more natural ways of saying that,
but at the same time,
making sure we don't go so far as assume everything you say is
possibly a command when you actually wanted it to be dictations.
That's where we want to go with that.
In the near-time, we're also
unlocking more capabilities, more parameters.
I mentioned that we recently added support
for operating on a phrase or a range,
but we also want to have some limited support for numbers.
Today, you can say delete
three or backspace five, and that would work.
But actually our language understanding
can support a lot more than that.
If you wanted to italicize
the next three words or the previous five paragraphs,
all of that we can understand.
It's just a matter of being able to add them into the product.
I think even more in the future,
what would be great is to be able to
understand personal styles for people,
not just in terms of the commands,
but what they're working on.
That maybe even more powerful commands, like go,
jump to the summary, or the document,
or make a table of contents can be actually understood.
Rather than just telling the dictation step-by-step,
some of these individual commands,
we might be able to do almost like macro style things.
Definitely in the short term,
since this is a new feature,
we're just interested in proving it and squashing any bugs,
improving the fit and finish,
and getting feedback on what it is that
our customers want out of
the feature in this scenarios that they want.
>> That's awesome. Nayer, as I'm contemplating this,
we hit upon speech to text.
But my sense is that there's more of
a robust pipeline of
cognitive services interacting in the back end.
Can you tell us what services they're using and how
customers can take advantage of
it for something that they might want to build?
>> Mostly, it comes from the custom speech,
the ability to train custom speech models
in order to do the extractions,
and then the language understanding cognitive service in
order to extract the elements
and then you jump into your old code.
To do that, there are
some other cognitive verticals that integrate
these solutions to enable
conversational AI experiences that are more out of the box,
like custom commands, for example.
Integrates the custom speech with
custom LUIS models in
order to enable scenarios for command and control experiences.
There are other cognitive services that
verticals that try to blend the fundamentals,
cognitive services to speech,
visual, language, and decision into verticals
or applications that are usable by the end-user immediately.
>> Well, this has been amazing, my friends.
I love learning about cool stuff that I didn't know
about that I can literally go and use right now.
Alan, where can people go to find out more?
>> Since the feature is available right now.
If you just go to Word Online,
you should have the dictate button at your disposal.
Remember, for new customers,
one common pain points seems to
be getting that microphone access allowed,
otherwise it won't work.
Therefore you need to do that.
>> Absolutely. Nayer, from you,
where can people go to find out more about
these awesome services that they can use in their own products.
>> For cognitive services,
you can go to cognitive services, aka.ms/cognitive services.
Specifically for the language understanding,
you can go to Luis.ai.
>> Well, thank you so much for
spending some time with us, my friends, Alan, Nayer.
Thank you so much for watching and learning all about how to
enable productivity tools using language understanding.
It's pretty cool. Thank you so much for watching.
Hopefully, we'll see you next time. Take care.
[MUSIC]