JON LECROY: Hi, I'm Jon LeCroy, Outlook program manager.
And I'll be talking to you today
about a couple of new foundational technologies
that'll be driving new innovation in Outlook
for years to come.
Let's dive right in.
I'm excited to cover these two new technologies
for the first time today.
The first one is augmentation loop,
and then we'll move into
Outlook on the web powered experiences.
Let's start with augmentation loop.
This feature powers some experiences
you may already heard about in our products.
So let's take a look at them to refresh
what they actually look like, how they feel,
and then we can dig into what powers them.
Spell-checking isn't new,
but what is new is being able to see the synonyms
for each word so that you can pick the right spelling.
Editor also includes grammar and phrasing suggestions.
And with all of them, you are in control.
With text predictions,
Outlook helps you write better emails faster.
And by using the augmentation loop,
our intelligent services are able to suggest
typical responses to complete your sentences with,
personalized suggestions based on your own writing,
and as well taking into account message context,
such as who you're writing to, the subject line,
and even how you typically close out your emails.
So what is the augmentation loop?
Well, fundamentally, it's a new Microsoft service
that allows our clients to talk to our intelligent services
by connecting to one place
and passing a copy of the user context one time,
maybe this is the letters they're typing,
a paragraph they've updated,
a new slide design that they've applied,
and this service talks to all of our intelligent services,
distributes the context and collect the responses
and passes them back to the client in a standardized way.
So as the user is typing those keystrokes
for intelligent services that act on keystrokes,
they're sent up to the aug loop service,
sent to the relevant services that handle
that type of data, the responses, if any,
are sent back through the aug loop to the client,
and perhaps the user acts on them,
and that might drive their next change.
And then they make their next change.
And that change is sent up to the aug loop service.
And this is why we call it a loop.
We continuously are processing the relevant data types
so that we can ensure that we're providing intelligence
at the right time in the right context,
so that it is most useful to our customers and users.
So how did we get here?
As we added intelligent services to our ecosystem,
we quickly found that on the app side,
we needed to add new connection logic
for every individual intelligent service.
We needed to out our data protocol and transformations
and experiences and manage each of these
for every discreet service that we wanted to interact with
from a given client.
And from the service side
for a team that has built a new way
to add intelligence to our products,
they had to go and talk to every app across every platform
and work out all the same things.
So by introducing the augmentation loop service,
we solved a number of problems.
One is that we only connect once to our service.
We only have to pass the user context once
to our collective services,
and that the augmentation loop
facilitates getting this context out
to the relevant services that care about it
and getting there augmentation, so their suggestions,
back in a consistent way that goes back down to the client
into a generic store.
And this means that we're able to add new services
at a lower cost and it's less work and we can do it faster.
So for instance, editors inclusiveness grammar checker.
We already know how to do grammar checking,
but by adding logic and perhaps a service
for the inclusiveness aspect of that,
we can plug that in to the existing model.
We already have the context coming up.
It's already being distributed
through our augmentation loop.
We add the new service.
Register for caring about paragraph level data.
And from then on we can provide suggestions
so that when someone types gentleman's agreement,
for example, this service can look at the phrases
and keywords and social context of these things
and respond back saying perhaps unwritten agreement.
And all that's necessary
is to build the specific service context
and logic processing and models
and perhaps some UI handling,
but all the plumbing is there through generic types
that allow us to connect all of these things.
This service is a standard global Microsoft 365
highly available service,
and it's coming soon to Outlook for Windows as well.
So let's talk through how that works.
So to provide all of the rich formatting
and editing capabilities that you know and love in Outlook,
we are of course using Word and we have for
a very long time.
And so this means that Word running within Outlook
needs to eventually get
all of this user context around typing
and whatever else is happening
out eventually to those intelligent services.
So as you're typing the Word aug loop integration component
can take those for types that Word knows about
such as paragraphs,
send those to the augmentation loop common library
that is common across Office,
in this case hosted by Outlook,
which then knows how to connect to the service,
sends that context along, as we just talked about,
the aug loop service then runs that through
the different intelligent services,
comes back with suggestions or augmentations
rather than returned back to the augmentation loop library,
and then passed back to the Word AL integration component
and store it on the client for the UX to respond to
with callbacks and such.
So that if a new spelling check augmentation comes back,
then they know to draw the red squiggle and so forth.
There's an opportunity for client intelligence as well,
as it tees off to both the service
and any client's intelligence
such as offline spell-checking or anything like that.
And so this gives us an end-to-end way
to quickly provide new intelligence scenarios
in our Office and related Outlook teams and so forth apps
for low cost and consistent experience.
And of course, with all of the data
that we're talking about sending up,
these are optional experiences.
There are admin controls to turn these off,
group policy and so on.
So you're in control of your data there
and what you send and leave.
So that's aug loop.
So now let's talk about
Outlook on the web powered experiences, or OPXs for short.
As you may be aware,
we've recently rewritten Outlook on the web
using React and other modern frameworks.
OPXs give us the ability to pull features from web Outlook
directly into Outlook on native clients,
such as Mac and Windows.
We're using Edge's new WebView2 control,
which gives us a client library wrapper
for the new Chromium based Edge.
And that gives us a modern stable platform
to display and run our web app within our native clients.
By doing this, we can write features once in React
and surface them seamlessly in our native apps,
allowing us to release features faster and more consistently
across the Outlook family.
You may be familiar with Trident and Spartan,
Edge's WebView2 control is the successor to these.
Trident implements IE, they're not Explorer,
Spartan implements the pre-Chromium Edge
and WebView2 implements the Chromium-based Edge.
Let's take a look at Room Finder now
and see how this looks.
Here we are in Outlook for Windows
scheduling a meeting for which we want a conference room.
We click on Room Finder and the panel that opens
is actually a WebView2 control.
And this version of Room Finder
is actually the one from the web.
Here's web Outlook side by side to compare.
Unlike a standard eye frame,
we can select the room
and update the native code-based form,
as well as update the date and refresh the times
and available conference rooms.
We can even pass control messages
to do things like change the theming consistently
across both the native and web UI,
so that we have a consistent look and feel
by using web code within native Outlook.
So how does all this work?
So we have Outlook for Windows,
which loads and Edge WebView2 control,
which navigates to Outlook on the web
and loads a specific feature.
To start that, Outlook for Windows
contacts the Outlook servers
and gets the OPX configurations,
tells us the deep link
and other parameters necessary to load the OPX.
We instantiate the WebView2 control
and tell it to navigate to the deep link URL
that we received in the configuration.
Once this is set up,
the other feature begins loading in the WebView2 control
and we set up the control channel that I used
to change the theme and select conference room
in the Room Finder demo.
As we've seen the Edge WebView2 control
is a critical part of these OPX features.
So let's talk a little bit about
how this ends up on the system,
how it's managed and so forth.
If you'd like to try the OPX features in Office beta,
you will need either a pre-release Edge browser package,
such as Beta, Dev or Canary,
or to install the preview WebView2 standalone installer
that includes WebView2.
Once released in Office Monthly Channel broadly,
Office will go ahead and install
these standalone WebView2 installer.
This standalone WebView2 installer is itself
an application of its own.
It will stay up to date and keep the control up to date
independent of Office.
So if you have Office semi-annual
or another differed channel,
even as the Office bits maybe older,
based on older code, the Edge WebView2 control
will stay current and secure and updated.
Also the full package that ends up being deployed here
does not include anything that could launch Edge
as a browser itself.
So what Office will install
is effectively just the control for you
that's within Office applications
and does not allow users to run Edge.
So if you have, you will retain control
of which browsers are running on your end user system.
This is purely a control like Trident or Spartan
for use by applications.
Speaking of any application can install
this Edge WebView2 installer
and any other application can use it.
So if something installs it before Office does,
we will go and use it.
And if we install it,
other applications can take advantage of it.
It will become a dependency of Office and Outlook,
so we will need it to be there
and we will make sure that that happens.
As a side note, the Office web add-ins infrastructure
is also moving to WebView2.
So that's coming as well.
While we're here,
let's talk about supportability a little bit
as this is going to change a few things here.
Since we're loading web,
we will need potentially different network access
than you've seen from, you know, Windows Outlook.
And you might need some
different troubleshooting methodologies
for some of the features if they're not working
how you expect.
So say if the Room Finder doesn't load,
the first thing to try would be to load Edge Chromium
and try and go to OLA.
And if you can't get OLA to load in the full browser,
there's something that's not Outlook
that is in the way between that device and the service.
If it does load just fine in OLA,
but it doesn't load an Outlook,
then that's something Outlook or Office specific
and we'll have to dig in there a little bit further.
One way to do that, I'm happy to share some reg keys here.
This WebView2 control is enough of a full browser
under the covers without being able to launch it as such
that does include the dev tools,
but all the configuration of the control
is still wrapped in Office.
And so these are Office keys
that manage the use of the control.
So I've included here the reg value
for enabling the dev tools for OPXs.
So if you have focus on an OPX control, such as Room Finder,
you can set this reg key, restart Outlook,
click into that box and hit F12,
and you'll get the dev tools.
This may or may not be useful
depending on what the problem is.
But specifically I'm thinking of the console.
If something isn't loading,
if you have a firewall with some specific rules in place,
you can get some insight there
on perhaps what isn't loading in Outlook.
Maybe you have app-based firewalls
that need to be opened up, whatever it is.
This will just give you some more insight
into kind of what's going on there.
And then also the help menu we have troubleshooting.
And this is a system that talks to another Outlook service,
the Outlook diagnostic service,
and will pass some context from your Outlook client
up to that service, which can examine the context
and recommend potential mitigation.
So when we know we have a bug or there's a service issue,
this will actually tell you that
and give you some next steps.
So hopefully it saves some calls to support or frustration,
and you should take a look.
So we have one more OPX feature to share with you today.
And that is Meeting Insights.
If you've ever had to hunt down for meeting notes
or a pre-read that were associated with a given meeting,
you'll understand that sometimes that search takes a minute
or you have to try and track that down.
So we've included Meeting Insights
that uses the intelligence of our services
to pull in relevant content about that meeting.
So here you can see there is an email and a file,
and you can quickly access that now from the meeting itself,
using an OPX built in Outlook for the web
and pulled into Outlook for Windows.
Thank you, it's been great to tell you about
augmentation loop and our OPXs today.
We're very excited about these new technologies
and we will have a lot of new features to come
based on them.
Thank you.