TITLE: [MUSIC]
RIDDHI AMESER: Hello and welcome to
this Microsoft Build session.
My name is Riddhi Ameser and I am
a Product Manager on the Windows IoT team.
Today I'm excited to talk to you-all
about a new feature that we have
for our IoT Enterprise and that is called Dual Operator.
I'll start by giving you a brief overview of where
Windows IoT Enterprise falls
within the Microsoft IoT portfolio.
From there, we'll talk about how
this capability is positioned to work with
fixed-purpose devices that run Windows IoT Enterprise.
We'll then walk through the capability as well as
a projected user experiences and
the supported scenarios that
come with the various types of input.
Lastly, we'll take a peek at
how it all works under the hood,
and how you can develop solutions that will
enable this capability for IoT Enterprise.
Let's start by talking about where
Windows IoT Enterprise falls in the IoT stack,
especially since this capability is
unique to just this operating system.
Windows IoT Enterprise is
a world-class solution and the Microsoft IoT portfolio.
Microsoft's IoT portfolio is made up of two parts;
the intelligent Cloud services,
that are powered by Azure and a whole host of
devices that empower the Intelligent Edge.
Windows IoT Enterprise is a part of
the Intelligent Edge that supports MPU-class devices.
Diving a little deeper,
you can see that there are a wide variety of
devices that run IoT Enterprise,
from connected solutions to essential devices.
There are gas pumps, kiosks, medical devices,
voting machines, and
even industrial machinery that run
the Windows IoT operating system.
These devices can be classified as
fixed purpose as they're are
lockdown for a single application.
Because Windows IoT Enterprise has
such a large set of devices that are fixed purpose,
we offer many features and
capabilities to create a lockdown environment.
These features include assigned access,
multi-app kiosk mode, keyboard filter,
along with other policies that are central to how
the Dual Operator capability will function on devices.
This is why this capability is being
introduced to Windows IoT Enterprise exclusively.
We already have a portfolio of
solutions and operations that go ahead and
configure your devices to be in
this lockdown fix purpose state that
we will ensure that your experience is seamless.
Let's get to it. What is
this Dual Operator that I keep talking about?
Well, it's an incredible capability and
we're very excited to showcase this to you,
as it's been a demand from
our ecosystem and our partners for quite some time.
OEMs can go ahead and build
a Windows IoT Enterprise device
with dual input and dual monitors that have
an interactive experience for two users who will
simultaneously use the device
with two in-focus applications.
If you think about this,
you can actually have a couple of
different scenarios that might come to mind.
The ones listed on the screen
are of a point of sale device
where you have a cashier on
one screen and a customer on the other,
and they often will be doing different types of
input to go ahead and process their payment.
Maybe one's on touch screen,
maybe you have a cashier who's using a keyboard,
but essentially there's two screens and there's
two actions happening simultaneously.
This can also be done in a medical setting
where you have a patient interface on
one screen and a doctor interface or
maybe a hospital administrator
interface on the other side.
The idea is that the combinations in
use cases for a dual input are endless,
and we really encourage you to go ahead and find
other offerings and other ways so
you can go ahead and use this capability.
Let's talk about the enablement and how you can
actually get started with using Dual Operator.
Today we have a licensing agreement that goes ahead and
allows you to have
two users simultaneously using a device.
Traditionally, Windows client is a one user,
one device type of model,
and we are shifting that for
only fixed-purpose devices that
run Windows IoT Enterprise.
As you can see, we have updated our licensing language
as of December of 2021
to go ahead and support this capability.
It says, a single customer may be
locally and simultaneously interacted with,
with up to two end-user operators.
Of course, to assist with the enablement of
this capability as there is no in-box code,
but instead, we are leveraging
everything that's already built into the OS,
we have an open-source sample that
we'd love for you to go ahead and review.
This open-source sample consists about three parts.
The first is a sample Win32 application.
The second is a UWP application,
and the third is the Dual Operator utility itself,
which is orchestrating the whole capability.
We also have supplemental documentation
that goes ahead and showcases a lot of
the information that I'll be covering in
these slides to showcase how you can
go ahead and set this scenario and
capability up for your solution.
We also have a demo video,
which I encourage you to also check out,
showcasing this capability in action.
You can learn more and check out
this open-source sample at aka.ms/DualOperator.
Let's walk through the user experience for
Dual Operator and how a solution integrator
can go ahead and orchestrate
a solution to support this capability.
The open-source sample consists of three parts.
The first is a command-line utility
DualOperator.exe as well as two associated applications,
the Win32 and the UWP.
The Dual Operator utility here is a main orchestrator,
and it takes in the configurations that
the solution should abide by and it
needs to know which applications and what
are the respective inputs of that application.
This utility relies on
the Raw Input API and the Winuser.h functions,
which we will go ahead and review shortly.
We also have two applications that showcase
how different types of input are being received here.
If you look at the Win32 App
1 and the UWP Application 2,
you see that they look identical
because they have the same components.
The first component being buttons,
which go ahead and showcase a touch or a mouse input.
Then, you see the second piece,
which is a text box,
which could be for input of
a touch screen keyboard or
a USB keyboard, and then finally,
we have a free form area at the bottom,
which could be potentially
used for a signature or maybe a
picture that could be done by
your finger for a touch screen or even a Stylus.
Next, we're going to jump into the command line utility,
which is going to showcase how you can take in parameters
set by the solution integrator
for the desired experience.
This really involves specifying the application,
the type of input application it's receiving,
and if there's any associated device ID with
the type of input peripheral of your choice.
This screen capture showcases
some of the combinations that can be applied.
There's a lot more combinations,
I can assure you of that.
Well, let's go ahead and take a look at
State 3 and what State 3 has available.
State 3 has application 1,
which in this case is a UWP application.
It is taking in input via a USB type of device,
and then we have application 2,
which is going to be a Win32 app
which has an on screen keyboard,
and that is the input of choice.
The way that a user will go ahead and actually
invoke this type of configuration
is by selecting the DualOperator.exe.
As its utility, it'll go ahead and
select app 1 and the location
or file path for application
1 and say that it's a USB type of input.
In this case, they went ahead and included
the GUID for the USB.
Then, it went ahead and set
up application 2's experience,
which again gave the file path for application
2 and specified the USB type,
which in this case would also
be for a touch screen keyboard.
By going ahead and setting that up,
you're setting up an experience
that now the application knows
exactly what type of input it's looking for.
If you look back at the top,
we have a whole list of
different parameters that this utility
you will go ahead and take in,
and essentially, this is setting up
the configuration for how this whole scenario should be.
We have pieces called list, at the top,
which will go ahead and showcase
all the various 16 states
that this utility can go ahead and provide,
as well as what application 1 needs,
application 2 needs,
the different types of input,
and then lastly, how touch works.
Touch capture, specifically, no sending,
which means that you don't want to send it
to the application itself,
will go ahead and tell us so we don't
need to move forward with that.
Touch is a very important part
of this whole entire experience,
and we'll cover the complexities related to
touch screen input in a couple more slides.
As you can see, we have two fields for touch.
The first being touch capture no send,
and the other being touch capture send,
which means which application and
where in the application do you
want the touch to be sent to
or the touch input to be sent to.
As you can see,
it's a little bit more involved
as you go ahead and set up the experience,
but once you know exactly
which peripherals you're going for,
and exactly which applications they should route to,
it's a pretty smooth ride.
As we focus on the user experience that we're trying
to bring from this capability,
you can go ahead and take a look at the picture,
which we'll hopefully go ahead and
illustrate some of these components.
Input from User 1 is running Application 1,
which is displaying on Monitor 1,
and that's all highlighted in blue.
Then we have input from User 2,
which is running Application 2
and displaying on Monitor 2.
Again, the crux of
this whole thing is that it's running only on
one Windows IoT Enterprise device
box compared to running
two different boxes or having
completely different scenarios that
then have to interact with each other.
We're trying to bring everything into one box
and have two applications running simultaneously,
so two users can go ahead and use
a single Windows IoT device.
Now, we're going
to get a little
bit into the nitty gritty,
and really under the hood to understand
exactly how we're making this magic happen.
We have some applications which are supported,
which I mentioned earlier,
we have a Win32 as well as a UWP.
We are allowing one Win32,
and one UWP as well as
two Win32 applications to be simultaneously run.
There are two important APIs that are
integral to making the magic happen with the solution.
The first is a raw input API,
and this API is responsible for the USB peripherals,
and the on-screen keyboards in
making that functionality work.
Then we'll go ahead, and take a look at
the register pointer input target function,
which is a part of the Winuser.h
class, which will go ahead,
and help orchestrate the touch input
that comes in with the solution.
Now, when you go ahead, and leverage
these APIs, as I mentioned,
they're used for two different types of input styles,
your raw input is really going
ahead and defining that input source,
whether that be the keyboard,
the mouse, the touch screen keyboard,
and the API will respectively
direct the input to the specified,
and designated application,
and specifically it'll go ahead,
and ensure that it's being redirected properly,
and that's the API that we're going to go
ahead and leverage for these types of devices.
Commonly today, this is actually used in head macros,
if you're familiar with that, I'm not sure if we
have some flight simulator fans,
but essentially that same remapping of keys,
and remapping of functions is what is
relied upon for raw inputs,
specifically of this Win32 API class.
Next, we have the register
pointer input target function,
which is a mouthful to say,
but it's really used in Microsoft's narrator service,
which is actually how we discovered it.
We did a hackathon not too long ago,
I believe, last fall,
where we actually work with folks from
the accessibility team who use this API,
and actually use this type of
classes quite frequently to go ahead,
and set up different types of
services for folks that are
going to need enhanced experiences
when it comes to Windows client.
Really, when we went ahead,
and took apart the narrator service,
we understood what was actually happening,
and that was that they have a boundary on
the application itself or on the windows shell itself,
and are able to go ahead,
and redirect any of the input that happens there to then
be registered by the application
before it actually hits the OS input stack.
Rather than actually hitting a browser,
for example, the narrator service
will tell you that you're hitting the browser,
and the action won't actually happen.
This redirection when it comes
to touch is an integral part of the solution,
just like the redirection that happens with raw input,
which is again making sure that any of
the keyboard strokes are making sure
the touch screen keyboard touches that you're doing
are being registered, and redirected properly.
Essentially our raw input,
and our register pointer
input target function are there to go ahead,
and ensure that our types of
input peripherals are being targeted,
and also redirected to the appropriate application,
that's pretty much the first component of this.
Now, that we're redirecting,
I feel almost like a conductor when I'm doing this
as we're starting to orchestrate a solution together,
and really what is helping us orchestrate or almost
conducting the solution is the User32.dll,
and now what this does is it goes ahead,
and actually looks at what's on the input stack,
and our command line utility will tap
into this dll to figure out,
and filter what various inputs we're looking for,
and what the OS is seeing,
and how we're able to filter that out.
For example, if our input
is being targeted for an assigned application,
so I'm only looking for all keyboard input,
we can go ahead,
and ensure that's flagged,
and then given the authority to go ahead, and act.
Oftentimes,
there might be an accidental touch screen touch,
or there might be a mouse click that
wasn't specified for that application,
and because that is not specified by the utility itself,
it will go ahead, and be ignored.
Essentially, we're listening in,
we're making sure all the right functions are
happening as a conductor, and we'll,
as a result, appropriately act,
and the application will
appropriately act to make sure that we are good to go.
But before we get excited,
I'm sure you guys are thinking, but really,
what about focus,
how are we going to go ahead and make sure that
two applications are not bouncing back,
and forth as a user decides to
click on one screen or do a touch screen in another,
well, this is what brings the beauty
of Windows IoT Enterprise into the picture.
We have a lot of fixed purpose devices,
and there's a whole talk about that in
a Joe Koko's session of the Windows IoT Roadmap,
and overview, as well as in the security sections,
we have a lot of different features that go ahead,
and build Windows IoT for what it is.
As I mentioned, the fixed purpose solutions
earlier in this presentation,
a signed access,
keyboard filter, those go ahead,
and actually put into
frame how these devices are being used today.
We envision our OEMs using Shell Launcher V2 or
Multi-App Kiosk Mode to really fit this experience,
and what that really means is that
this application is going to be full screen,
and it's going to be locked out in an environment.
When you are going ahead as a user to potential users,
you might be doing it at a cash register,
you might be doing in a doctor's office where there's
already one application that you're focused on.
Unfortunately, when you're going shopping,
you can't open up the Internet edge browser,
and so to go shopping,
you really have a locked down experience
from the get go,
this is why we're focusing this capability to
be targeted for fixed purpose solutions.
Essentially, this also helps us from
an engineering perspective as
both applications being in full screen,
we know exactly where the inputs are being targeted.
We know if it's happening on Monitor 1 versus Monitor 2,
which again helps us isolate these two experiences.
With the applications in full screen,
you're not getting that browser back,
and forth or that big gray bar tab bouncing back,
and forth because you are in
a full screen mode experience.
With that, let's talk about our supported scenarios,
and how you can use the dual
operator utility to go ahead,
and bring to life some of the solutions that you have
depending upon the input styles of your choosing.
In our supportive scenarios,
we have two matrices here,
and before we jump in and look at
exactly which types of input you should use,
pay attention to the types of
application styles that are being used.
On the left we have a UWP application,
and a Win32 application,
and on the right we have a matrices that shows
the various input styles for both Win32,
and another Win32 application.
Overall, the command line utility logic will go ahead,
and assure you that if there are
any unsupported solution scenarios,
that you're not necessarily
going to be able to configure it.
But because this is an open source solution,
we're more than welcome to go ahead,
and have you investigate,
and hack into it to see if
you really want to go ahead, and make it work.
The sample that we provided today,
will only be able to provide
the solutions that are supported,
but we really encourage you to help us develop,
and build this capability further.
Now, taking a look at UWP,
and Win32 Matrix, you have the ability to go ahead,
and pick, and choose some supported scenarios
as well as showcase,
which are some of the unsupported scenarios.
As I mentioned, we want you to go ahead,
and see what works for your scenario,
and maybe you can contribute to
the open source GitHub sample if
that is something that you're looking
to actually configure.
For example, if you decide to use
a Win32 application with a USB peripheral,
you can't use a UWP application with
touch bar as that solution is not necessarily supported.
However, there's many other options for how you can make
touch buttons work as well as USB peripherals work,
we encourage you to go ahead and look at the matrix
or hack the matrix
to make sure it works for your solution.
Again, we have focus in here as well to go ahead,
and showcase to you on the back-end really what
application is in the foreground
versus in the background.
The utility as a whole, we'll go ahead,
and manage that for you
since both applications will be in full screen,
but in case you're curious,
that is what's happening in terms
of the focus aspect of things.
We also have a very similar table or
matrices structure for two Win32 applications,
now this has more supported scenarios,
but we still do not necessarily offer
two touch keyboard offerings at the moment.
But again, you're more than welcome to hack it up,
and let us know what you find,
we're very excited to hear from you all.
But in the end of the day,
we really want you to go ahead,
and leverage these functionalities in box,
especially if you decide to use
the Win32 applications as you have
a plethora of different things to do.
All of this documentation will go ahead,
and be provided at our GitHub link,
as I mentioned earlier,
but hopefully, this walkthrough
was helpful for you to go ahead,
and understand how to use
a dual operator capability today.
With that, we love for you to go ahead and give
Dual Operator a try
today, the link is ak.ms/DualOperator,
and we'd love for you to go ahead,
and try the open source sample,
and for you to share the various customer solutions,
and scenarios that you've unlocked with this capability.
If you're interested in learning more
about the Windows IoT platform,
and the amazing features that our team has developed,
we really encourage you to go ahead,
and check out some more build sessions.
We have one on our roadmap,
on our security, on Kubernetes,
especially as some other
offerings that we're going ahead,
and putting together for our comprehensive platform.
We really encourage you to go
ahead and check it out if you're interested,
and we really hope that you
having a wonderful time at Build.
With that, I like to thank you for your time,
and we look forward to getting your feedback,
have a wonderful Microsoft Build.