>> I'm going to spend the next 15
to 20 minutes talking to you
about Building Windows Mixed
Reality Applications with Unity.
So, hopefully people here are familiar with
Windows Mixed Reality or Virtual Reality.
If not, I'm going to give you a really quick intro
and then we're going to talk about building
experiences with Unity.
So, for those of you not familiar with Mixed Reality,
that is the term that Microsoft uses when we talk
about the spectrum of experiences that we can have.
On one side we have physical reality where you
are probably familiar with augmented reality experience,
and on the other side we have
pure digital reality where you're familiar with
potentially virtual reality where you'd put on
a headset and they take you
somewhere else to a different world.
So, with Microsoft we use
the terms Mixed Reality because we
want to have experiences that
expand across the whole continuum.
You can write one experience that goes from
augmented reality all the way to virtual reality.
So, that's the term we use that's
just some terminology for you to be familiar with.
What does that mean for you with
regards to a user experience?
Well, you can explore your environment,
you can be right here in this room
and interact with the floor,
interact with the walls,
you interact with everything else and bring
digital objects or holograms into the space.
That's why we call it Mixed Reality.
So, on our site today,
I'm going to show you the immersive headset which are
the virtual reality headsets. These are occluded.
With these we can take you to
a different world instead of with maybe using
the holograms which you're probably
familiar with where we show you
holograms that come into
the physical world from the digital world.
So, today we're going to talk about
the Mixed Reality or immersive headset
because last October we launched with
partners five or six new different headsets.
So, today I'm going to show you the Acer,
but Samsung has a headset,
Acer's has a headset, HP and Lenovo,
they both have virtual reality headsets that you can use,
and it's the exact same platform.
So, everything you learn about developing a game today,
for Mixed Reality, is going to
apply to any of those headsets the exact same.
You don't have to touch any of your code.
So, how is our Mixed Reality
offering different from everybody else's?
Well, the one thing that we have is
we're very, very easy to set up.
If you're looking at here, this enviroment.
There's no Lighthouse's,
there's nothing tracking my controller.
We're actually using the inside
out tracking technology that we have in HoloLens.
So, to get this connected all I did was connect
my HDMI port and my USB port and that's it.
So, I can carry these to 30 feet from now,
I can take it back to my hotel room and I
don't have to have any loud assets or anything set out.
That's what Microsoft is doing with
our Mixed Reality experiences and that makes
it really convenient for some scenarios
like commercial, enterprise scenarios,
training, users who don't
want to put Lighthouse's in their living room,
they can use these and we get
the exact same experience as
a virtual reality without having to make any compromises.
The other thing that makes us different
is that with our platform you write
the application once and it
can run across a myriad of devices.
So, for example if you have
an Xbox game and you want to compare it to Mixed Reality,
you're using the exact same platform,
the exact same tools,
pretty much everything that you're already familiar with.
You're submitting to the exact same store
and you're using everything that
you're already familiar with.
So, we have one ecosystem
that you would actually be able to use.
Same thing for the store, we have one store
where every user will have the same store.
So, for example, Windows 10 has
500 million users out there that are
constantly going to the store and then downloading like
casual games or they're downloading enterprise app,.
but it's the exact same store.
So, when they go to the store, they
can discover your game and
find it very quickly and then install it from there.
So, it's a really good way for you to get discovery.
And they can buy other things like
books and media, videos.
So, the store is a place that they visit very often.
So, it's a good way for you to get discovered.
So, that's what Mixed Reality is.
So, now let me show you how easy it is to take
a Windows Unity game
and actually port it to Mixed Reality.
And the game I'm going to show you today,
it's called Laser Cop and it's one
of the Unity samples so that everybody here who is
familiar with Unity can go download
the sample and see it and see how easy it is to port it.
So, before we start, one of
the things that you might want to know is what are
the prerequisites for you to write
a Mixed Reality game or for a user
to run your Mixed Reality game, so it's very simple.
All we need is Windows 10 which is the latest version of
Windows the Fall Creators Update
is the ones we released last October,
and that you didn't even have
the latest version of Windows.
And then you as a developer will need
Visual Studio which you can use
community edition and it's free.
And then in this case we're going to talk about
Unity so you can use Unity which
again also has a professional license
but they also have a personal license that you can use.
Do not have to pay anything for you to use it.
If you're using as
an indi or you're a student or something
similar where you're not using it for
commercial use from a large enterprise.
So, getting started is super, super simple.
So, with that let me show you what our game looks
like and how we would take it in
Target Windows Mixed Reality.
The game right here is in Unity.
And what I have right here
is this game, it's called Laser Cop.
And first, I'm going to show it to you as
Unity ship set which is for Gear VR.
So, before I make any changes
you'll see the game as we play it.
So, I'm going to go right here in to Unity.
And just out of the default I'm going to use the game,
it's written for Gear VR, but
I'm going to try to get it to run in our headset.
So, what you have to do in order to
get it to run on our headset is very simple.
You go into Unity and just change your build settings.
So, I can come back here and set "File" "Build Settings."
And here in this case, if it was Gear VR it
would say it's for Android for
Gear VR or if you were
Steam VR or Rift, you would say it's for PC.
And all I have to do is select right here
"Universal Windows platform" and
that's going to make it target
our Universal Store which is
again what we use for
any universal app whatever its HoloLens,
Mixed Reality or Xbox.
And then normally you would just do switch platform here.
I already done that for you because that takes
a few seconds and that
automatically makes it target Windows Universal platform.
And then the last thing that I want
to do is go into "Player
Settings" and tell the game
that we're going to use Mixed Reality.
So, the way you do that is you go
right here into "XR Settings".
And then you just check Mixed Reality.
Here, if I have multiple ACK like
for example Rift or Steam,
I would actually have those ACK's listed here.
In this case, we're only going
to default to Mixed Reality so
that's the only one we're going to use. And that's it.
Those two settings are all I have to do in
order for me to be able to play the game in my headsets.
So, now you can see what the
game-play is like in the headset.
So, I can just go back here in
Unity and then just say "Play".
Obviously, this is going to be the
editor play experience but you'll see right here.
It's going to come
up and hopefully you see part of the game.
You can see only one eye.
[inaudible] the two eyes in the headset.
But you see the game right here.
And you see that for some reason,
it didn't find my controller.
So, you see the game right here
and you see that for example,
right now there's no controller
because this game was written for Gear VR.
So, even though it's doing the tracking,
the controller model is not listed there.
And the controller is still interactive where
I click "Play" and do all the right things.
So, in this case,
that's the game as it started.
And if I go back here,
and show you this experience right here,
you'll see the Gear VR controller.
So, I'm just going switch gear to the original scene.
And you'll see the exact same scene.
Oh, Unity just crashed.
Sorry. I have a 2017 version
of Unity which should be pretty stable
but for some reason it crashed that time.
So, let me just restart Unity real quick.
That has nothing to do with Mixed
Reality that's just Unity.
Do you guys use Unity? So, you know that that's true.
It doesn't happen that often, but it does happen.
So, I'm going right here, exact same experience.
Going to Unity. I'm going to open
the original experience, the original scene.
And just make sure the settings are the same,
which should have saved
automatically, because they were already saved.
And now you can see the game.
So, you can see here, this is actually
now showing you the Gear controller.
So, even though it's actually using
a Microsoft controller,
it shows you the Gear controller
because that's how Unity rotate.
So, you can see here, I can click play
and then it turns into a lifesaver.
And then the game is called Laser Cop.
And it's about these people that are attacking me.
For example, they're sending me these
bombs and I can just kill them.
And then these drones are going to shoot
every now and then,
and with laser, I can deflect what they are shooting.
So that's what the game is about.
And we're going to just put it
very quickly to Windows Mixed Reality.
So I'm going to stop it there.
Basically, that's Gear VR game,
and then we're going to convert
it to Windows Mixed Reality.
We were already down one half of the equation which
was targeting Windows Mixed Reality.
The next thing that I'm going to show you
is how to do controllers.
What you have to change when you
code in order to use
the Windows Mixed Reality controllers.
And before I do that, in code,
I'm going to give you a little bit
of the options of how do
we target so that you
get familiar with the API's decoder.
So some people ask, well,given that
Microsoft is using their own tracking
and it's inside out tracking,
do I have to do anything different for tracking to
Mixed Reality. And the answer is no.
You use the standard Windows API as you were
using them for Gear or for Rift,
and everything work the exact same.
So you don't have to change anything for the headset.
The controller as you saw
earlier in the game that I show you,
it was already tracking automatically
and it was doing everything as if it
was a Gear VR headset or a Mixed Reality headset.
There's nothing special that you have to do.
When it comes to controllers,
we do give you a few more options.
Our platform ships with
these Windows UI special interaction which is what you
would use if you were writing
a direct text app or your own platform,
your own game, maybe an OpenGL game.
But since you're using Unity,
you can use a Unity API.
So there Unity gives you two options.
One of them is you can use the Unity input tracking APIs.
If you've ever written a Gear experience a
Gear or a Rift or Steam experience,
you've actually use that API.
That's the open XR API
and that still works with our platform.
So that means you shouldn't have
to change your game at all.
Now, if you don't- if you want to
use our API, the Windows APIs,
you can also use the new API that Window
offers that Unity reps call interaction manager.
And that gives you a little bit more functionality.
In a minute, I'm going to show you
what the actual functionality is.
So there are just a few extra features that you can have.
And also for those of you who are using SDKs like
VRTK or Newton VR or
any other open source
SDKs out there that people use for riding
cross platform Windows or virtual reality experience.
You can use any of those the same.
We support for all of them with Windows Mixed Reality.
So with regards to what interaction manager gives you,
they just give you an extra post.
For example, you have
a pointer post that comes out of the pointer right here,
and a grip post and
that's the only thing that's different.
Today, I'm going to use the grip post so that you can see
the functionality exactly as
it comes out of the controller.
So let's go back to experiencing and add
a couple changes so that we can use our controllers.
So I'm going to go back here to Unity.
And this is our original experiences as we had it.
So let me show you what
the Windows Mixed Reality experience
looks like and when I change to
make the controllers work.
So here I'm opening the exact same scene
and the only thing that I've done
is made a couple of changes.
Here on the player ring you see they
have the left hand where the controllers were.
And here they had
all these physical game objects
that were the Gear VR controllers.
So all is I disable all of these.
So I hit all of the Gear VR controllers and then I added
these track controller script
that comes from the Windows Mixed Reality,
the Virtual Reality toolkit that we have
which is Microsoft Mixed Reality toolkit.
And all I did is added the functionality to
the controller and then I told it which hand to track.
In this case, I'm doing the left hand
and then I told it to render the system controller.
And the reason we do that is because in our case,
because Microsoft has headsets that come
from Samsung and HP and Acer,
the controllers or the headsets
can be slightly different.
So we're going to render a controller model,
you want to query it from the system and
tell the system to give you the controller model.
And that's what this script does.
All it does is it actually goes in and says as a driver,
"Hey, give me the controller model."
And then it renders it on Unity.
So it's trivial for you to do.
It's a few lines of code for you to
add and then that's all I had to
do for us to have
Mixed Reality support within this Laser Cop game.
So now I can come back here and play the game.
And you'll see that now.
We will render the controller and you can see right here,
now it's rendered to Mixed Reality controller,
and all the animations and everything else is there.
And the controller still works.
For example, if I click play again,
it turns into a [inaudible] and everything has worked.
And there we have the controller model.
[inaudible] Actually comparator [inaudible]
can have Windows Mixed Reality in the store.
If you have a game for Steam or for Rift or for Gear VR,
it should be pretty easy for you if it's written in
Unity to convert it into Windows Mixed Reality.
And now you have a game
that's ready for Windows Mixed Reality.
You can submit it to the Windows Store and get
a lot more users to see your game and
to solve your game and give you the money.
So that's how easy it is to convert
Mixed Reality game and that's what our platform is about.
So let me go back here just quickly.
Now, obviously, I want to keep it very short.
So I only show you how to do controllers
but there's other things that you would have to do.
You saw earlier that
the headset tracking works automatically,
so you don't have to do anything extra for gaze.
But gaze is something that we use in Windows a lot.
So make sure that you use gaze when you write your game.
Another thing that we have is voice.
With Microsoft because of HoloLens,
alot of our users that use
Windows Mixed Reality are used to
talking to the controller- to talk into the headset.
So make sure if you write
your game that you implement voice.
It's super trivial. It's again
a few lines of code just like the ones
that I showed you to convert it and to do Mixed Reality.
On top of that, the next thing that you would want to
do is maybe to replace your assets.
For example, you want to replace your splash screen so
that when your Windows experience
comes up you get a nice Windows flash screen.
We have this thing called 3D
launcher which is something that you see in Cliff House.
Let me show you what Cliff House looks
like so you could be familiar with those a little bit.
So I'm going to go right here
and I'm going to navigate into our Cliff House.
If you've never seen what the Windows Reality home,
this is what the Windows Mixed Reality home looks like.
From right here, you are right here in
the Cliff House which is where I put all my applications.
And navigating with the controller,
I can do things like teleport,
I can interact with everything in the Cliff House.
For example, I can interact with this object right here.
And it's animated, and
that's actually an icon for an application.
So that's what real launchers are.
So, for example, right here,
I can be in
there and I can navigate within the Cliff House.
And you can see here, for example,
this is the Hologram's application,
and that's a 2D slate as we call it.
So that's the icon for the application,
but that's a 2D icon.
You can also get 3D icons.
For example, if I walk right
here into the immediate room,
you can walk right here and you'll see for
example this is the icon right here for
the Hulu application or for
example that's the light icon for John VR.
And this is the icon for the White House,
the people's house,
which is a video experience
that you can use from Felix and Paul.
And so that's the icon and you can
see here I can interact with them.
I can move them around. So that's what 3D launches.
That's one of the things that you would want to do.
If you put your application to Windows Mixed Reality,
you want to have a really nice 3D launcher that people
can use to decorate their Cliff House as they go.
So that's one of the things that you would have to do.
The rest of it, the last thing that I want to highlight
that you would want to show from your experience,
if you're going to portray experiences.
With Windows Mixed Reality,
we actually support a very wide gamut of hardware.
So if you're familiar with Steam or Rift,
mostly with Steam you're only
supporting high-end hardware like Steam only
runs in GTX 970 video card or above.
With Windows Mixed Reality because we have
a broader set of experiences and a broader set of users,
we allow users to run Mixed Reality in low-end PCs.
So, for example, if all you're doing is
rendering a 360 video experience,
that can run fine on an Intel GPU,
and Windows Mixed Reality will you do that.
So when you put some media application to our store,
you need to make sure you decide,
hey I want to target what we call a mainstream PC.
And if you do that, you just check a box and then
anybody who has a mainstream PC can actually run it.
And that way you get more users for your application.
So that's one of the other things
that you would have to tweak,
and that's literally all you have to look at.
Those are the higher level things that you would have to
do if you wanted to
convert your Mixed Reality experience.
If you want to learn more about how to get started,
the easiest way to do that is to go to aka.ms/iwantmr.
And then what you do is if you have an experience,
you profile your experience there.
You tell them who you are,
you tell them what your application is.
If it's Steam or something, you give them a link.
And then once you do that,
they from there will send you a developer kit.
So you get the same answer kit that I'm showing
you with the controllers for free.
So as long as you have an application
that you want to port, they profile you're there,
and if they think your application
will be a good candidate for us to port,
they'll send you the kit for
free and then they'll give you
access to our newsletters and developer resources.
So if you have questions, you need
some help with somebody porting
your experience or something,
you'll actually get developer resource people.
You can call real people,
you can talk to on the phone not just
forums and they can
help you with porting your experience.
It's super easy and most experiences can
ported in a week to a month.
We have a lot of experience.
We do these at Hackathon a lot,
and people have a really good experience porting,
and it's relatively fast.
And then you can get discovery in
the Windows Store so you have a new
distribution channel for your application.
So that's it from me today.
I'm happy to take any questions.
If you want to come up to the front
and ask me any questions
about anything you saw and I can
go into more details
on the things that I didn't show you.
So, thank you very much for coming guys.