[MUSIC]
>> Hi. I'm Thomas Lewis.
I'm the Cloud Developer Advocate here at Microsoft Build Live,
and I have the wonderful opportunity
to talk to my guest who is Duncan Lawler.
You're a principal software engineering manager.
Did I get that right?
>> Yeah.
>> Okay. Good. So today, what we're going
to do is to talk about something that your team has built.
You've already announced it but you're talking
about it a bit more here at Build.
So tell me what you've built?
>> Yeah. So what we just announced is
a map control for the Unity of the Development Environment.
It's primarily targeted for
Mixed Reality or augmented reality scenarios,
and it's used for applications to be able to put
together a virtual tabletop map like you see in Sci-Fi movies.
>> Right, very cool.
When I saw it, I thought that it was really
interesting because I always called
these things 3D islands, this little square.
Now, we've seen other being mapped controls in the past,
and how is this one particularly different
than what we've had already in 3D?
>> Well, the map control for unity is most closely related
to the map control that's built into
Windows in terms of the 3D functionality it has.
But the main target for this is
Mixed Reality which none of the other map controls currently have.
So all of the controls will support
the same basic geospatial functionality,
you can view any part of the world
and do your own overlays on them.
But this one allows you to give me
the full stereo Immersive Visualization.
>> Interesting. So why did you, in particularly,
think of Unity as where to have this components sort of drop into?
>> Well, let's actually, we experiment with
a couple of different options before we settled on Unity.
What we found out is that
most developers who we're targeting
Mixed Reality we're using the Unity Development Environment.
It's just very friendly to a developer to be able to very
quickly put together an application that has rich visualizations,
and when we looked at what was out there,
there was really no solution for displaying
real-world data within a Unity context,
and especially there was no way for
a developer to put together something very simply.
Everything that was out there was custom build your own,
and what we're really aiming for here was
a single component that you can just drop
in and have a fully functional map working.
>> So I don't have to do a pull from GitHub,
do a certain set of builds,
and then go to Unity and ask them hey,
can I put this in, it's just a drag-and-drop plugin.
>> Yeah. You just download the Unity component.
Once you have that you add it to a game object as
a map render component and
you've got to fully functional visual map.
You don't have to do any customization with scripts or anything.
You just call the API on the map control,
you tell it the location on the world you want to look
at and the level of detail that you want to view,
and it'll handle downloading all of
the data and making them visible to the user.
>> That's very interesting. Very interesting.
Now, what I've seen in the past a lot of times
with controls especially with mapping technology
is usually you have to get some license.
So you get the license,
you sign up to be a part of the thing and it's
usually like a business contract that you have to go through.
Is it pretty much the same thing for us with is?
>> Yeah. The licensing model that
the Mixed Reality map control uses is the same
as all of the other licensing is for our Bing maps controls.
You sign up for a developer account you get an application key,
and you provide that application key to the control on startup.
After that, we have a very generous free terms of use,
where you can have a certain number of
sessions after that you pay per session.
It's the same key that you can use for all of
the other Bing map services that are online,
all of our Rest services like Geocoding, directions, search, etc.
>> So if they already have signed up for the Bing maps services,
this is, in addition, you get it for free comparatively?
>> Well, up to a certain transaction level
if there is a free terms of use,
and then beyond that, you pay on a per session basis.
>> Okay. Very interesting.
So what are the platforms that are targeted with this?
is there something I can only use in Windows context?
>> Currently, when you embed it into Unity scene,
you have to say what you're targeting,
and we support all of the mixed reality devices
that are available on the Windows Platforms,
so all the Windows Mixed Reality headsets,
and other headsets like the Oculus Rift and the HTC Vive,
and you can also build for the HoloLens 1 and 2.
We don't support the ARKit and ARCore on Android and iOS yet,
but it's something we're considering.
>> So with Windows, it's not just the desktop apps,
but it's HoloLens, Windows Mixed Reality,
and then did you also say Oculus?
>> Yeah. It should support any of
the headsets that run on the Windows Platform,
and you could also build the desktop app with it,
there's just no restriction on it.
But it's primarily targeted more for Mixed Reality scenarios.
>> Right. With using Unity because
a lot of the folks that are interested in Mixed Reality
spend a lot of time in Unity,
and so how does it integrate with other components?
You were talking a little bit about the game object,
and so maybe for folks who don't use Unity,
and we're like, "Hey, game objects,
of course, everybody knows what that is."
How would you explain what a game object is?
Then why is it interesting that
your component can actually attach to that?
>> In the Unity Development Environment,
it's a visual tree where everything is
a game object or a child of a game object.
So it's a hierarchical visual tree.
The way the map control works is it attaches to
a game object and then it takes
over the rendering for that object,
and then you can take any other game object you put in
your Unity scene and you could
request that it be attached to the map control.
At that point, the map control takes
over the position and the scaling
of rendering that game object
and automatically makes it sync with a map control.
So when you ask them that control to zoom to a new location,
it'll handle moving that object to that new location as well.
>> Does this do the same thing for the camera.
So like I noticed here in the animation we have behind us.
You have the ability to go from one place maybe like
the Seattle Space Needle to gas works,
but do you have the ability to also have the camera do
the same thing where if I want to look at
a different view of it it can control the cameras as well?
>> Sure. So in Unity,
you have control over the position of a camera,
and in Mixed Reality contexts.
Typically the position of the cameras
controlled by the user's head.
>> Right.
>> So if you wanted to look at the other side of the map you
would just walk around and look at the other side of the map.
>> I got it, got it.
So we have this Mixed Reality toolkit that we contribute to,
it's open source as well,
and a lot of folks in Mixed Reality are using that toolkit.
Does this interact with that toolkit?
If you can, tell me what the toolkit actually does
for the people who may not know.
>> The Mixed Reality Toolkit is
a great set of code that Microsoft has released
as open-source to help you provide
user interface within a Mixed Reality context.
A lot of Unity developers have a problem with
putting up just simple UI things and in Mixed reality,
it's even harder because there's no fixed interaction model.
There's no equivalent of a list box.
>> Right.
>> What the Mixed Reality Toolkit has
tried to do is provide that for developers.
So our map control integrates well
with that as a matter of fact that demo application
that we put together which you can download from
the Microsoft Store right now, it's called Outings.
Or if you want to wander over to our booth you can see it live.
It uses the Mixed Reality Toolkit for all of the UI
surrounding the map and then has
the map control interacting with that.
So as you select items from the Mixed Reality Toolkit,
UI the map control response to that.
>> Interesting. The map control
I'm assuming pulls from being data.
What are the data sources that I have available to me?
Can I have flat models if I wanted to?
Or could I change from 2D to 3D within that construct?
>> The data that they control uses is
the same data that the Windows Control uses.
It's primarily three main data types that we pulled from.
The most detailed data that we have is our texture 3D city models,
which we have a very high resolution up to five centimeters per
pixel and we have those for about 400 cities in the world.
Outside of these detailed city model areas,
we have an elevation model for the world.
It's a seamless global elevation,
so there's no cliffs or islands of new data.
>> Right.
>> The resolution varies depending on where you're looking at
from about 30 meters to 60 meters at a minimum,
to about one meter per sample.
On top of that, we will take
orthographic aerial imagery that
comes from either planes and satellites.
It's typically at least 50 centimeters per pixel,
and that gives you a general feel
for natural terrain features at least.
>> Okay. Some people probably are going to be like,
"Well, what does that mean actually?"
So if you were to fit it in layman's terms,
I mean he's that like, "Wow. I'm getting like really,
really deep clear views of the thing that I'm looking at."
Is it like, "You're going to be like 20,000 miles
in space looking down at this little cube."
>> Yeah. For five centimeters per pixel,
you could see objects as small as this on the ground.
>> That's amazing. Is that something that's just
available to everybody like do all of the mapping providers,
give you that depth?
>> There are very few companies in
the world that have this level of detail.
>> Okay.
>> Basically, this map control is
backed by petabytes of data that are
seamlessly streamed across global data-centers
with millisecond latencies around the world.
>> Wow. I would assume because that's all of
the work that you-all have been doing with being overall.
>> Right.
>> That's amazing.
Yeah. Sometimes a lot of folks don't realize that,
number 1 is this data is huge.
you-all have had to figure out like, "Hey,
how do we deal with this amount of data?"
Then again to get that precision,
takes a huge amount of effort.
I mean, what's the size of a team to do this thing?
>> Well, when we were producing the data,
we scaled up the team a bit and we
had a team of about 200 people that were
primarily dedicated just to
the production of the detailed 3D city areas.
>> The cities that you have,
you said there's 400 cities.
>> We have about 400 cities worldwide.
They're mostly in North America,
and Western Europe, and Australia as well.
But the scale of the data is a little unprecedented.
I like to think of it in terms of,
there's been a lot of press around
these large open-world games and some of them have talked
about how huge the areas they have modeled 3D data for.
The largest games out there have
areas around 100 square kilometers in size.
Our smallest city is bigger than 100 square kilometers,
our largest city areas are on
the order of 8,000 square kilometers,
and we have a total detailed 3D modeled area
on the order of a 150,000 square kilometers.
Unlike a game where they can reuse the same textures and models,
every pixel is unique in all of that data.
So the scale is just amazing.
>> This is the plan to like add more cities as you go?
>> Yeah. We typically look and refreshing
our data every three to five years.
Our focus area for data collection is
primarily on where the populated centers are.
I would estimate in the United States for example, well,
the surface area that we have covered compared to
the surface of the United States is not large.
In terms of the population coverage,
we probably have more than 60 percent of
the populated areas covered now.
>> Yeah. So how do you create the 3D data?
I mean, I assume that you don't have
3D modelers like going and then going, "Okay.
Today's that you'd think is the Pike Place Market in Seattle."
>> Right. That just wouldn't scale.
So the process involves,
first, flying high resolution imagery.
It's not widely known because
other providers often referred to it as Satellite Imagery.
But all of the really high resolution imagery,
the five centimeter pixel stuff has to come from
aerial photography where we fly an aircraft over a city.
It has a very specialized high resolution 280
megapixel camera that takes pictures that look both straight down,
and to the side, and front, and back.
So you get views that are angled as well as straight down.
It basically just goes back and forth
across the city on a lawnmower pattern,
taking a picture every second.
The result is tens of terabytes of
data that we run through a very specialized process in
an Azure Compute Cluster to create
the 3D models from all of those overlapping images.
>> Wow. So what are some of the future considerations?
I know that you-all are open with sort of like,
"Hey, what are we doing?
What does our road map look like?"
What are some of the things that are on the horizon
that you-all are now investigating?
>> The scenario that we've primarily
targeted for this first release is, like I said,
the virtual table top map where you can have
collaborative people looking in on the same virtual map.
We've also heard from developers that
there's a lot of interesting scenarios
around where they want to have more of an immersive field of view.
So rather than just have a map with strong edges to it,
they want it to go out to the horizon,
be able to stand there and look
as you would when you're standing in first person.
We also want to bring more of
the vector data that we have available to annotate the map.
Without labels and metadata as to what's there,
the map is nothing more than just a
train visualization which is useful
but it's much more useful to be able to say,
"This is that city, this is that road,
this is the business that's sitting there."
Not just look at the picture.
We also want to have the ability for developers to
add more of their own context in.
One of the scenarios we've heard a lot in
architecture for example is they want to be able to visualize,
"What would happen if I removed
this building and put my own building there?"
So we want to be able to edit our data,
flatten it out, and then put a new model in its place.
>> That's very interesting.
So again, thank you very much
for taking the time to chat with us about this.
I'm assuming that folks can go online and find where it's at.
At Bing, I would assume that they could use to search for it.
>> Yes. If you look for it,
the Maps SDK, a garage project is the official name.
We've got some blog post stuff on the Bing,
that's blog that point to all the documentation.
>> All right. Well, that's a wrap.
Thank you very much for taking the time to talk with us,
and chat with us, and we will have
more on the Microsoft build live.
[MUSIC]