If you have ever looked for your car keys,
or been lost in an unfamiliar place,
you know how incredibly useful it
is to understand and remember your surroundings.
Project Tango is adding this capability to mobile devices,
so they can track their motion, recognize faces, and scan
the surroundings in 3D.
Its ability to recognize spaces means
it can determine its exact position, find your car keys,
and remember where you put your virtual pet monster.
In this episode, we will talk about area learning concepts
and how to get started with Project Tango Explorer.
Project Tango combines visual and inertial data
to give mobile devices an understanding
of position and orientation.
It gets the visual data from its cameras,
and the inertial data from its built-in acceleration
and rotation sensors.
This is combined into motion tracking data, which provides
the device relative motion.
Area Learning extends this data to remember areas
and to estimate the device position within them.
You can use this for multiple purposes,
such as precise indoor navigation,
placing virtual objects in the physical world,
or to share a virtual world with other users.
To use area learning, you must first
learn an area so the system will be able to recognize
the area later.
The combination of learning and localization
is similar to what humans do.
When you enter a new building, you quickly
get a picture of its layout, and you
can use this memory to find your way around at a later time.
Project Tango saves the learned memories
into an area description file, or ADF.
This file contains descriptions of visual landmarks
that the system uses when looking for known places.
An ADF enables the device to estimate its position
through a process we call localization.
Another important aspect of area learning
is that the device can refine its position estimate
relative to the area it is in.
This is great if you want to fixate a virtual world
in the physical world.
For example, if you want to put an elephant in the living room
and have it stay there, without area learning
the system can only estimate its position
based on small increments in the motion of the device.
This method introduces inaccuracies,
as it requires incredibly precise sensors
to be able to track motion like this.
Just imagine you had to count your steps
and turns to be able to know where you are.
It's error prone.
Area learning sidesteps this whole issue
by storing visual references that it can
use to calculate its position.
In other words, the ADF is the frame
of reference in area learning.
To make area description files, you
can either implement area learning in your own app,
or use a developer tool like Project Tango Explorer
to learn an area.
But first you need to know that area descriptions are not
created equal.
It's possible to learn an area with different levels
of quality depending on how much of the area you have learned.
For instance, it's going to make a big difference if you only
take a picture from one perspective versus taking
pictures of all the perspectives of a room.
The more coverage you have, the easier
it will be for the device to recall it has been in the area
before.
It is important to note that the success of area learning also
depends on visual characteristics
of the environment.
An ideal environment has distinguishing visual features,
mostly stationary objects, and does not change much over time.
Environments that are visually bland,
such as long, undecorated hallways,
or places with a lot of moving people
can be harder for the system to remember.
Light also changes the way things look.
So if the lighting conditions vary a lot in the area,
you may need to have a couple of ADFs
for the brightest and darkest time of day.
Project Tango allows you to learn and load an ADF.
The ADF will be stored in the flash memory,
and can be used in other apps to find the position
and orientation.
You can download Explorer from the Play Store,
but let's go through how it works in practice.
In the default mode, click the learn button
to create a new ADF, and walk around
in the area in one direction.
Remember, the system benefits from having
seen more of the space, so once done walking around,
walk back in the opposite direction.
This increases the coverage and makes it easier
to use the ADF later.
In the end, we save the ADF to a file.
It then takes three steps to localize in the area.
First, be in an area that has been learned
and for which there is an ADF.
Second, load the ADF for the area.
And third, move the device to compare the ADF to the area.
Once the system has recognized the area,
you will be receiving real-time updates on the position
and orientation of the device.
You will only see a blue trace in the beginning,
and until the system figures out where it is.
This is the relative motion tracking trace,
which you can learn more about in the Introduction
to Motion Tracking video.
As soon as the system recognizes the area using the ADF,
a green trace will appear.
This is the position of the device
in the area you have learned.
The system now uses the learned data to correct the device
position within this area.
Area learning adds the ability to visually recognize
known areas, and locate the device within them.
Whether you want to have a monster under your bed,
or have an elephant in your living room,
area learning will allow you to do that.
Visit our Google+ community, and join us on our journey.
We're excited to see what you will build with Project Tango.
[MUSIC PLAYING]