♪ [MUSIC] ♪
[STEVE] Hi, everyone. Thank you for joining our session for Unite Now.
This is Using AR Foundation with Magic Leap.
I'm Steve Lukas, a senior manager
of the Magic Leap Developer Relations team
in charge of education and support.
My co-host here for today is Chris Reilly.
[CHRIS] Hi, everyone. I'm Chris Reilly
from the Developer Relations Engineering team at Magic Leap.
Happy to be here. [STEVE] Thanks, Chris.
Let's go over the agenda for our session.
We'll start with an overview of AR Foundation for Magic Leap,
then go over the setup requirements and process.
[CHRIS] I'll talk about integrating additional Magic Leap functionality
with extended features that go beyond AR Foundation.
[STEVE] We'll discuss some technical notes, followed by a recap.
With that, let's get started.
What is AR Foundation?
It's a cross-platform API that supports augmented reality.
This includes mobile device AR,
such as that found on iOS ARKit and Android ARCore,
but also supports popular head-mounted displays
that you wear directly over your eyes
through glasses like Form Factor, such as our Magic Leap 1.
AR Foundation is completely built and supported by Unity
through its Package Manager,
with Magic Leap support added in Unity Editor 2020.1.
See the chart at the right for the feature matrix,
and note that AR Foundation supports different sets of features
for each platform.
For Magic Leap, the most notable ones we'll cover today
are device tracking, plane detection,
and image tracking.
Note that Magic Leap does offer more features
than AR Foundation covers, such as use of our 6DoF controller,
as well as hand-tracking and eye-tracking.
In order to use these features, you'll need to integrate
the Magic Leap Unity Package provided by your SDK,
and we'll show you later how to use these together.
[CHRIS] The biggest benefit of AR Foundation
is the ability to build once and deploy to multiple platforms.
While the user interface should adapt to each platform,
the AR-specific features can be driven by identical APIs.
[STEVE] Setup. One of the challenges of cross-platform development
is keeping track of all the moving parts.
Due to all of the dependencies from various software providers,
it's easy for any one of them to break compatibility
due to them all having an independent update cadence.
As a result, here are recommended requirements for getting started
with AR Foundation in Magic Leap.
Your Unity Editor must be set to a version of 2020.1.
Any version of 2020.1 should do,
but the last verified version for us
was Unity 2020.1.10f.
From the Unity Package Manager, you'll need to have
the AR Foundation package set to 4.0.8
with the Magic Leap XR Plugin set to version 6.0.0.
For the Magic Leap SDK, it gets trickier.
The Lab is a developer tool provided
at Magic Leap's developer website: developer.magicleap.com
and it has its own Package Manager
to deliver Magic Leap's developer tools.
From here, you'll need the Lumin SDK version 0.24.1
and optionally, the Magic Leap Unity Package version 0.24.2.
Finally, for the AR Foundation samples,
you'll be using branch 4.0.
Before we go further, we are making a number of assumptions here
as far as what you already have available to you or installed.
First, you'll need access to Magic Leap hardware.
AR Foundation features do not work with our simulator,
so you'll need to have access to an actual physical device.
Then as mentioned before, our developer tools
need to be installed via The Lab, which is available
on our Developer Portal at developer.magicleap.com
You need to install the Lumin SDK from The Lab,
and optionally, the Magic Leap Unity Package,
both using the versions listed on the previous slide.
In order to deploy development builds to your device,
you do need to download a development certificate
along with the associated private key,
both of which can also be found on our Developer Portal.
You'll need a version of Unity 2020.1
already installed from the Unity Hub.
And this is optional, but having git installed
will make it easy to grab the correct branch of code
for the AR Foundation samples.
Step by step.
Now we're going to walk through the steps needed to get started
with the AR Foundation samples, but for Magic Leap.
First, we're going to want the right AR Foundation samples branch,
which is 4.0.
We're using git clone to get these samples,
but you can download a zip too if you prefer.
Then we're going to open up the project with Unity Editor 2020.1
preset to the Lumin target.
Once that completes, it's time to set the Lumin SDK path
in the Preferences so it knows where to find
the Magic Leap compilation tools.
Then we'll update the packages in the Package Manager.
First, let's confirm that AR Foundation is at 4.0.8,
and now, let's update the Magic Leap XR Plugin to 6.0.0.
Now we need to make sure we add the Magic Leap Provider
in XR Plug-in Management over here.
This is found in the Project Settings.
We also need to set up application Privileges
in the Manifest as well.
First, there's the ControllerPose, which allows access
to the controller tracking.
PcfRead offers access to spatial anchor-like features
known as persistent coordinate frames.
WorldReconstruction enables spatial mapping
and plane detection.
And CameraCapture provides access to the camera for image tracking.
Now we edit the Bundle Identifier to be compatible with Magic Leap.
This is in Project Settings > Player > Other Settings.
Now, if you're not familiar with the Bundle Identifier,
it's the unique string ID for your application.
Think of it as your application's username,
and the recommended standard is use reverse domain name notation.
For example, for us, we may use com.magicleap.applicationname,
such as "com.magicleap.test".
For development, we may use "com.magicleap.dev.test".
Keep in mind that the Bundle Identifier should be all lowercase,
avoid spaces and special characters besides the period, to be safe.
Then specify the location of your development cert.
It's here in Project Settings > Player > Publishing Settings.
Note here the folder structure where the development cert
is in the same folder as the associated private key.
OK, almost done.
Now let's reset the included Scenes.
Go to File > Build Settings and uncheck all the Scenes.
Now, that was a lot, but we're all set now.
To play around with Magic Leap on AR Foundation,
there's two Scenes that can be run without any changes.
The first is plane classification.
To select a Scene in Build, we can do one of two things:
we can checkbox the Scene in the Build Settings
or we can leave all Scenes unchecked,
open the plane classification Scene,
then build and run, same with the basic image tracking Scene.
There are two more Scenes which work out of the box,
but need a couple of changes to avoid single-eye rendering issues.
As is, these Scenes will produce content
that only appears in the left lens of the Magic Leap.
By following the Unity documentation for Custom Shaders
and editing the NormalOverlay and FeatheredPlaneShader.shaders,
the Scene's normal Meshes and feathered planes
can be deployed as well. Give them a shot.
[CHRIS] Now that we have discussed the Magic Leap functionality
that you get with AR Foundations by default,
we're going to take a deeper look at some additional features
that are available to you with the Magic Leap SDK
and Unity Package.
These include 6DoF controller support using the Magic Leap control
for accurate manipulation of virtual elements;
discreet hand pose recognition including the OK sign, pinch,
thumbs up, et cetera;
hand meshing which can used for occlusion or for changing
some of the visual elements of the hand to better fit your experience;
there's also hand tracking of several key points
along your wrists, palms and fingers;
eye tracking is available as well.
To get this functionality within AR Foundation,
we have made a custom version of the samples.
It was forked from the AR Foundation sample repo on git.
It can be found on the link on this slide
at slukasml/arfoundation-samples
That's Steve Lukas without the "teve."
These modified samples require the 0.24.2 Magic Leap Unity Package
to be imported into your project.
We had this listed as an optional step earlier,
but this package does include some additional scripts and models
that we are utilizing in these samples
along with other examples for Magic Leap-specific functionality
to ensure the most optimized Magic Leap development experience.
What do you get with our altered Magic Leap AR Foundation examples?
The anchors example now supports the Magic Leap controller,
and pressing the trigger of the controller
will place an anchor in the World.
We have also added the ability to press the trigger
to toggle plane detection on and off in the plane's example.
If you're like, "Wow, I love seeing all of these planes in my room.
I had no idea they were even there,"
it's just a lot to handle, and to help control your excitement levels,
you can take a break and just pull the trigger
and it will shut plane detection right off,
sending you right back to the non-augmented world.
The note at the bottom there is another call-out
to the dependency on the Magic Leap Unity Package.
Again, this is required for these samples
as we are utilizing some Controller scripts and models
from that package.
Here's a short video showcasing some of the Magic Leap-specific features,
such as access to our human-centered computing model
of gestures and hand tracking.
The controller also offers a unique model
of precision 6DoF interaction,
while also providing added comfort for the user.
Play around and see what you can come up with.
[STEVE] We're really excited about getting developers onboard
as fast as possible,
and AR Foundation helps make that even easier.
Still, the rapid pace of innovation always comes with some caveats,
and here's a list of things to watch out for
in this current release.
Clipping plane:
When using AR Foundation's version of the camera,
the clipping plane defaults to 10 cm, which is much closer
than our recommended clip plane size of 37 cm.
[CHRIS] Wait...
What?
[STEVE] For production apps, we recommend setting
the near clip plane to 37 cm or greater
to avoid any visual discomfort for your users
and for our store compliance.
[CHRIS] That's a big one, Steve. [STEVE CHUCKLES]
[CHRIS] Camera privileges with Magic Leap XR Plugin 6.0.0:
For our image tracking to work on AR Foundation,
Magic Leap XR Plugin needs to be set to version 6.0.0,
not the default version 5.1.2.
However, this version of the plugin also automatically
turns on the camera for the duration of the experience
when the Camera Capture privilege is set.
Some experiences may want to provide a way
to only use the camera when needed and turn it on or off
during the actual experience.
If that's what your app wishes to do,
you will need to use version 5.1.2
and integrate the Magic Leap Unity Package
using the Magic Leap SDK version of image tracking.
[STEVE] World origin coordinates:
World origin coordinates are not reset at application start,
which you may be used to from some other platforms.
Instead, they're based on the device session.
This can refer to when the device is powered on,
or when head pose is lost and reinitialized.
The World origin coordinates reset to 0,0,0 at these moments
so keep this behavior in mind.
[CHRIS] Hand tracking:
In this setup configuration with Magic Leap Unity Package
version 24.2, hand tracking will stop working
when you suspend and resume the app,
with the only resolution being to close the app and restart it
or to downgrade to Unity 2019.3 with Unity Package 24.1
without AR Foundation support.
[STEVE] Unity 2020.2:
With the recent release of 2020.2,
expect Magic Leap support for this platform
to arrive in early 2021.
To recap, AR Foundation is a quick way to get started
with cross-platform development of AR on mobile and HMD,
including Magic Leap.
[CHRIS] If you'd like to use Magic Leap-specific features,
you can integrate the Magic Leap Unity Package and use it together
with AR Foundation for a richer experience.
[STEVE] Finally, ensure proper near clip plane settings
when using AR Foundation's Camera Settings
to accommodate user comfort.
[CHRIS] For more documentation and support,
check out our website and forums at developer.magicleap.com
Thanks for attending our session. [STEVE] Goodbye!
♪ [MUSIC] ♪