BO PANG: Hi, I'm Bo.
I'm a software engineer on ARCore,
Google's Platform for building augmented reality experience.
I'll be showing you how to implement face effect using
the Augmented Faces API in ARCore with the Unity engine.
Links to Android and iOS versions of this video
can be found in the description.
Augmented Faces doesn't require a special helper,
such as a depth sensor.
Instead, it use phone's camera and machine
learning to provide three pieces of information
in real time synchronized with the camera feed.
First, a center pose for a face, which
helps you do things like render a head.
Second, region poses, which are useful for placing assets
on or near the forehead or nose.
And third, a 468 point dense 3D face mesh,
which allow you to paint detail texture that accurately
follow facial movement.
So, pretty cool, right?
Now, let's see how you can utilize the Augmented Faces
feature in ARCore to build your own app using Unity engine.
You will need to download the ARCore Unity
SDK from our GitHub Res page and import it into a Unity Project
as a custom package.
To start your project for ARCore development,
make sure to set the Unity target platform to Android
and enable ARCore supported checkbox
in Unity Player Settings.
Next, let's see how to enable ARCore in your own Unity scene.
First, delete the main camera and add ARCore device prefab
into the scene.
You will notice that the prefab contains two game object.
The ARCore device game object uses ARCore session component
to manage the ARCore session lifecycle.
An ARCore session can be passed by disabling this component.
The first person camera game object
serves as the main camera of the scene.
ARCore automatically update transforms each frame
based on the device movement.
It is also configured to render the camera image on the screen.
Now, let's set up the ARCore session.
First, create a new ARCore session config asset
inside the project window.
Let's call this augmented face session config.
Next, select the session config asset.
Disable plane finding and light estimation as those features
are not compatible with augmented face yet.
Then set the Augmented Face mode to mesh.
Now, this session config is set up for Augmented Faces.
Select the ARCore device game object and drag
the session config asset into the session config field.
Once you have done that, you will notice
there is an error at the bottom saying augmented face
mode on mesh require front facing camera.
This is true.
So let's change the device camera direction
to front facing on ARCore session component.
With the current scene setup, the app
will start the ARCore session and render the front camera
image on the screen.
Even though it is set up to detect face in the camera view
port, we can't see any effects because we haven't put anything
there yet.
Now, let's add the example fox effect from the ARCore Unity
SDK to your app.
First, let's create a root game object called face attachment
to hold our fox face effect game objects.
Next, let's add the 3D meshes from the fox effect
and configure it to update with the face regions.
Drag the fox sample FBX to create
an instance inside the face attachment object.
This FBX contains fox ears and nose
models, which are connected to a predefined face rig.
This makes sure those models are always
attached to the face regions.
You can create similar face assets
using the fox sample FBX and the canonical face mash FBX
in ARCore unity SDK as a reference.
Now, add the ARCore augmented face rig component
to the fox sample game object and check the Auto Bind
checkbox.
With Auto Bind enabled, this component
will automatically connect to the augmented face
object detected by ARCore and update its face rig
using face region poses.
Next, let's add the fox face or texture overlay.
Inside the face attachment object,
create a new game object called face texture.
Add the ARCore Augmented Faces mesh filter component
and check the Auto Bind checkbox.
With Auto Bind enabled, this component
will automatically connect to the detect face
and generate a 3D face mesh.
Here, we want to use the face mesh to render
a texture on the face.
So let's add the mesh render component
and set it to use the fox face material.
Now, build and run the app on an ARCore compatible Android
phone.
You'll see that the fox face effect
is overlaid on your face.
Now you have everything you need to get start
with ARCore augmented face in Unity,
you'll be able to use your own assets to create a custom face
effect.
For more information about this tutorial and ARCore in general,
you can check out our sample app source code
and documentation in the description below.
We can't wait to see what you build
and hope you share it with using the ARCore hashtag.
Thank you for watching.