- with introduction.
In recent years, a new medium of
performed 2D characters has emerged.
Cartoon characters have started to appear live
on streaming platforms and broadcast TV.
The primary motion of the character is controlled by
the performer with face tracking or mouse and touch input.
The primary animation is the motion of the main body
of the character, or the parts of the character
that are most important to the story.
Notice as the user heads moves,
the character head here also moves.
In addition, a critical aspect of creating expressive
animated characters is the presence of secondary motion.
This is the small movements of parts of the character,
like hair, clothing, tails and fur, which react to
and emphasize the primary motion of the character.
Notice that the performer does not control the secondary
animation of the dreadlocks, when instead it reacts to
the performer's, or the character's, head.
Our goal is to make it easier for artists to add
secondary animation to their illustrated characters.
The input of our system is a performance of the primary
motion of the character, as shown on the left.
In our result on the right, we add secondary animation.
Our solution augments the layer-based representation
of illustrated characters to enable the creation
of automatic secondary animation.
To better understand the variety of secondary motion
that animators typically apply to 2D illustrated characters,
we curated a set of 29 short animations
created using an layer-based approach.
These examples, of which a subset are seen here,
are between 30 seconds and 12 minutes long,
and include short films, advertisements,
TV series and promotional videos.
For each animation, we identified
all instances of secondary motion.
We found a total of 86 instances of secondary animation
that were mostly applied to tails, clothing, hair,
large appendages such as ears, and hanging objects
such as necklaces and ties.
From these, we were able to develop the secondary animation
categories of trailing motion, swaying and jiggling.
Trailing motion occurs on appendages
like tails and tentacles, and the motion of the appendage
roughly follows the trajectory of the character.
In most cases, length does not change.
However, squash and stretch can be applied
to exaggerate the motion of the character.
Long skinny parts, like necklaces,
scarves or hair, often exhibit swaying.
Whereas heavier objects such as a kid's arm,
may sway in a slower, stiff manner,
lighter objects, like a necklace or tie,
often move more freely.
Jiggling occurs in the small protrusions or appendages,
like tufts of fur, spikes, or pieces of clothing.
They continue to move once the character stops
or changes direction, and the jiggling varies with
the properties of the relevant parts.
Previous work has been unable to satisfy easily adding
secondary animation to perform illustrated characters.
As I mentioned previously, performed animations benefit
from the proliferation of digital inputs, such as
motion capture, video puppetry and touch interfaces.
However, these works focus on the primary motion
of performance, not the secondary animation.
Procedural creation of secondary animation has been used
extensively in animation systems such as K-Sketch,
Draco, Kitty, Motion Amplifiers and Energy Brushes.
These systems are useful for authoring pre-defined motions,
but not online performed animations.
While the secondary amplifier motion amplifiers could
potentially generate jiggling effects for performed motion,
it does not handle external forces like collisions, gravity
and wind that are critical in many performed settings.
In offline or key frame animation software, simulation
is a go-to method for automatic secondary animation
of hair, cloth and collisions.
Simulations assume full knowledge of the system,
including geometry and other properties.
This requirement makes it difficult to apply simulation
to a layered composition of 2D artwork.
Buildings on the findings in these papers, we based
our approach within a constrained dynamics formulation,
which gives us a general purpose mechanism for delivering
plausible secondary effects that also track
performance-induced constraints.
Most commercial tools, such as Maya and After Effects,
are designed for offline, rather than online, animation.
While these tools support general physical simulation,
which can generate secondary effects,
doing so requires extensive low-level setup.
Game frameworks, like Unity and Spine 2D, also support
real-time animation under human control.
Most games add secondary effects
to a fixed set of characters.
These effects are specific movements so game developers
can plan ahead by tuning the behaviors for the
particular motions in the game.
In contrast, our goal is to enhance performed storytelling
with user-generated artwork and performances.
The previous work all have drawbacks which make them unable
to easily add secondary effects to performed 2D characters.
A key contribution of our work is a simple but powerful
rigging system that leverages physical simulation
and makes it easier to create secondary effects
for layered 2D artwork.
Our approach has two parts. An offline portion where
the user designs, draws and rigs the character, and an
online portion, where the user performs the primary
animation and we simulate the secondary animation.
One challenge when simulating the
secondary animation is that we do not know
the performed primary motion ahead of time.
As a result, we have to design a system that will respond
to any possible primary motion in real time.
To add secondary motion for performed 2D animation,
we introduce a set of physically inspired rigs.
The follow rig, the rest pose rig and the collision rig.
They are designed to propagate motion between the layers
of an illustrated character in a way that produces
convincing and controllable secondary animation.
Collectively, they cover the range of secondary motions,
trailing motion, swaying and jiggling, that we determined
from our classification of examples.
When designing our rigs, we made sure
to satisfy four properties.
First, we wanted our rigs to be easy to create.
By simplifying the rigging process, we enable users
to quickly add secondary effects to new characters.
We also streamlined the parameter-tuning process
by exposing a small set of parameters that nonetheless
support a wide range of effects.
Second, our rigs create plausible motion by using
physical simulation to propagate motion.
The use of simulation enables secondary parts
to respond naturally to external forces
like wind, gravity and collisions.
Next, our system is modular, and accommodates
changing the simulation process as appropriate.
As a result, our approach can leverage the ever-evolving
state-of-the-art in simulation techniques, and even combine
multiple simulations to capitalize on the
unique benefits of each method.
Finally, our rigs are composable.
Multiple rigs can be applied to the same character
to induce secondary animations that compose the effects
of different motion propagation methods.
To achieve the three types of secondary motion described
previously, trailing motion, swaying and jiggling,
we use our workflow to add our rigs
to an illustrated character.
First, we leverage the fact that most artists already
create their layers, their characters, in a layered format
using image-editing software such as Photoshop.
To animate these characters, the layers deform.
Notice the body of the snowman
and the scarf bending as he sways.
These deformations are controlled by handles.
We calculate the deformations from a set of handle positions
using bounded biharmonic weights by Jacobson et al.
For online performed animation, the animator cannot control
all of these handles in real time.
Hence, we only want the user to control a couple
of these handles, and we need to specify the motion
of the other ones to produce secondary motion.
In the workflow that we propose, there are a variety
of handle types that an artist can add to the artwork.
The control handles are directly controlled by the user
via face tracking, body tracking, or mouse and touch input.
And these specify the primary motion of the character.
Attachment handles specify constraint
between the parent and child layers.
To apply our secondary animation rigs, the user adds
two additional types of handles.
Origin handles inherit the primary motion
of the character, as specified by the performance.
Response handles determine how the primary motion
is propagated to different parts of the character.
Only the control handles will be
controlled during a user's performance.
The movement of all the other handles
is automatically computed.
To make the animations physically plausible,
we implement our approach within
a constrained dynamics framework.
In particular, we use the popular Box2D implementation
of 2D constrained dynamics, which provides a set
of useful low-level primitives for specifying constraints,
including springs, pivots and prismatic joints.
We construct our secondary motion rigs
with these primitives.
The first rig is the follow rig, which pulls
a trailing appendage along a trajectory
defined by the primary motion of the character.
It produces trailing motion.
To apply the follow rig to this airplane,
the user creates a set of ordered handles.
The first handle is the origin, in dark purple,
and the following handles are response handles.
From this set of ordered handles,
we create mass bodies and connect them with springs.
As the origin handle moves, we construct a set
of reference points, one for each response handle.
These lie on the trajectory of the origin handle.
The exact positions of the reference points
are determined by 1D spring simulation that aims
to keep the arc-length distances between the handles
the same as in the rest post of the appendage.
Then we set all the springs to be the same strength,
and finally, we connect the reference handle bodies
to the reference bodies to encourage the appendage
to follow along the origin trajectory.
The user can change the parameters of 'k,' which we
call the stretchiness, and the follow strength,
which is the stiffness of the light gray springs.
Here is our result with the follow rig
applied to the banner.
It trails behind the path of the plane.
The second rig is the rest pose rig.
It allows appendages to return to their rest pose
and causes jiggling and swaying effects.
The user specifies origin handles, in dark blue,
and also a set of response handles, in light blue,
that control how the part deforms
in response to the primary motion.
From these handle positions, we create bodies
and connect the nearest neighbors with springs.
Next, we create reference bodies which define
the rest shape of the underlying part based
on the current state of the character.
We position orient the reference bodies where the
response handles would be, given the overall deformation
induced by a set of control handles at the current frame.
If we simply set the response handles to the corresponding
reference point positions and orientations,
the underlying part would just deform
based on the primary motion.
Instead, we attach reference points to the corresponding
handles with linear springs that directly pull each handle
towards its reference in a more organic manner,
creating this lattice rest pose rig.
In this example, we apply a rest pose rig to the ghost.
As the primary part moves, the secondary part
sways and jiggles in response, eventually returning
to the rest pose when the motion stops.
If a character part is long and skinny,
like the snowman's scarf, a lattice connecting the handles
is not necessary since the structure's artwork
is better represented as a 1D rope.
Please see our paper for a description.
During animation, the head motion is performed
with face tracking, and the arms are controlled
with multi-touch input.
The scarf is automatically driven
by two rope rest pose rigs.
To handle collisions between character parts,
we propose the collision rig.
The user defines the handles of the collision rig,
which interact with other parts of a scene.
Then from these user-specified handles, we create bodies,
and we also create rectangular bodies for colliding.
The length of each rectangular body is equal
to the distance between two handle bodies.
This diagram shows the path pulled apart.
We add pivot joints between the handle bodies
and the rectangles for bending, and then we add
springs in between the rectangular bodies.
Here, the lily pads are pushed by the front
of the frog's head, but are free to overlap
the back of the frog, which is meant to be underwater.
We combine a collision rig on the frog's head
and a rest pose rig on the water and the frog's legs.
For this dancer, her leg will collide
with the first red layer of the dress.
As that one deforms, it will push against and move
the orange layer, which will then affect the yellow layer.
As her leg dances up and out, her dress floats
and ruffles along with the main movement.
To combine the different rigs described above,
the user can either attach separate rigs with
separate sets of handles to different parts of a character,
or they can reuse the same set of handles to produce
multiple types of secondary motion as shown here.
In this example, the octopus has two rigs attached:
A follow rig, and a rest pose rig,
both using the same purple handles.
In addition, we add a rest pose rig to the kelp.
Adding wind to the scene creates
a water swaying effect as the octopus swims.
Notice when she stops swimming, that her tentacles
slowly return to their rest pose.
This dragon has a follow rig in purple applied
to the main skeleton, and a rest pose rig in blue
applied to his mane and tail.
Our rigs add trailing motion to the dragon's body
and jiggling to the dragon's mane and tail.
Here, we apply collision and rest pose rig to the umbrella
and add a rest pose rig to the girl's hat.
In our result, the umbrella bounces as the rain falls on it,
and it keeps the girl dry.
Her hat also jiggles slightly as she walks.
To investigate the effectiveness of our approach,
we conducted an exploratory study
where people use our system and provided feedback
on our rigging and performance-driven animation tools
by animating these three characters.
Overall, the participants were able to quickly produce
a range of secondary effects across the three tasks.
They found it easy to create
the various rigs and were pleased with
the secondary motion that they produce.
Participants had some usability challenges, such as
forgetting to explicitly create an origin handle
rather than creating only response handles,
and some participants were also confused
about the parameter names and ranges.
However, after spending a few minutes
adjusting the parameter settings,
they were able to achieve a pleasing result.
While we cover a large range of secondary animation
on a variety of characters, there are
additional areas for exploration.
One can improve the process of creating
layered character artwork for animation.
There could be an automatic process that decomposes
a single flat drawing into a layered hierarchy
by learning from examples.
Another area of future work, the creation of extra controls,
was proposed by participants in our user study.
For example, they requested the ability to separate
handles of our rigs into groups, to have finer control,
rather than just changing the parameters for the whole rig.
Finally, we'd like to relax our assumption that the
underlying artwork driven by the rigs is unchanging.
In some cases, artists might prefer to swap out
the artwork while keeping the secondary animation
of parts temporally coherent.
For instance, if a running squirrel turns around,
the trailing motion of the tail needs to flow smoothly
behind it during that curve, and the animator
might have to switch the artwork in order to prevent
unpleasant deformations in the tail.
We proposed a set of tools which allow artists
to easily add and manipulate secondary motion
on parts of their characters.
We built these tools on top of common physical primitives
such as springs, pivots and bodies with mass
that most physical simulators already support.
Our proposed rigs enable novice and experienced animators
to add extra richness to their characters, enhancing
the live performances that those characters star in.
Thank you for your attention.
(applause)
- [Brandon] Brandon Dalton,
University of Southern California.
So, I was wondering if you guys were thinking about
creating a different kind of collision rig that allows
for some aspect of semi-permeability?
Which could have applications in biological animations.
- We hadn't considered that, but that
would be an interesting area to explore.
- [Brandon] Thank you.
- [Man] I want to ask about layers that, most of the time
you want to be acting as secondary animations,
but sometimes you want to be controlled.
You mentioned that the dancer's dress,
where most of the time it's flowing down, but sometimes,
the animator wants to actually pick it up
and do something with it.
Do you support that?
Is that something that would be easy to add?
- That would be easy to add.
I think you would have to think about
how many controls there are on the character.
So, for instance, if you already had two or three
other spots that the user is touching and controlling
during performance, it might be hard to schedule
switching from one of those to actually
picking up the dress and moving it.
- [Ariel] Hi. Ariel Weingarten from UCSD Design Lab.
You know, this tool eliminates a lot of work.
It's kind of like a form of automation,
and so you've freed up a lot of people's time.
Where in the creation process would they refocus that time,
or would they just like leave work early?
(laughter)
- Wherever they choose.
I mean, they could design more complicated characters,
or they could put more effort into their, perfecting
their performance that they want to do with the animation.
- [Ariel] Okay. Cool. Thanks.