We present SimpModeling, a novel
sketch-based system for assisting
amateur users to create desired
animalmorphic heads easily.
Creating visually reasonable 3D
characters has always been a fundamental
task in the modern game,
film and social media industries.
Articulated animalmorphic characters,
specifically their head shapes,
are highly demanded and play an important
role in 3D character design.
How to quickly create human heads has
been researched for a long time, but few
works have focused on how to quickly
create animal-like heads
due to their rich shape diversity and
rich geometric details.
There are various traditional
commercialized softwares for modeling
animal-like heads, such as Maya and
ZBrush.
However, these softwares require users to
possess considerable expertise and
manipulate lots of complex tools in 3D
modeling, which is labor-intensive and
time-consuming,
thus hindering the most novice users
from
instantiating their creative ideas.
Unlike traditional interactive modeling
pipelines,
sketch-based modeling enables amateurs
to get involved in 3D modeling in an
easy and intuitive way.
The research in sketch-based modeling
can date back to the last century.
As a seminal work of sketch-based
modeling, Teddy introduces a real-time,
interactive method for constructing 3D
shapes from input sketches.
FiberMesh aims to model freeform
surfaces from a collection of 3D
curves.
And these 3D curves would serve as
handles for further shape editing.
However, these traditional sketch-based
modeling systems are based only on
surface optimization methods,
which would result in over-smooth shapes
and lack of details.
In the past few years,
3D deep learning has become an option
for shape analysis and reconstruction.
Thanks to the recent advance in 3D deep
learning and rich datasets on human
faces
such as 3DMM and FaceWarehouse,
DeepSketch2Face proposed an
intuitive sketch modeling system
for modeling human faces
with diversified shapes and exaggerated
expressions,
but their system is limited to the
parametric human face space.
Due to the rich shape diversity over
animal heads,
parametrizing animalmorphic heads into
a low-dimensional shape space is very hard.
To address this, SAniHead introduced a
view-surface collaborative mesh
generative network to enhance geometric
details,
but it does not support accurate control
of global shapes
and it’s also hard to generate
reasonable geometric details.
Another limitation of deep-learning-based
animalmorphic heads modeling is the
lack of datasets.
To address these problems,
our key idea is as follows.
To depict rich shape diversity and rich
geometric detail of animalmorphic head,
as seen from the figure in the middle, we
separate coarse shape design
and geometric detail specification into
two stages
and respectively provide different
sketching means.
One stage is used to create global
shapes, and the other one is responsible
for adding local details.
Our experiment shows that this two-stage
pipeline can well support the accurate
control of global shapes and the
generation of local details.
To speed up the interactive modeling
process and further enhance the
generation of local details,
we introduce Advanced implicit-based
neural networks to our pipeline.
And we also constructed a large dataset
to allow our implicit neural networks to learn
the strong prior knowledge of
animalmorphic heads.
As you can see from the figure on the
upper right, compared with fibermesh,
which is only based on surface
optimization, our result looks more
natural due to the introduction of prior
knowledge
of the implicit neural network.
Based on the decompose of the pipeline, our system
provides two stages for mesh modeling:
coarse shape sketching stage and
geometric detail crafting stage.
Here is our workflow.
Now we use some examples to show the
workflow of our system.
In stage I, users start this stage by drawing a closed
curve on the sketch pad for depicting a
desired animalmorphic head profile
contour. Our system will then immediately
generate a smoothed shape as an initial
animalmorphic head mesh through
interpolation.
Then users can use the provided
extrusion tool to model head components,
like ears and horns.
Users can also use the add-curve tool to
add a 3D handle.
Besides, users can use the handle-deformation
tool to adjust the global
shape.
Next,
users can click the surface carving tool
to switch to Stage II.
In Stage II, users may intuitively carve
organic surface details, such as eyes and
noses.
Our deep-learning-driven backend will
immediately generate expected surface
details after users click the inference
hot-key.
In this section,
We will introduce the back-end algorithm
of our interactive sketch-based
modeling system.
In the Coarse Mesh Modeling Stage,
as seen in the left figure,
given 3D curves drawn by users as input,
our system generates an initial mesh M0
through surface optimization and
predicts a corresponding voxel-aligned
implicit field from the 3D curves.
Then the initial mesh M0 is deformed
with the guidance of the voxel-aligned
occupancy field.
After that, a coarse mesh M1 is generated.
Here is the pipeline of our voxel-aligned
implicit network. For more details
about the voxel-aligned implicit
network, please refer to our paper.
In Geometric Detail Crafting Stage,
once users draw detailed sketches on M1's
surface,
our system first renders the front and
back depth maps from the coarse mesh M1,
and renders a 2D sketch from the 3D
strokes.
Then a normal map is predicted by an
image-to-image translation network.
Here we use pix2pixHD.
Afterward, all these four images are
taken as input for the pixel-aligned
implicit learning module to predict a
pixel-aligned occupancy field.
Similarly, this occupancy field is used
to guide the deformation of M1
to produce a more detailed mesh M2.
Here is the pipeline of our pixel-aligned
implicit network.
For more details about the pixel-aligned
implicit network, please refer to our paper.
Now, Let’s take a look at the
dynamic process of our algorithm.
In Stage I, given 3D curves drawn by users
as input, our system generates an initial
mesh M0 through surface optimization and
predicts a corresponding voxel-aligned
implicit field from the 3D curves.
Then the initial mesh M0 is deformed
with the guidance of the voxel-aligned
implicit field.
After that, a coarse animalmorphic mesh
M1 is generated.
In Stage II, once users draw detailed
sketches on M1's surface, our system first
renders the front and back depth maps
from the coarse mesh M1, and a 2D sketch
from the 3D strokes.
Then a normal map is predicted by an
image-to-image translation network.
Afterward, all these four images are
taken as input for the pixel-aligned
implicit learning module to predict a
pixel-aligned occupancy field.
Similarly, the learned pixel-aligned
occupancy field is used to guide the
deformation of M1 to produce a more
detailed mesh M2.
Here is our dataset: 3DAnimalHead,
which is the largest animalmorphic
head dataset by far.
Here are some representative models
created using our system. It can be seen
from this figure that our system
supports users with limited knowledge in
geometrical modeling to create
animalmorphic heads with diversified shapes
and rich geometric details.
Here are some representative models
selected from the Comparison Study.
Obviously, given the limited time, the
participants can create more appealing
shapes and vivid details with our system.
Finally, let’s see the results gallery.
The final is the gallery of some
representative models created using our
system by 15 amateur users.
It can be seen that our system supports
users with limited modeling knowledge to
create animalmorphic heads with
diversified shapes and rich geometric
details. Thank you for watching.