Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
In computer games and all kinds of applications where we are yearning for realistic animation,
we somehow need to tell our computers how the different joints and body parts of these
virtual characters are meant to move around over time.
Since the human eye is very sensitive to even the tiniest inaccuracies, we typically don’t
program these motions by hand, but instead, we often record a lot of real-life motion
capture data in a studio, and try to reuse that in our applications.
Previous techniques have tackled quadruped control, and we can even teach bipeds to interact
with their environment in a realistic manner.
Today we will have a look at an absolutely magnificent piece of work, where the authors
carved out a smaller subproblem, and made a solution for it that is truly second to
none.
And this subproblem is simulating virtual characters playing basketball.
Like with previous works, we are looking for realism in the movement, and for games, it
is also a requirement that the character responds to our controls well.
However, the key challenge is that all we are given, is 3 hours of unstructured motion
capture data.
That is next to nothing, and from this next to nothing, a learning algorithm has to learn
to understand these motions so well, that it can weave them together, even when a specific
movement combination is not present in this data.
That is quite a challenge.
Compared to many other works, this data is really not a lot, so I am excited to see what
value we are getting out of these three hours.
At first I thought we’d get only very rudimentary motions, and boy, was I dead wrong on that
one.
We have control over this character and can perform these elaborate maneuvers, and it
remains very responsive even if we mash the controller like a madman, producing these
sharp turns.
As you see, it can handle these cases really well.
And not only that, but it is so well done, we can even dribble through a set of obstacles,
leading to a responsive, and enjoyable gameplay.
About these dribbling behaviors.
Do we get only one, boring motion, or not?
Not at all, it was able to mine out not just one, but many kinds of dribbling motions,
and is able to weave them into other moves as soon as we interact with the controller.
This is already very convincing, especially from just three hours of unstructured motion
capture data.
But this paper is just getting started.
Now, hold on to your papers, because we can also shoot and catch the ball, move it around,
that is very surprising, because it has looked at so little shooting data, let’s see…yes,
less than 7 minutes.
My goodness.
And it keeps going, what I have found even more surprising is that it can handle unexpected
movements, which I find to be even more remarkable given the limited training data.
These crazy corner cases are typically learnable when they are available in abundance in the
training data, which is not the case here.
Amazing.
When we compare these motions to a previous method, we see that both the character and
the ball’s movement is much more lively.
For instance, here, you can see that the Phase-Functioned Neural Network, PFNN in short, almost makes
it seem like the ball has to stick to the hand of the player for an unhealthy amount
of time to be able to create these motions.
It doesn’t happen at all with the new technique.
And remember, this new method is also much more responsive to the player’s controls,
and thus, more enjoyable not only to look at, but to play with.
This is an aspect that hard is hard to measure, but it is not to be underestimated in the
general playing experience.
Just imagine what this research area will be capable of not in a decade, but just two
more papers down the line.
Loving it.
Now, at the start of the video, I noted that the authors carved out a small use-case, which
is training an AI to weave together basketball motion capture data in a manner that is both
realistic and controllable.
However, many times in research, we look at a specialized problem, and during that journey
we learn general concepts that can be applied to other problems as well.
That is exactly what happened here, as you see, parts of this technique can be generalized
for quadruped control as well.
This good boy is pacing and running around beautifully.
And…you guessed right, our favorite biped from the previous paper is also making an
introduction.
I am absolutely spellbound by this work, and I hope that now, you are too.
Can’t wait to see this implemented in newer games and other real-time applications.
What a time to be alive!
Thanks for watching and for your generous support,
and I'll see you next time!