little virtual soldiers to camp and teaches them how to fight.
This is a follow up work to an absolutely amazing paper called ASE. This is a previous
work with little AI soldiers that trained for 10 years. 10 years? How? 10 years of
their in-game world that is, which is fortunately when using a fast machine,
is only 10 days in our time. And they learned a great deal. They went from a bunch of sorry
little recruits to battle-ready soldiers. Except this chap. This chap goes like “Sir,
I’ve been training for two years, I’ve had enough! And now, I shall leave… in style.”
In this new work, these little soldiers trained for 5 billion training steps on a single graphics
card. It’s not a cheap one, but still, a single graphics card. So, what do we get for these 5
billion training steps? Can this new one do anything better than the previous one? Well,
first, it can learn from unlabeled motion capture data. What is that? Well, a bunch
of dots moving that represents movements of real humans that have been recorded.
Step number one. Basic training. Here, the moving dots go in, and a neural network is
asked to create motions that are similar to it. So, are we done? Well, not quite. You see,
we wish to use this in video games, and this is not controllable yet. Imagine a video game where
you press a button on the controller, and nothing happens. So now comes step number two. Precision
training! Here, it now needs to perform the movements, but also listen to our controls,
especially when we use the stick to steer it in different directions. And step number three,
here we not only give it directions, but intuitive commands as well, such as striking or running.
So, all that is great, but what can it really do now? Well, let’s see together.
For instance, it can interpolate between two movement types. This means that it works like a
gymnast who has perfected a difficult routine. It can't jump straight from one move to another. It
needs to transition smoothly to maintain balance and perform the routine successfully. Thus,
it first starts running, and then, look, it gradually dreams up a smooth transition from
sprinting to crouching. That is a really nice and believable transition. Love it.
And now comes the best part. Hold on to your papers Fellow Scholars,
because now you’ll see how it can do these types of transitions on-demand. That means,
as a reaction to our button mashing on the controller. So good!
However, not even this technique is perfect. It still has some issues to
be worked out. For instance, check this out. Now,
little AI, move towards the target. Attack, great and now, celebrate!
Whoa! Careful! Moments like this show that this work is still experimental. I don’t think you
will see this in a new game tomorrow, but this is what research is about. Taking something
highly unstructured and using learning-based techniques to create something useful from it.
Now, wait a second, in the intro, I also said that this is a follow up to the ASE paper. So
how does it compare to that one? Oh my. Are you seeing what I am seeing? Because I am seeing two
excellent news here. Now, I hear you asking, okay Doctor, what are the excellent news here?
Well, one, let’s start with the most obvious one - the controllability of the simulation has
improved leaps and bounds. That is incredible. And second, what is even more incredible,
this controllability did not come at the expense of the diversity of the movesets. That is perhaps
the best part of the paper - you see, this is in a class of neural network-based based techniques
that often suffers from a phenomenon called mode collapse. What is mode collapse? Well,
imagine asking a gymnast to perform incredible acrobatics, and this gymnast can do a variety
of things, but unfortunately, just keeps doing the same thing over and over again. Cartwheels
all day every day. Imagine looking at an olympic gold-medal athlete who just
keeps doing cartwheels. And, it is not the case here. This is an athlete that
can do everything. I absolutely love this. Bravo! What a time to be alive!
And finally, some fantastic news for the end - the source code of this project is available.
I would like to send a big thank you to the scientists on this project making it
available for all of us, free of charge. And for now, let the experiments begin!
Thanks for watching and for your generous support, and I'll see you next time!