Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
This glorious paper from about 7 years ago was about teaching digital creatures to walk,
the numbers here showcase the process of learning over time, and it is clear that the later
generations did much better than the earlier ones.
These control algorithms are not only able to teach these creatures to walk, but they
are quite robust against perturbations as well, or, more simply put, we can engage in
one of the favorite pastimes of a computer graphics researcher, which is, of course,
throwing boxes at the character and seeing how well it can take it.
This one has done really well!
Well, kind of.
Now we just noted that this is a computer graphics paper, and an amazing one at that,
but it does not yet use these incredible new machine learning techniques that just keep
getting better year by year.
You see, these agents could learn to inhabit a given body, but I am wondering what would
happen if we would suddenly change their bodies on the fly?
Could previous methods handle it?
Unfortunately, not really.
And I think it is fair to say that an intelligent agent should have the ability to adapt when
something changes.
Therefore, our next question is, how far have we come in these 7 years?
Can these new machine learning methods help us create a more general agent that could
control not just one body, but a variety of different bodies?
Let’s have a look at today’s paper, and with that, let the fun begin.
This initial agent was blessed with reasonable body proportions, but of course, we can’t
just leave it like that.
Yes, much better.
And, look, all of these combinations can still walk properly, and all of them use the same
one reinforcement learning algorithm.
But do not think for a second that this is where the fun ends, no, no, no.
Now, hold on to your papers, and now let’s engage in these horrific asymmetric changes.
There is no way that the same algorithm could be given this body and still be able to walk.
Goodness!
Look at that!
It is indeed still able to walk.
If you have been holding on to your papers, good now squeeze that paper, because after
adjusting the height and thickness of this character, it could still not only walk, but
even dance.
But it goes further.
Do you remember the crazy asymmetric experiment for the legs?
Let’s do something like that with thickness.
And as a result, they can still not only walk, but even perform gymnastic moves.
Wo-hoo!
Now it’s great that one algorithm can adapt to all of these body shapes, but it would
be reasonable to ask, how much do we have to wait for it to adapt?
Have a look here.
Are you seeing what I am seeing?
We can make changes to the body on the fly, and the AI adapts to it immediately.
No re-training, or parameter tuning is required.
And that is the point where I fell off the chair when I read this paper.
What a time to be alive!
And now, Scholars, bring in the boxes.
Ha-haa!
It can also inhabit dogs and fish, and we can also have some fun with them as we grab
a controller and control them in real time.
The technique is also very efficient as it requires very little memory and computation,
not to mention that we only have to train one controller for many body types instead
of always retraining after each change to the body.
However, of course, this algorithm isn’t perfect, one of its key limitations is that
it will not do well if the body shapes we are producing strays too far away from the
ones contained in the training set.
But let’s leave some space for the next followup paper too.
And just one more thing that didn’t quite fit into this story.
Every now and then I get these heartwarming messages from you Fellow Scholars noting that
you’ve been watching the series for a while and decided to turn your lives around, and
go back to study more and improve.
Good work Mr. Munad!
That is absolutely amazing and reading these messages are a true delight to me.
Please keep them coming.
Thanks for watching and for your generous support, and I'll see you next time!