Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today we are going to see how DeepMind’s incredible AI teaches these virtual
characters to move, play football/soccer and so much more. Now when reading this
paper I expected that they would learn the basics, but what happened afterwards, I did
not expect at all. They learned some absolutely amazing tricks and even more surprising things,
and yes, we are going to look at all of them, but first, the humble beginnings.
Now, this is a physics-based game, so to move, these AI agents need to use their joints and
produce just the right kind of forces and torque on them to create sophisticated movements and
amazing plays. So, let’s see. Oof. Unfortunately, we are seeing none of that here. Why?
Well, of course, because they haven’t had enough time to learn yet. So,
are these just the humble beginnings, or is this it? Is this the limit of their AI?
Not quite! But first, they need to learn to move. How? Well, scientists at DeepMind say by imitating
motion capture data. In other words, this tries to copy how real humans move. This is not bad, at
least it can now perform some basic movements, so that’s great. However, that is not nearly enough,
look. It cannot control the ball yet. Not even close. But once again, let’s come back 5 days
later and now, can it pull it off? Now, hold on to your papers, and…oh yes! Yes it can,
and with flying colors! It can now follow a moving target, dribble, kick, and so much more.
Now, get this, we will put these agents into a training camp for no less than 5
years and see how they do. We started out like this, and now. Holy mother of papers,
these little AIs can play! And they can do so much more than just moving around. Look,
they can do these little tricks where the ball bounces back from an invisible wall,
and not only that, but this player tries to pass to the other AI this way, and will they score?
Oh yes! Fantastic. I have to say, these are now pretty good players. Good job, little AI!
However, wait a second, what about the 5 years part? Did scientists at DeepMind have to wait
for 5 years for this paper and hope that something good comes out of it? Well, not quite! You see,
in real life, one second passes exactly in one second. That is not new information to
anyone. However, if we have a quick computer, in a simulation, one second in the game can
be simulated in a matter of milliseconds in real life, thereby speeding up time itself.
Not our time, of course, but the time that the AI lives in. So this 5-year training camp only
took 3 days in real life. That’s not that much. Hm. Are you thinking what I am thinking? Yes,
if it’s so quick, let’s run it for even longer. See what happens.
They ran it for 50 days, and let’s see the level of plays these can now do. And when
I saw these results, I almost fell off the chair. They have become even better. Let’s
see what they have learned. Now, they not only can think a few steps ahead, but they
are even anticipating the behavior of their teammates and position themselves accordingly.
They also learned to use body to body contact to their advantage.
High kicks are also being performed to catch the defenders off-guard, and I absolutely loved how
they developed these quick turns to get away with the ball. This is incredible. I love it.
And they can even play their way out of really difficult situations. Look at this one.
Oh boy, the defending red player would have to clear this ball without scoring an own goal,
so it would have to kick the ball in this direction, and the margin of error is razor
thin. So far, good thinking, but can it pull this off? Wow. Good job, little AI!
The saves they are capable of are also outstanding. And as we don’t
really have a referee around, these AIs haven’t exactly been encouraged
to be gentle with each other, so this can happen too. Ouch.
And here is one of my favorite moments, after a little altercation, the red player falls,
and it almost seems like throwing a little tantrum. This is super fun,
especially that he was the one who started it. I am sure some of you
Fellow Scholars will point out that this is not unlike some real football players.
And of course, this little AI appears so injured it cannot get up, but later when it is really
needed to be there to defend, look! A miracle happened! And of course, once again, this guy
just cannot help it. This is one of the most fun papers I’ve read in a while. So cool.
However, we are experienced Fellow Scholars over here,
so we would like to know so much more than what we’ve seen here. So,
let’s look under the hood and see some more about what they have really learned.
These are the curves that I am looking for, yes, and I see that their Elo
ratings increase over time. This means that as they learn more, they get better. That is
very reassuring, however, I still haven’t seen what my heart desires yet, so let’s continue.
Now we’re talking! More detailed data. This is so cool to see, look. Over time, they can get up
quicker, on average, they use their joints better to run faster, and yes! There we go. Division of
labor. Yes yes yes! Wow. That is what my heart desires. This is where we struck gold. This is
the graph that shows that these two players learned not only to move properly, but how to
work together as a team. This is also reflected in their passing frequency, which is increasing over
time. But the division of labor, that is where the real magic happens. You see, this is one of those
elusive qualities that is quite difficult to learn properly, often even for humans, and it really
shows here too. How? Well, look, this is one of the very few cases in the paper that does not show
steady linear growth. At times, it even worsens over time. After one day of training, the agents
know just enough to score a goal alone, and thus they become quite selfish. We all know that guy,
don’t we? But, over time, they realize that as a team, they can perform even better. Wow. What an
incredible piece of work. And this is the moment where I got goosebumps when reading this paper. I
love moments like this and I am so happy to share it with you. Remember, this is where we started,
and this is where we ended up, and all this through the power of modern learning algorithms.
Yes, this is Two Minute Papers, the corner of the internet where we look at a blue
curve and make happy noises. So good. If you enjoyed this, make sure to subscribe
and perhaps even hit the bell icon if you wish to see more amazing papers like this.
So, what do you think? What would you use this for? Let me know in the comments below!
Thanks for watching and for your generous support, and I'll see you next time!