Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Dr. Papers.
Today you are going to see an insane paper. One of the most fun papers I’ve read in a while. Earlier,
scientists at DeepMind were looking to teach a bunch of AI agents to play football/soccer.
Unfortunately, they started out like this. Not very impressive. But then, they trained
them for 5 years. What? 5 years? Well, their training took what felt like 5-years for them,
but only 3 days for us, because these are not real robots, and thus, on a fast computer,
they can be simulated faster than real time. So what happened in that time?
This happened! Whoa! Now they can not only play amazingly well, but,
they can even think a few steps ahead, and they are even anticipating the
behavior of their teammates and position themselves accordingly.
The saves they are capable of are also outstanding. And as we don’t
really have a referee around, they also beat up each other. And when I say that,
I really mean it. It’s bad. Real bad. Also, the only thing they like more than throwing
each other is throwing tantrums. And it even gets worse. This lovely chap
almost seems like someone who is faking an injury, but then, look! A miracle happened!
And today, scientists at DeepMind brought this work to the next level. How?
In a different paper, scientists at OpenAI simulated a robot hand in a video game world.
Then, the things it learned in this simulation could be brought into life by deploying it into
the real world. So train in a simulation, and then play in the real world. Yes, that is fantastic,
but, why am I saying this? Oh wait, is is possible that the same happened here too? Real
robots playing football? Yes! That is exactly what happened. So with that, let the insanity begin.
The new environment contains 2 real robots with 20 actuated joints each. So what did they learn
first? Well, first, they learned to get up, and also, score goals. All this in a simulated robot
body. That is very impressive already, but wait, because that’s just the start. Now it is time for
random perturbations. What does that mean? It means that forces are randomly exerted on the
AI to make sure it can recover from an impact. And, are you thinking what I am thinking? Oh yes,
now let’s go to the real robots, and let’s test that impact. Aww! That is not very nice. But
then…wow! Look at that! What it had learned in a simulation now really applies to a real
life robot. It can get hit, and it is able to get up over and over again. My goodness. But,
there is a problem. What is the problem? Well, their knees get injured all the time. So what
did scientists at DeepMind do with that? Well, hold on to your papers Fellow Scholars,
because they taught the robot to get up and move in a way that is not too hard on the knees. Yes,
that’s right! And thus, they didn’t suffer that many knee injuries. What a time to be alive!
So with that, finally, they can now run, and score goals. But wait, playing a match and
winning takes more than just running and kicking. They need to have a variety of other skills and
combine them in order to win. So, did they? Oh yes they did! They can now control the ball,
turn with it, block the ball with their bodies, shoot accurately, even in the
case of a moving ball, which is even harder. And that is the miracle of research papers:
such incredible progress can happen just one more paper down the line.
And, yes, I know, I know. You are asking the question, okay, but Károly,
did they beat up each other here too like in the previous paper? And the answer is no. You see,
they are penalized for getting too close to each other, so luckily that did not happen.
But this kind of improvement in just one paper, I am wondering what these little AIs will be capable
of two more papers down the line? Well, what do you think? Let me know in the comments below!
Thanks for watching and for your generous support, and I'll see you next time!