What do I see here? This is DeepMind’s new AI playing video games. But, not just Atari Breakout,
where they built an AI that could play it incredibly well 8 years ago. And it is not
even like NVIDIA’s AI that can play Minecraft. It played Minecraft well, but only that.
You see, many of these works are tailored to one specific game. If you wish to play
a different game, you need a different AI. One AI, one game. And now, this is perhaps the first
time an AI can do well at a variety of modern 3D games at the same time. Yes! One AI, many games.
And even though DeepMind’s new AI learned to play a variety of games, the goal is not
quite to maximize the points or to beat the game, no-no. The goal is something else here.
Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Yes, the goal here instead is to be able to follow instructions from Fellow Scholars,
and that is us, humans. And it plays in a way that I like to call the Holy Grail of
AIs playing video games. And that is, not looking at numbers that describe the game,
like coordinates, scores and other data. No sir! It looks at the screen, reads the pixels,
and uses the keyboard and mouse to try to perform these actions, much like a human would.
That is incredibly difficult. An excellent paper for Two Minute Papers episode 850.
So when we ask it to drive that car in Goat Simulator, well, it drives that car, but it
is not impressive car driving. However, what is impressive is that it just looks at a 2D pile of
pixels, and from that, it understands the 3D world it represents, and how it changes over time. That
is not trivial at all, for us, this requires the human vision system. And now an AI can do that. It
sees not just a pile of pixels, it sees a bit more like we humans do. That is extremely impressive.
Especially for these complex 3D games. This was much easier for Atari Breakout,
but here, that is a huge leap in just 8 years. Wow.
And something from the paper really surprised me. I did not expect this at all. Now hold
on to your papers Fellow Scholars, because after it has trained on more than one game,
it was able to play all games better. Isn’t that the definition of intelligence? It is
able to gather knowledge from somewhere and apply it to somewhere else. So, can it do that? Oh my,
look at that! This is amazing. Here is the baseline performance of a specialist agent
that played one game for a long time, and here is the agent that played less, but has played several
different games. And it beats the specialist AI at its own game. And not by a little. Wow.
Now, limitations. Clearly, this is not human level performance. Yet. Success rate is quite
reasonable for a first crack at the problem, and we are not looking for a 100% everywhere,
even humans are not doing 100% on these tests, so there is room to grow. And
what do you think version 2 will score here? Let me know in the comments below.
Also, the length of these sequences is limited to 10 seconds, and it cannot do intense longer-term
strategic planning. For instance, you could ask it not just to jump the fence,
but to find resources and build a camp in a strategy game. That is going to be the plan
for the next follow up papers on this. But you remember the First Law of Papers. The First Law
Of Papers says that research is a process. Do not look at where we are, look at where we will be
two more papers down the line. And I think they may have broken through. I think that through
incremental improvements, we can get to something truly special from here. What a time to be alive!
And ultimately, this is not just an AI to play video games,
although I am quite sure there was lots of fun to be had at the DeepMind lab with
this paper. The goal is to be able to create AI systems and agents that can
understand us and help us out in a wide range of challenging tasks in a 3D world.
I will note that I had the luck of being able to access this paper a little earlier,
and by the time you are watching this I am likely on a plane to visit the DeepMind lab
to have a look at their newest works. I don’t yet know what it will be, but I can’t wait to
tell you more about it as soon as it will be possible to talk about them. Subscribe and
hit the bell icon if you don’t want to miss out on those. And thank you so much for watching me
reading papers and flipping out together with me for 850 episodes. This is a huge honor. Thank you.