Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Two years ago, in 2019, scientists at OpenAI published an outstanding paper that showcased
a robot arm that could dexterously manipulate a Rubik’s cube.
And, how good were the results? Well, nothing short of spectacular. It could not only dexterously
manipulate and solve the cube, but they could even hamstring the hand in many different
ways and it would still be able to do well. And I am telling you, scientists at OpenAI
got very creative in tormenting this little hand. They added a rubber glove, tied multiple
fingers together, threw a blanket on it, and pushed it around with a plush giraffe and
a pen. It still worked…but probably had nightmares about that day in the lab. Robot
nightmares, if you will.
And now, let’s see what is going on over DeepMind’s side, and have a look at this
work on manipulating objects. In this example, a human starts operating this contraption.
And after that step is done, we leave it alone, and our first question is, how do we tell
this robot arm what it should do? It doesn’t yet know what the task is, but, we can tell
it by this reward sketching step. Essentially, this works like a video game, not in the classical
setting, but the other way around! Here, we are not playing the video game, but we are
the video game. And the robot is the character that plays the video game, in which we can
provide it feedback and tell it when it is doing well or not. Then, from these rewards,
it can learn what the task is.
So, that all sounds great, but what do we get for all this work? Four things.
One, we can instruct it to learn to lift deformables. This includes a piece of cloth, a rope, a
soft ball and more.
Two, much like OpenAI’s robot arm, it is robust against perturbations. In other words,
it can recover from our evil machinations. Look! We can get creative here too, for instance,
we can reorganize the objects on the table, nudge an already grabbed object out of its
gripper, or simply just undo the stacks that are already done. This diligent little AI
is not phased it just keeps on trying and trying, and eventually, it succeeds. A great
life lesson right there.
And three, it generalizes to new object types well and does not get confused by different
object colors and geometries.
And now, hold on to your papers for number four, because here comes one of the most frustrating
tasks for any intelligent being, something that not many humans can perform. Inserting
a USB key correctly on the first try. Can it do that? Well, does this count as a first
try? I don’t know for sure, but Dear Fellow Scholars, the singularity is officially getting
very, very close. Especially given that we can even move the machine around a little,
and it will still find the correct port. If only the first machine could pronounce my
name correctly, then we could conclude that we have reached the singularity.
But wait, we noted that first, a human starts controlling the arm. How does the fully trained
AI compare to this human’s work? And, here comes the best part. It learned to perform
these tasks faster. Look, by the 6 second mark, the human started grabbing the green
block, but by this time, the AI is already mid-air with it, and by 9 seconds, it is done,
while the human is still at work. Excellent! And we get an 80% success rate on this task
with only 8 hours of training. That is within a single working day.
One of the key ideas here, and the part that I liked best is that we can essentially reprogram
the AI with the reward sketching step. And I wonder what else this could be used for
if we did that? Do you have some ideas? Let me know in the comments below!
Thanks for watching and for your generous support, and I'll see you next time!