Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Do you see this new table tennis robot? It barely played any games in the real world, yet, it can
return the ball more than a 100 times without failing! Wow. So, how is this even possible?
Well, this is a sim2real paper, which means that first, the robot starts learning in a simulation.
OpenAI did this by teaching their robot hand in a simulated environment to manipulate
this Rubik cube, and Tesla also trains its cars in a computer simulation. Why? Well,
in the real world, some things are possible, but in a simulated world, anything is possible! Yes,
even this. And the self-driving car can safely train in this environment,
and when it is ready, it can be safely brought into the real world. How cool is that?
Now, how do we apply this concept to table tennis? Well, in this case, the robot would not move,
but it would play a computer game in its head, if you will. But, not so fast! That is impossible!
What are we simulating exactly? The machine doesn’t even know how humans play! There is no
one to play against. Now check this out! To solve this, first, the robot asks for some human data.
Look, it won’t do anything, it just observes how we play. And it only requires these short
sequences. Then, it builds a model of how we play, then, embeds us into a computer simulation,
where it plays against us over and over and over again without any real physical movement.
Training the brain, if you will. And now comes the key step,
this knowledge from the computer simulation is now transferred to the real robot, and now,
let’s see if this computer game knowledge really translates to the real world! So,
can it return this ball? It can…well, kind of. One more time. Okay, better, and now…well,
it missed again. I see some signs of learning here, but this is not great. So, is that it? So
much for learning in a simulation and bringing this knowledge into the real world, right?
Well, do not despair, because there is still hope. What can we do? Well,
now it knows how it failed, and how it interacted with the human. Yes,
that is great! Why? Because it can feed this new knowledge back into the simulation. The
simulation can now be fired up once again, and with all this knowledge, it can repeat until
the simulation starts looking very similar to the real world. That’s when the real fun begins. Why?
Well, check this out. This is the previous version of this technique, and as you see, this does not
play well. So, how about the new method? Now, hold on to your papers, and marvel at this rally.
82 hits and not one mistake. This is so much better! Wow! This sim2real concept really works!
And, wait a minute. We are experienced Fellow Scholars here, so, we have a question. If the
training set was built from data when it played against this human being,
does it really know how to play against only this person? Or did it obtain more
general knowledge, and can it play with others? Well, let’s have a look!
The robot hasn’t played this person before. And let’s see how the previous technique fares.
Well, that was not a long rally. And neither is this one. And now, let’s see the new method.
Oh my, this is so much better. It learns much more
general information from the very limited human data it was given.
So, it can play really well with all kinds of players of different skill levels, here you see
a selection of them. And all this from learning in a computer game with just a tiny bit of human
behavioral data. It can even perform a rally of well over a hundred hits. What a time to be alive!
So, does this get your mind going? What would
you use this sim2real concept for? Let me know in the comments below!
Thanks for watching and for your generous support, and I'll see you next time!