Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
None of these faces are real.
And today we are going to find out whether these synthetic humans can, in a way, pass
for real humans, but not in the sense that you might think.
Now, through the power of computer graphics algorithms, we are able to create virtual
worlds, and of course, within those virtual worlds, virtual humans too.
So, here is a wacky idea.
If we have all this virtual data, why not use these instead of real photos to train
new neural networks?
Hmm…wait a second.
Maybe this idea is not so wacky after all.
Especially because we can generate as many of these virtual humans as we wish, and all
this data is perfectly annotated.
The location and shape of the eyebrows is known, even when it they are occluded, and
we know the depth and geometry of every single hair strand of the beard.
If done well, there will be no issues about the identity of the subjects, or the distribution
of the data.
Also, we are not limited by our wardrobe or the environments we have access to.
In this virtual world, we can do anything we wish.
So good!
But of course, here is the ultimate question that decides the fate of this project.
And that question is: does this work?
Now, we can give all this data to train a neural network, and the crazy thing about
this is that this neural network never saw a real human.
So here’s the ultimate test.
Videos of real humans.
Now, hold on to your papers and and, let’s see if the little AI can label the image and
find the important landmarks.
Wow.
I can hardly believe my eyes - it not only does it for a still image, but for even a
video, and it is so accurate from frame to frame that no flickering artifacts emerge.
That is outstanding.
And get this, the measurements say that it can stand up to other state of the art detector
neural networks that were trained on real human faces.
And…oh my!
Are you seeing what I am seeing?
Can this really be?
If we use the same neural network to learn on real human faces, it actually won’t to
better at all, in fact, it will do worse than the same AI with the virtual data.
That is insanity.
The advantages of this practically infinitely flexible synthetic dataset shows really well
here.
The paper also discusses in detail that that this only holds if we really use the synthetic
dataset well and include different rotations and lighting environments for the same photos.
Something that is not always so easy in real environments.
Now this test was called face parsing, and now comes landmark detection.
This also works remarkably well, but wait a second.
Once again, are you seeing what I am seeing?
The landmarks can rotate all they want, and it will know where they should be, even if
they are occluded by headphones, the hair, or when they are not visible at all.
Now, of course, not even this technique is perfect.
Tracking the eyes correctly requires additional considerations and real data, but only for
that, which is relatively easy to produce.
Also, the simulator at this point can only generate the head and the neck regions, and
nothing else.
But, you know the drill, a couple papers down the line, and I am sure that this will be
able to generate full human bodies.
Thanks for watching and for
your generous support, and I'll see you next time!