Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today we are going to generate human faces, and even better, we will keep them intact.
You will see what that means in a moment.
This new neural network-based technique can dream up completely new images, and more.
However, this is not the first technique to do this, but this…this does them better.
Let’s look at 3 amazing features that it offers, and then discuss, how, and why it
is better than its predecessors.
Hold on to your papers for the first example, which is my favorite, image toonification.
Would you like to see what the AI thinks you would look like if you were a Disney character?
Well, here you go.
And these are not some rudimentary, first-paper-in-the-works kind of results.
These are proper toonifications, you could ask for money for some of these, and they
are done completely automatically by a learning algorithm.
Woah.
At the end of the video, you will also witness as I myself get toonified.
And what is even cooler is that we can not only produce these still images, but even
compute intermediate images between two input photos, and get meaningful results.
I’ll stop the process here and there to show you how good these are.
I am blown away.
Two, it can also perform the usual suspects.
For instance, it can make us older or younger, or put a smile on our face too.
However, three it works not only on human faces, but cars, animals, and buildings too.
So the results are all great, but how does all this wizardry happen?
Well, we take an image, embed it into a latent space, and in this space, we can easily apply
modifications.
Okay…but what is this latent space thing?
A latent space is a made-up place where we are trying to organize data in a way that
similar things are close to each other.
What you see here is a 2D latent space for generating different fonts.
It is hard to explain why these fonts are similar, but most of us would agree that they
indeed share some common properties.
The cool thing here is that we can explore this latent space with our cursor, and generate
all kinds of new fonts.
You can try this work in your browser, the link is available in the video description.
And, luckily, we can build a latent space, not only for fonts, but, for nearly anything.
I am a light transport researcher by trade, so in this earlier paper, we were interested
in generating hundreds of variants of a material model to populate this scene.
In this latent space, we can concoct all of these really cool digital material models.
A link to this work is also available in the video description.
Now, for the face generation algorithms, this embedding step is typically imperfect, which
means that we might lose some information during the process.
In the better cases, things may look a little different, and that’s not even the worst
case scenario.
I’ll show you that in a moment.
For the milder case, here is an earlier example from a paper by the name StyleFlow, where
the authors embedded me into a latent space and it indeed came out a little different.
But not so bad.
A later work, StyleClip, was able to make me look like Obi-Wan Kenobi, which is excellent.
However, the embedding step was more imperfect.
The bearded image was embedded like this.
You are probably saying that this looks different, but even this is not so bad.
If you want to see a much worse example, look at this.
My goodness.
Now this is quite different.
Now that we saw what it could do, it is time to ask the big question.
How much better is it than previous works?
Do we have an A-B test for that?
And the answer is yes, of course!
Let’s embed this gentleman and see how he comes out on the other end.
Well, without the improvements of this paper, once again, quite different.
The beard is nearly gone.
And when we toonify the image, let’s see…yup, that beard is gone for good.
So, can this paper get that beard back?
Let’s see…oh yes!
If we refine the embedding with the new method, we get that glorious beard back.
That is one heck of a toon image, congratulations!
Loving it.
And now, it’s my turn.
One of the results was amazing, I really like this one.
How about this one?
Well, not bad.
And I wonder if it can deal with sunglasses?
Well, kind of, but not in the way you might think.
What do you think?
Let me know in the comments below!
Note that you can only see these results here on Two Minute Papers, and a big thank you
to the authors for taking time off their busy day and doing these experiments for us.
And here are a few more tests, let’s see how it fares with these.
The inputs are a diverse set of images from different classes, and the initial embeddings
are, well, a bit of a disappointment.
But that is kind of the point, because this new technique does not let it stop there and
iteratively improves them, yes, getting better, and by the end, my goodness.
Very close to the input.
Don’t forget that the goal here is not to implement a copying machine, the key difference
is that we can’t do too much with the input image, but after the embedding step, we can
do all this toonification and other kinds magic with it, and the results are only relevant
as long as the two images are close.
And they are really close.
Bravo!
So good!
So, I hope that now you agree that the pace of progress in machine learning and synthetic
image generation is absolutely incredible.
What a time to be alive!
Thanks for watching and for your generous support, and I'll see you next time!