Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Approximately 7 months ago, we discussed an AI-based technique called StyleGAN2, which
could synthesize images of human faces for us.
As a result, none of the faces that you see here are real, all of them were generated
by this technique.
The quality of the images and the amount of detail therein is truly stunning.
We could also exert artistic control over these outputs by combining aspects of multiple
faces together.
And as the quality of these images improve over time, we think more and more about new
questions to ask about them.
And one of those questions is, for instance, how original are the outputs of these networks?
Can these really make something truly unique?
And believe it or not, this paper gives us a fairly good answer to that.
One of the key ideas in this work is that in order to change the outputs, we have to
change the model itself.
Now that sounds a little nebulous, so let’s have a look at an example.
First, we choose a rule that we wish to change, in our case, this will be the towers.
We can ask the algorithm to show us matches to this concept, and indeed, it highlights
the towers on the images we haven’t marked up yet, so it indeed understands what we meant.
Then, we highlight the tree as a goal, place it accordingly onto the tower, and a few seconds
later, there we go!
The model has been reprogrammed such that instead of towers, it would make trees.
Something original has emerged here, and, look, not only on one image, but on multiple
images at the same time.
Now, have a look at these human faces.
By the way, none of them are real and were all synthesized by StyleGAN2, the method that
you saw at the start of this video.
Some of them do not appear to be too happy about the research progress in machine learning,
but I am sure that this paper can put a smile on their faces.
Let’s select the ones that aren’t too happy, then copy a big smile and paste it
onto their faces.
See if it works!
It does, wow!
Let’s flick between the before and after images and see how well the changes are adapted
to each of the target faces.
Truly excellent work.
And now, on to eyebrows.
Hold on to your papers while we choose a few of them, and now, I hope you agree that this
mustache would make glorious replacement for them.
And there we go.
Perfect!
And note that with this, we are violating Betteridge's law of headlines again in this
series, because the answer to our central question is a resounding yes, these neural
networks can indeed create truly original works, and what’s more, even entire datasets
that haven’t existed before.
Now, at the start of the video, we noted that instead of editing images, it edits the neural
network’s model instead.
If you look here, we have a set of input images created by a generator network.
Then, as we highlight concepts, for instance, the watermark text here, we can look for the
weights that contain this information, and rewrite the network to accommodate these user
requests, in this case, to remove these patterns.
Now that they are gone, by selecting humans, we can again, rewrite the network weights
to add more of them, and finally, the signature tree trick from earlier can take place.
The key here is that if we change one image, then we have a new and original image, but
if we change the generator model itself, we can make thousands of new images in one go.
Or even a full dataset.
Loving the idea.
And perhaps the trickiest part of this work is minimizing the effect on other weights
while we reprogram the ones we wish to change.
Of course, there will always be some collateral damage, but the results, in most cases still
seem to remain intact.
Make sure to have a look at the paper to see how it’s done exactly.
Also, good news, the authors also provided an online notebook where you can try this
technique yourself.
If you do, make sure to read the instructions carefully and regardless of whether you get
successes or failure cases, make sure to post them in the comments section here!
In research, both are useful information.
So, after the training step has taken place, neural networks can be rewired to make sure
they create truly original works, and all this on not one image, but on a mass scale.
What a
time
to be alive!
Thanks for watching and for your generous support, and I'll see you next time!