Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today, we have an abundance of neural network-based image generation techniques.
Every image that you see here and throughout this video is generated by one of these learning-based
methods.
These can offer high-fidelity synthesis, and not only that, but we can often even exert
artistic control over the outputs.
We can truly do so much with these.
And if you are wondering, there is a reason why we will be talking about an exact set
of techniques, and you will see that in a moment.
So the first one is a very capable technique by the name CycleGAN!
This was great at image translation, or in other words, transforming apples into oranges,
zebras into horses, and more.
It was called CycleGAN because it introduced a cycle consistency loss function.
This means that if we convert a summer image to a winter image, and then back to a summer
image, we should get the same input image back.
If our learning system obeys to this principle, the output quality of the translation is going
to be significantly better.
Later, a technique by the name BigGAN appeared, which was able to create reasonably high quality
images and not only that, but it also gave us a little artistic control over the outputs.
After that, StyleGAN and even its second version appeared, which, among many other crazy good
features, opened up the possibility to lock in several aspects of these images, for instance,
age, pose, some facial features and more, and then, we could mix them with other images
to our liking, while retaining these locked-in aspects.
And of course, DeepFake creation provides fertile grounds for research works, so much
so that at this point, it seems to be a subfield of its own where the rate of progress is just
stunning.
Now that we can generate arbitrarily many beautiful images with these learning algorithms,
they will inevitably appear in many corners of the internet, so an important new question
arises - can we detect if an image was made by these methods?
This new paper argues that the answer is a resounding yes.
You see a bunch of synthetic images above, and real images below here, and if you look
carefully for the labels, you’ll see many names that ring a bell to our Scholarly minds.
CycleGAN, BigGAN, StyleGAN…nice!
And now, you know that this is exactly why we briefly went through what these techniques
do at the start of the video.
So, all of these can be detected by this new method.
And now, hold on to your papers, because I kind of expected that, but what I didn’t
expect is that this detector was trained on only one of these techniques, and leaning
on that knowledge, it was able to catch all the others!
Now that’s incredible.
This means that there are foundational elements that bind together all of these techniques.
Our seasoned Fellow Scholars know that this similarity is none other than the that the
fact that they are all built on convolutional neural networks.
They are vastly different, but they use very similar building blocks.
Imagine the convolutional layers as lego pieces, and think of the techniques themselves to
be the objects that we build using them.
We can build anything, but what binds these all together is that they are all but a collection
of lego pieces.
So, this detector was only trained on real images and synthetic ones created by the ProGAN
technique, and you see with the blue bars that the detection ratio is quite close to
perfect for a number of techniques, save for these two.
The AP label means average precision.
If you look at the paper in the description, you will get a lot more insights as to how
robust it is against compression artifacts, a little frequency analysis of the different
synthesis techniques, and more.
Let’s send a huge thank you to the authors of the paper, who also provide the source
code and training data for this technique.
For now, we can all breathe a sigh of relief that there are proper detection tools that
we can train ourselves at home.
In fact, you will see such an example in a second.
What a time to be alive!
Good news!
We now have an unofficial discord server where all of you Fellow Scholars are welcome to
discuss ideas and learn together in a kind and respectful environment.
Look, some connections and discussions are already being made - thank you so much for
our volunteering Fellow Scholars for making this happen!
The link is available in the video description, it is completely free, if you have joined,
make sure to leave a short introduction!
Meanwhile, what you see here is an instrumentation of this exact paper we have talked about,
which was made by Weights and Biases.
Weights & Biases provides tools to track your experiments in your deep learning projects.
Their system is designed to save you a ton of time and money, and it is actively used
in projects at prestigious labs, such as OpenAI, Toyota Research, GitHub, and more.
And, the best part is that if you are an academic or have an open source project, you can use
their tools for free.
It really is as good as it gets.
Make sure to visit them through wandb.com/papers or just click the link in the video description
and you can get a free demo today.
Our thanks to Weights & Biases for their long-standing support and for helping us make better videos
for you.
Thanks for watching and for your generous support, and I'll see you next time!