Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér.
Today we are going to talk about a paper that builds on a previous work by the name Deep
Image Priors, DIP in short.
This work was capable of performing JPEG compression artifact removal, image inpainting, or in
other words, filling in parts of the image with data that makes sense, super resolution,
and image denoising.
It was quite the package.
This new method is able to subdivide an image into a collection of layers, which makes it
capable of doing many seemingly unrelated tasks, for instance, one, it can do image
segmentation, which typically means producing a mask that shows us the boundaries between
the foreground and background.
As an additional advantage, it can also do this for videos as well.
Two, it can perform dehazing, which can also be thought of as a decomposition task where
the input is one image, and the output is an image with haze, and one with the objects
hiding behind the haze.
If you spend a tiny bit of time looking out the window on a hazy day, you will immediately
see that this is immensely difficult, mostly because of the fact that the amount of haze
that we see is non-uniform along the landscape.
The AI has to detect and remove just the right amount of this haze and recover the original
colors of the image.
And three, it can also subdivide these crazy examples where two images are blended together.
In a moment, I’ll show you a better example with a complex texture where it is easier
to see the utility of such a technique.
And four, of course, it can also perform image inpainting, which, for instance, can help
us remove watermarks or other unwanted artifacts from our photos.
This case can also be thought of an image layer plus a watermark layer, and in the end,
the algorithm is able to recover both of them.
As you see here on the right, a tiny part of the content seems to bleed into the watermark
layer, but the results are still amazing.
It does this by using multiple of these DIPs, deep image prior networks, and goes by the
name DoubleDIP.
That one got me good when I’ve first seen it.
You see here how it tries to reproduce this complex textured pattern as a sum of these
two, much simpler individual components.
The supplementary materials are available right in your browser, and show you a ton
of comparisons against other previous works.
Here you see results from these earlier works on image dehazing and see that indeed, the
new results are second to none.
And all this progress within only two years.
What a time to be alive!
If like me, you love information theory, woo-hoo!
Make sure to have a look at the paper and you’ll be a happy person.
This episode has been supported by Weights & Biases.
Weights & Biases provides tools to track your experiments in your deep learning projects.
It is like a shared logbook for your team, and with this, you can compare your own experiment
results, put them next to what your colleagues did and you can discuss your successes and
failures much easier.
It takes less than 5 minutes to set up and is being used by OpenAI, Toyota Research,
Stanford and Berkeley.
It was also used in this OpenAI project that you see here, which we covered earlier in
the series.
They reported that experiment tracking was crucial in this project and that this tool
saved them quite a bit of time and money.
If only I had an access to such a tool during our last research project where I had to compare
the performance of neural networks for months and months.
Well, it turns out, I will be able to get access to these tools, because, get this,
it’s free and will always be free for academics and open source projects.
Make sure to visit them through wandb.com/papers or just click the link in the video description
and sign up for a free demo today.
Our thanks to Weights & Biases for helping us make better videos for you.
Thanks for watching and for your generous support, and I'll see you next time!