Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér.
Whenever we build a website, a video game, or do any sort of photography and image manipulation,
we often encounter the problems of image downscaling, decolorization, and HDR tone mapping.
This work offers us one technique that can do all three of these really well.
But first, before we proceed, why are we talking about downscaling?
We are in the age of AI, where a computer program can beat the best players in Chess
and Go, so why talk about such a trivial challenge?
Well, have a look here.
Imagine that we have this high-fidelity input image and due to file size constraints, we
have to produce a smaller version of it.
If we do it naively, this is what it looks like.
Not great, right?
To do a better job at this, our goal would be that the size of the image would be reduced,
but while still retaining the intricate details of this image.
Here are two classical downsampling techniques…better, but the texture of the skin is almost completely
lost.
Have a look at this!
This is what this learning-based technique came up with.
Really good, right?
It can also perform decolorization.
Again, a problem that sounds trivial for the unassuming Scholar, but when taking a closer
look, we notice that there are many different ways of doing this, and somehow we seek a
decolorized image that still relates to the original as faithfully as possible.
Here you see the previous methods that are not bad at all, but this new technique is
great at retaining the contrast between the flower and its green leaves.
At this point it is clear that deciding which output is the best is highly subjective.
We’ll get back to that in a moment.
It is also capable of doing HDR tone mapping.
This is something that we do when we capture an image with a device that supports a wide
dynamic range, in other words, a wide range of colors, and we wish to display it on our
monitor, which has a more limited dynamic range.
And again, clearly there are many ways to do that.
Welcome to the wondrous world of tone mapping!
Note that there are hundreds upon hundreds of algorithms to perform these operations
in computer graphics research.
And also note that these are very complex algorithms that took decades for smart researchers
to come up with.
So the seasoned Fellow Scholar shall immediately ask - why talk about this work at all?
What’s so interesting about it?
The goal here is to create a little more general, learning-based method that can do a great
job at not one, but all three of these problems.
But how great exactly?
And how do we decide how good these images are?
To answer both of these questions at the same time, if you’ve been watching this series
for a while, then you are indeed right, the authors created a user study, which shows
that for all three of these tasks, according to the users, the new method smokes the competition.
It is not only more general, but also better than most of the published techniques.
For instance, Reinhard’s amazing tone mapper has been an industry standard for decades
now, and look, almost 75% of the people prefer this new method over that.
What required super smart researchers before can now be done with a learning algorithm.
Unreal.
What a time to be alive.
A key idea for this algorithm is that this convolutional neural network you see on the
left is able to produce all three of these operations at the same time, and to perform
that, it is instructed by another neural network to do this in a way that preserves the visual
integrity of the input images.
Make sure to have a look at the paper for more details on how this perceptual loss function
is defined.
And, if you wish to help us tell these amazing stories to even more people, please consider
supporting us on Patreon.
Your unwavering support on Patreon is the reason why this show can exist, and, you can
also pick up cool perks there, like watching these videos in early access, deciding the
order of the next few episodes, or even getting your name showcased in the video description
as a key supporter.
You can find us at patreon.com/twominutepapers, or, as always, just click the link in the
video description.
Thanks for watching and for your generous support, and I'll see you next time!