Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today we are going to end the rule of
ugly, blurry photos with a new AI technique developed by Google.
You see, most smartphones today have multiple cameras, and can take these beautiful portrait
mode photos where the subject is in focus, and the rest of the image is blurred out.
This is all well and good if we have shot the image perfectly, and we have in focus exactly
the things that we want before pressing the button. If we missed our chance, unfortunately,
that’s it. Refocusing these images would be amazing, if we could do it. So, can we?
Well, unfortunately we can. Now I hear you asking Doc, why do you say, unfortunately we can? Well,
we have to use light fields to properly refocus an image. This unfortunately requires an expensive
and inconvenient array of cameras to capture all the required information in and out of focus.
Or, we can try focus stacking. That is, capturing a series of images with different
focus points. Focusing on different things in different photos. This is not expensive,
but unfortunately, it is time consuming, and not trivial to perform.
But, wait a minute. I just said at the start of the video that most smartphone brands today
have multiple cameras. That typically means a wide and an ultrawide lens.
If we take a photo with both of these lenses at the same time, we get two
images with different parts of the real scene in focus. If you think that sounds reassuring,
it gets worse, look. The two lenses give quite different results. The colors are different,
the focus is different. Hmm. Perhaps a learning-based algorithm could stitch
these two together into an image that we can refocus, but it looks quite difficult.
And today, Fellow Scholars, fortunately, that is a possibility. This is the result from a
work named DRBNet from just one year ago. This is supposed to deblur the whole image and give
us a sharp result. Well, it could have been a lot better, look here. Not too great. But now,
hold on to your papers Fellow Scholars, and have a look at the new technique. So much better.
And when we try the old technique for refocusing an image, we get this. But,
with the new technique, we get this instead. That is a huge difference just one more paper down the
line, especially that the previous work was published just a year ago.
And with this new work, we can now refocus all of these beautiful images. And even though none
of them are as perfect as having a real camera focusing at all the objects, I find these results
to be absolutely incredible. Not perfect by any means, but remember, the two lenses are really
different. This technique is like a classic music conductor having two different orchestras playing
together in harmony. Neither of the orchestras are perfect by themselves because they are
both missing some instruments, but together, if directed well, they can play the perfect symphony.
You can now take in this symphony, not with your ears, but with your eyes.
Now two more things that I absolutely loved about this paper. One is that it describes a
method that is not specific to a particular device. And two,
these ultrawide lenses are often of lower quality than the other lenses in smartphone
cameras and are typically underused and overlooked. But not here. Love it.
So, from now on, unusable blurry images of important
family moments shall be banished forever. What a time to be alive!
Thanks for watching and for your generous support, and I'll see you next time!