Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today we are going to have a look at how Photoshop just got transformed by the generative
AI revolution. And more. Previously, if we had an image with an object that we wanted to cut out,
but in the meantime, also fill in the gap with sensible information,
we could use the content-aware fill feature, and then, this happens. It has done a passable job,
and a little fiddling with the parameters probably yields an okay solution. So,
can we do even better? Well, through the power of the papers, yes, now we can!
Now, experiment number one. Let’s try the new generative fill feature where we can add a text
prompt, and it shall be filled with exactly that. So that is a wooden table, little AI,
please fill in a wooden table. Wait a minute. That is not exactly what I was thinking about,
but perhaps it did exactly what it was asked. That is a wooden table, can’t argue with that.
Alright, experiment two, this one was a sharp object in focus, but can it deal with blurred
objects in the background. Let’s change this bottle to a flower pot. Does it know
how this needs to be done? Will it be out of focus? That is an excellent solution,
great job! I like that we can choose from several different variants.
Now, experiment number three. I am a light transport researcher by trade, so I would
love to see some caustics, that is, light refracted through the glass here on the table.
And…ouch. Okay, this one is a little rough around the edges at the moment. Although if
we are looking for reflections, it can do that reasonably well.
Now, experiment number four. When trying this on not just some small thing,
but on higher quality images, created with some of the best generative AIs out there,
the quality differences are apparent. This will not always be able to keep up.
Experiment five. This is going to be very interesting and relevant. It can do not only
generative inpainting, but outpainting as well. Oh yes, if we say that we wish to extend an image
outwards, it can do that really well in this case. I don’t see any seams, or in other words, I don’t
see the traces of the algorithm anywhere. If I was shown this image, I would not be able to tell that
some wizardry was going on. Others also seemed to get really good results with this already.
However, don’t take these results and flaws for granted, these models are improving at
an incredibly rapid pace. For instance, a new paper describes a technique that is capable
of something even more incredible. And that is, magically changing the posture, proportions and
the pose of already existing images. This is incredible. And do you know what is even more
incredible? You are not going to believe this - the paper has only been out for a few days,
and an unofficial implementation of it already exists, which means that with a little technical
expertise, you can already try it yourself. The link is available in the video description.
And get this, there are even two new techniques for generating images without
even writing a text prompt. How is that even possible? Well, have a look at this. Here,
we just drop in an image, and it generates beautiful variants for it.
But, we are not done yet. Not even close. If we a looking for a bit more control,
but we still don’t want to enter a text prompt,
which basically sounds almost impossible, well, we have another new tool. Dear Fellow Scholars,
have a look at this prompt-free diffusion paper. Yes, this generates images,
and we don’t even need to enter text. All we need to do is add an image, and a little scribble,
and a new image shall be born that follows both the scribble and the style of the input image.
And everything that you saw here just happened not within the last few years,
but within the last few days. Wow. That is the amazing time we live in today. What a time to be
alive! The generative Photoshop extension is a bit behind the state of the art, that much is clear,
but it is super easy to use and will now be handed out to millions and millions of people. You see,
research scientists are writing these amazing papers to go beyond what is possible today,
and these are the next wave of techniques that you might see in industry standard
tools like Photoshop. So as you see, the papers are real. As real as it gets.
Thanks for watching and for your generous support, and I'll see you next time!