Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Oh my, here is Stable Diffusion version 2. So, what is this? Stable Diffusion is
a free and open source text to image AI, which means that we write a piece of text,
and it creates exactly that image for us. All this, for everyone,
for free! So good. And now, here are 10 things you should know about version 2.
One, it can now generate images with higher resolution. More details for free, that sounds
amazing, but, it doesn’t stop there, because two, it can also perform super resolution better. Super
resolution means that in goes a coarse image, and out comes a beautiful image with a lot more
detail. And when I say a lot, I mean a lot. Just look at how much better this is. So good.
Three, it can also go from depth plus text to image. What does that mean? Well,
we can give it an input image, and then, it fires up this paper. Oh goodness,
that is a powerful paper, it can not only estimate depth maps for photos really well,
but get this, it can even imagine if a drawing were a real photo, and estimate the depth of
that. Incredible. So, it creates the depth map for this image, and using our prompt,
fills it with information that makes sense. This is excellent if we wish to just specify the pose
of our hero, and it will generate as many similar variations for it as we can imagine.
Or, we can also do this. Lots of amazing variants
for the same concept. The limit is only our imagination. Loving it.
Four, image inpainting is now easier. This means that we can say
that we wish to retain this part of the image, and the AI should just fill in the
rest according to our instructions. This is super fun, and it will help with one of the
most important things we could all ask for: and that is quick turnaround times for our ideas.
Five, it can now generate more convincing images of refractive objects. You know that
I am a light transport researcher by trade and this makes me very,
very happy. Human eyes, and water, underwater images, not a problem. And some of these
images are extremely convincing. These are especially impressive because they
came about from pure prompting, no guide or any input images were used.
Six, now hold on to your papers for photorealistic humans. Whoa, these are shockingly good.
Especially that we are also humans, most of us anyway, and we really have a keen eye for other
humans, and if even the smallest things are off, we notice immediately. And the details
in this version are finally at the point where it can create believable virtual characters.
But if photorealism is not your cup of tea, and you like virtual worlds better, do not despair,
seven it is also excellent at creating cyberpunk book covers.
Eight, we can also generate incredible interiors with it. Now, not all of them are perfect,
there are some flaws here, but the pace of progress in AI research is
nothing short of amazing. Not so long ago, we needed proper light simulation
programs and hours and hours of work to create an interior like this. And now,
just a text prompt. 5 seconds. And just imagine what this will look like just two more papers
down the line. What a time to be alive! Note that I put a link to the authors of these
images in the video description so make sure to check them out if you wish to see more of these.
Nine, the textures are also incredible. Here, you don’t see super fancy images,
just simple concepts that are executed really well. I just kept on looking and
looking at each of these images and I can barely find any flaws.
Ten, of course, I can only imagine how excited you Fellow Scholars are to try it,
so, can you? Yes, and it gets better, because you can still try it in two different ways.
One, if you are patient, you can write a prompt here. You might have to wait for a bit, but as
of the making of this video, it works. Now what happens when you Fellow Scholars get over there,
who really knows, we have crashed plenty of websites before with our Scholarly Stampede. But,
if you don’t want to wait or run some more advanced experiments,
you can run the model yourself at home on a consumer graphics card. Loving it.
And don’t forget, Stable Diffusion is free and open-source, and thus it has captured the
imagination of you Fellow Scholars. With this, you can pop the hood, take out your virtual wrenches,
and, let the experiments begin! And with this, AI-based image generation is only getting
cheaper and more democratized from here on out. A little open-source competition for OpenAI and
Google. Power to the people, and for free. Double thumbs up! What a time to be alive!
By the way, Google already has an AI that creates not only images, but videos from your prompts,
and if everything goes well, I may or may not have exclusive access to it. If it comes to fruition,
there will be a video on it with my own Scholarly Prompts, so consider
subscribing and hitting the bell icon, you really don’t want to miss it if it comes.
Thanks for watching and for your generous support, and I'll see you next time!