Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today we are going to marvel at the amazing things that happen when a paper like Stable Diffusion
comes alive as a free and open text to image AI. To say that it is improving at a rapid pace would
be an understatement, and it gets even better, it also gives you the warm and fuzzy feeling of
running it yourself on your own hardware. Here are 8 amazing, non-trivial applications of it.
One, it can be combined with several other tools to create something magical. Here,
you see a ChatGPT prompt, then, a request to a different text to image AI to create the images,
finally, get this, these were combined into a video by Stable Diffusion. Lovely work. So cool!
Two, this incredible photorealistic image was created by a custom model.
These are typically specialized models that are fine-tuned to do particularly
well at a given topic. This one is for photorealism, but there are tons
of other ones out there as well that make Stable Diffusion an absolute powerhouse.
Three, we can also create a set of images,
and weave them together into one of those famous zooming videos. Depending on how we
choose the start and endpoints, we can even make gorgeous infinitely zooming videos. Loving it.
Four, I hear you asking, okay Károly, but I already work with existing tools like
Photoshop. Can I make Stable Diffusion work with it seamlessly? And the answer is yes! Finally!
Look at that! You can now write your prompts, and even remix your real or generated images
by inpainting them. This combines the powerful built-in tools of Photoshop with text to image.
I absolutely love this. Note that this requires a little fiddling, but all the information you need
is there in the video description if you wish to give it a try. This is going to change how
artists use Photoshop and make them a lot more productive. What a time to be alive!
And five, if you do not wish to use it in photoshop, it can still supercharge your
creativity if you are looking to draw only a rough outline for your ideas and fill them
in through text to image. This gives you even more granular artistic control over
the generated images. I think applications like this are fantastic because they put the
human in charge much more than just generating an image from thin air using a text prompt. So good!
Six, okay, but all of these images are 2D. What about 3D artists, can they also benefit from
Stable Diffusion somehow? Does it help them in any way? Well, look at that! Wow! It just textured
these 3D objects, and not one by one. No sir. It textured the whole scene in one go. My goodness.
Seven, it can also create tileable textures. Why does that matter? Well,
if we are asking for a big patch of material, we need to make sure that it is not just one image
copied and pasted all over again, but, it truly gives us a seamless texture. That is excellent,
especially if we combine it together with an alternative reality app where we look at our room,
and just ask a voice assistant to recolor the wallpaper to be a little more corn-themed. Wow,
that is a little too much corn. What about making it a little less
busy and more stylized? Now we see exactly what that would look like.
And finally, eight. This is going to be incredible. If we have an animation of
a digital human and we wish to give its appearance a complete makeover,
look no more. Stable Diffusion can take this video, and just one image of the new test subject,
and, bam, there we go. We get a completely new video with a much higher quality character.
And before we go, I have even better news. Yes, hold on to your papers Fellow Scholars,
because Stable Diffusion even runs on your phone too. Just imagine having this kind
of power in your pocket, wherever you go. And don’t forget, this is all out there in the open,
so you can contribute too, whether it might be code, prompts, experiments papers, anything. We
are building this together. Oh, and you can also track your experiments with Weights & Biases,
and it can even help you optimize the efficiency of your own image generation process. It is super
useful, that’s why I always talk about it here on Two Minute Papers. So I hope this
gave you some ideas to experiment at home or on the go, and for now, let the experiments begin!
Thanks for watching and for your generous support, and I'll see you next time!