Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today we are going to see some amazing, and some less amazing results from Stable Diffusion XL,
the popular text to image AI’s new version that you can also run for free online,
or even at home. So, what’s new here?
Well, one it offers higher resolution images and is better at challenging
concepts that previous text to image AIs faltered at. Human hands,
very specific spatial arrangements where we say that a woman should be in the background
chasing a dog in the foreground. These now work better. However, I’ll immediately say
that this does not mean perfect. Here are some of the results I generated locally. Obviously,
it cannot be anything else but images of Scholars holding on to their papers, or at least studying a
huge magical tome. It can do that, however, look. Hands still seem to be an issue.
If you have a favorite artist, and you love the style, but would like to imagine what if
he explored painting different subjects, you can now do it at home, for free. I think this
is an amazing tool to explore new artistic ideas, and I have to say it is incredibly fun too. And,
when comparing to Midjourney, I think the quality of the results are better there,
however, SDXL seems to be more true to the original style of the artist. Loving it.
I could also not resist trying Danielle Baskin’s amazing drink prompts,
which are also working quite well. Yummy.
Now, they say that users generally prefer the new technique’s results
to previous versions of Stable Diffusion, although I haven’t seen this user study
being linked to a peer-reviewed paper, so a wise Fellow Scholar like you are
does not take these results for granted. We are going to try some more ourselves.
Two, simpler prompting. If you have played with previous versions of Stable Diffusion,
you probably noticed that it requires a very detailed description of an image to create
something decent. But here, it is easier to create something with just a few words. My
experiments have mostly matched this. When asking for illustration of a small modern house in Osaka,
I got quite usable images that I liked. This layered cake in the style of a
landscape was also just a couple words, and the results are incredible. This is so much
fun! I hope you Fellow Scholars will also come up with some crazy ideas using this.
Three, it finally supports better text generation. Now, writing text is typically quite difficult
for text to image AIs. When I asked for the full “Two Minute Papers” in writing, I did get better
results than with most previous techniques I’ve used, but most of the time, this was a
little too much to ask for. After many tries, I had some success with it. Kind of. However,
I found that when asking for just “SDXL”, even then it took a bit of trial end error,
but I finally started seeing some more success. We are not there yet, but this is just the 1.0
version of Stable Diffusion XL, so I am excited to see how it improves over time.
Also do you remember ControlNet? ControlNet is a neural network structure that helps us give it
additional inputs beyond just text to image. And it is absolutely amazing. For instance,
we can provide just the edges of an input image. A rough sketch, or edges extracted
from a real photo, and we get a beautiful image with exactly that creature and framing. Other
kinds of inputs also work with ControlNet, and as the preliminary results show, this
feature is coming soon to Stable Diffusion XL, which will bump up its usability significantly.
And we get all of these for free, forever. That is excellent. It is all so new and so fresh,
there aren’t too many results out there yet. And don’t forget, there are many
ways to improve this base model through checkpoints and magical things called LoRAs,
so specialized versions of SDXL will come out as soon as weeks,
or perhaps days from now. It will be so much better. What a time to be alive!
And if you wish to try it in your browser or run it locally, I got you covered with
some links in the video description. So for now, let the experiments begin!
Thanks for watching and for your generous support, and I'll see you next time!