Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today I will show you some papers and results with Stable Diffusion,
a free and open source image generator AI. And I cannot believe how good these results are.
First, we had an earlier example with ControlNet and dancing. This means that in goes something,
for instance, a video of us, and out comes a hand drawn anime version of
it. Now, the advantage of this is that we don’t need to look perfect,
and we don’t even need to be at this exact place in the background, because all of this
can be synthesized. And today, someone took this idea to the next level. Look. And now,
if you think this is a blue haired woman converted into to a more painterly blue haired woman… well,
have a look at this. Oh yes, we don’t even need to be the character we want to be, we just need
to kind of look like the character we want to be. This is going to be an amazing tool for the
digital artists of the future. And remember, this needs a tiny bit of time and expertise,
but it is something that you can run for free on your own machine, and some of the
other examples, even in your pocket, on your smartphone. What a time to be alive!
Two, we can even use Stable Diffusion’s ControlNet to provide it input images for
it to redraw. For instance, we can give it this childhood drawing, and we can ask this little AI
to create a super high quality version of it. My goodness. This is incredibly good.
Three, now here is a crazy idea. Why not give it a QR-code as an input image, and see what
happens? Well, these look great…so this was a good thought experiment, but of course,
these won’t work as QR codes. Just as a piece of art. Wait a second. Hold on to your papers,
Fellow Scholars, do they actually work as QR-codes and are they readable from a smartphone camera?
Yes they are! Make sure to play with the distance a bit, perhaps increase the distance a little,
but every single one of these work for me. I can’t believe it. Artificial creativity. Wow.
Four, and these video to video workflows are becoming so incredibly good that…are
you seeing what I am seeing? Can that really be? These can now be generated
almost immediately. If we wish to create a smoother video, we would have to wait a bit,
but not in the order of hours. Just in the order of seconds, and super quick previews
can be done like this. This would not have been possible just a few months ago. Unbelievable.
Five, with the right prompts, Stable Diffusion can also create an 1800s
version of our favorite rockstars. Elton John, Freddie Mercury, Dave Grohl, Hendrix or Jagger,
you name it. I love how creative some of these solutions are and how they retain
their looks and personalities while putting them into an 1800s setting.
Six, we can also create custom portraits for our
dungeons and dragons characters. Just give the AI the description,
and there we go. The prompts and the workflow for some of these images are available,
which means that you can likely reproduce and tweak them yourself at home. So good.
Seven, in the meantime, Google just published a really cool paper on how to guide these image
generator AIs with natural language. First, we can ask for an image of a giant macaron
and croissant in front of the Eiffel Tower. Well, yes, I hear you saying, Doc, with today’s papers,
that is not surprising. What is surprising is that without generating a new image,
we can swap these objects, enlarge the macaron, and if we don’t like it,
replace it with a different one, or even better use our own photo and ask it to recapture its
appearance. Which it had done really well as the lighting now also has an effect on these
objects as well. Not just the background is different, everything is different. So,
interacting with text to image AIs via natural language. How cool is that?
Eight, creating zoom videos has also improved a great deal. Now we can choose a few images,
and stitch them together to create these zoom videos,
and they are now really smooth. Previous ones had some flickering issues,
but this one is now so much better. And all this improvement in just a few months! Unbelievable.
I put a link to all of these works in the video description,
and for now, let the experiments begin!
Thanks for watching and for your generous support, and I'll see you next time!