Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today we are going to talk about OpenAI’s amazing image generator AI, DALL-E 2,
and you will see that it can do two things that perhaps not many of you have heard of.
So, what is DALL-E 2? This is a diffusion-based AI model, which means that it starts out from noise,
and gradually changes it to become a beautiful image like this one. The image can be anything,
so much so that we can enter a piece of text, and magically, out comes an image that fits
this description. It would be hard to overstate how transformative this will be to digital art.
Imagine that you could now illustrate any novel…and not only that, but it wouldn’t even need
any additional work for you to write up the text prompts.
Just take excerpts from the novel, and there we go. Wow.
And, if you feel like adding crazy prompts, as you see, it won’t complain about that either. But,
it can do even more. For instance, if we have an image that we really like,
variant generation is also possible. This means that we can provide it an input image, it tries
to understand what it depicts, and create several versions of it. And it does that spectacularly.
But, that’s not the only little-known feature is does, it also two other amazing
things. One is image inpainting. What is that? Inpainting means that we take an input image
or video, cut out an undesirable part, and let an algorithm fill in the holes with data that
makes sense given the context of this image. Now, previous techniques were already capable of this,
but I am very excited to see what DALL-E 2 can do with this problem, because we know that it has a
really detailed understanding of the world around us. So, can it use its knowledge of the world to
create high-quality inpaintings? Oh boy, you will see in a moment that it can do it incredibly well.
Let’s give it a try. Oh yes, we can delete this squirrel, and here comes the key: oh my! Remember
that I said that inpainting techniques fill in the holes given the context of the image.
But DALL-E 2 does something even better! It fills in the image with any context we can think of.
Let’s try that! How? Well, this image is ridiculous - squirrels are known not to be
able to operate computers, so, let’s add a hippo instead. And there we go! Much better. Loving it.
Now let’s push it to the limits through 5 outstanding examples!
One. The ladybug. Here is a photo, but, look. Unfortunately,
some parts of the photo are out of focus. Oh yes, how about deleting those parts, and have
them inpainted with DALL-E 2? Could that work? It knows that this is a ladybug after all. Let’s see.
Wow. That is much sharper and fits the image really well. I would have to look
for quite a bit to find out the trickery. What about you? Let me know in the comments below! So
this is fantastic. But you know what is even more fantastic? Now hold on to your papers,
because it can also do not only image inpainting, but image outpainting!
What is that? Well, two. Ever wondered what might be outside of this frame? Well, let’s ask DALL-E
2 and…that is insanity. It is really doing it. And we can just zoom out and out and out and it feels
like it never stops. And it just gets crazier over time. If a human would have made this,
I would say that human is very creative. But this was done by a machine. Is that a hint of
creativity in a machine? Food for thought. I love this one. What a time to be alive!
Three, of course, we need to do that to the Mona Lisa too. What could be outside the frame?
Ah, I see! This is a more modern take on the old classic. Working at the office while
everyone else is enjoying life. Note that once again, a prompt was used to generate
the theme of the outpainting here. These are the prompts that were used for this example.
And, these are so good, let’s do a couple more. Four. Zooming out from
a house in the middle of a lush field to this. Very artistic. And it turns out,
wow…that we are on a floating island in the middle of the universe…with a twist.
And, last, but not least. Five, the Last Supper. But this time, let’s not zoom out, but let’s try
to answer one of history’s big questions: and that is, what might be going on in the adjacent rooms.
Look. I love it. And I wonder what DALL-E 3 will be capable of? Remember, the DALL-E
2 version came out just about a year after the first version, and it was leaps and bounds better.
And the competition is also coming, Google already has their Imagen and Parti AI’s,
and even that is just scratching the surface. As you see, the pace of progress in AI research is
absolutely incredible. These tools are going to transform the world around us as we know it,
especially now that OpenAI is aiming to deploy this AI to no less than 1 million people in the
near future. How cool is that! Democratizing art creation for all of us. Soon. Bravo!
So, does this get your mind going? Do you have any thoughts as to what DALL-E 3 could do?
Let me know in the comments below!
Thanks for watching and for your generous support, and I'll see you next time!