- What if I told you there is a system right now
that can take natural language input,
so whatever description you want, just make something up,
and it will take that text
and turn it into a surprisingly realistic image
of exactly what you described.
So you type an astronaut riding a horse,
and it spits out a brand new image
of an astronaut riding a horse.
You type teddy bears shopping for groceries,
and boom, there's an image of teddy bears
shopping for groceries.
You type a bowl of soup
that is a portal to another dimension,
and boom, my god, it's a bowl of soup
that's a portal to another dimension.
And it's not just one.
It actually spits out 10 different versions
across a spectrum of variation in any art style you want.
You name it and it can draw it.
So what is happening here?
How does it work?
And what happens if I try?
(relaxed music)
So first things first, yes, this does exist.
This is a real thing.
It's called Dall-E 2,
and it's an AI research project by a company called OpenAI,
one of many Elon Musk co-founded companies at this point.
And so the purpose of this AI, specifically,
is to create original, realistic images and art
from a text description.
This is Aditya Ramesh, a researcher and co-creator
of DALL-E 1 and DALL-E 2.
He's easily the most qualified person to explain
what's happening here.
- So the way DALL-E 1 generates images,
DALL-E 1 generates an image starting from the top left
and moving (voice muffling) order row by row.
So diffusion works completely differently.
The way diffusion works is we train a model
to reverse corruption process
that's applied to clean images.
- So it's kinda hard to wrap your head around,
but basically there's two main AI technologies
behind DALL-E 2.
They're called clip and diffusion.
So clip is the part that's matching images to text,
and basically uses that match to train the computer
to understand concepts in images,
so it can generate new images of the same concepts.
So when I asked it for an astronaut riding a horse,
for example, it's not just making a mosaic of images
it found online.
It knows the idea of what an astronaut is.
It knows what the concept of riding means.
It knows what a horse is.
And maybe most impressively,
it knows what's an aesthetically pleasing image to humans.
So then it can create a completely new visual version
of this idea that hasn't existed before.
Now, clip doesn't really have the ability
to do the pretty, high resolution images all by itself.
It's just more generating the gist of an image
based on those concepts.
So that is where diffusion comes in.
So diffusion is super impressive.
Basically by teaching a computer to corrupt an image
by adding Gaussian noise, it can then learn to un-corrupt
or enhance an image by removing that noise.
It's kinda like step one, draw the circle,
step two, draw the rest of the owl.
So I don't know if you've ever seen this website
called thispersondoesnotexist.com,
but if you haven't you should check it out.
It shows you a surprisingly realistic image of a face,
but as you might have guessed, this person does not exist.
It's not a real face.
It's actually using AI to look at thousands of faces
and then generate a new face with that information
that is shockingly realistic,
but it turns out is not a real human.
So DALL-E, DALL-E 2,
is like a way more advanced generalized version
of that for anything.
So when you open it up,
it's literally just a blank search text box
where you can type in whatever you want it to create.
Now, of course, as you can probably imagine
with all these concerns and possibilities
this isn't just a tool that's available to the public.
It's not like anyone can use it.
OpenAI has kept this mostly behind closed doors
to a very small, hand-selected group of people.
But for a day they gave me the keys,
and I was able to generate whatever I want,
which of course means I had to ask it
to finally reveal to us
what the long awaited Apple Car would look like.
I mean, this is an opportunity unlike any other.
So I typed it in, I waited my 10 seconds with bated breath,
and then the secret was finally unveiled.
- Good morning. - Oh, right, of course.
I don't know why I expected anything different.
But for real, okay, so the OpenAI team
was kind enough to allow me to feed DALL-E 2
whatever I want.
So I decided to start pretty simple,
and then get a little bit more complex as we go.
So a blue apple in a bowl of oranges.
So, okay.
These are good.
These are actually...
I mean, that was extremely easy.
But the sharpness, the realism, the lighting even,
to just create these brand new out of nothing,
there is so much detail in this one.
It's kinda hard to believe it isn't real.
Okay, an elderly kangaroo.
I mean, I don't know what I expected specifically
an elderly kangaroo to look like.
I guess maybe I pictured gray hair or something,
but I buy it.
I mean the fact that it's, again,
it's not a real photo,
but it looks like a real photo of an elderly kangaroo.
That is very impressive.
A wise elephant staring at the moon at night.
Whoa, okay, so that is definitely a wise elephant.
He or she is in fact staring at the moon,
and it is definitely at night.
That's not bad.
The moon does look a little bit wonky.
If you look a little bit closer on some of these.
It's not perfect.
But the elephant is very real looking.
Okay, let's get a little more specific here.
A teddy bear doing surgery on a grape
in the style of a 1990s cartoon.
Oh my god, look at these cartoons.
Sometimes it misses.
Totally understandable.
Also it seems to have chosen scissors
instead of maybe a more realistic actual surgery.
I'll get to why in a minute.
But the facial expressions, the feet, and everything.
I mean, that is a teddy bear doing surgery on a grape.
All right, this one's for Mac the studio dog.
A kooikerhondje,
I'm pronouncing that wrong,
using a camera on a movie set.
Wow.
That is, okay, if you couldn't already tell,
that is the name of the dog breed,
and the closer you inspect each individual image
the more the photorealism part kind of falls apart,
which maybe isn't shocking,
'cause this is a kind of a crazy thing to have a picture of,
but the detail in the dog breed,
and it actually using the camera in the pictures,
is crazy good.
I wonder if we could post that to Mac's Instagram,
if anybody would notice that it's not a real picture.
They'd probably figure it out.
All right, a robot woman guarding a wall of computers.
Wow, okay.
So many interesting details
and decisions being made in these images.
So the word guarding implies a bit of a pose,
and there's a couple different guarding poses here,
but that's cool.
The computers, for the most part,
are also pretty convincing if you don't zoom in too much.
And also it's interesting
that none of the walls of computers
go all the way up to the ceiling either,
which is cool.
But that is definitely a robot woman
guarding that wall of computers.
What if we go a tiger discovering the lost city of Atlantis?
Wow, okay, these are more of an art style,
probably because, one,
there won't be any photorealistic reference images
of the lost city of Atlantis.
So I imagine it'll look better this way.
And two, this is a crazy image to create.
So with each of these,
they're great without zooming in and pixel peeping,
and they very much accomplished the goal
of illustrating a tiger discovering Atlantis,
like I asked.
The crazy part here, though,
to me is how much imagination it's using.
Like I'm actually getting more than what I asked for.
The facial expressions, poses,
orientation of things, reflections,
even the accurate lighting and shadows is crazy.
I asked for a tiger discovering Atlantis here,
but it's decided to add trees and birds
and a moon all by itself.
All right, here we go, here we go.
A painting inspired by the Mona Lisa
of a goat taking pictures with an iPad.
I...
This is my new favorite thing.
You can really just go off the rails with complexity
and it just gets them right.
Almost all of these goats have hands, too,
which is hilarious,
but the drawings themselves
have actually also stayed true to the theme.
It's a painting in the style of the Mona Lisa,
and the tablets are all varying levels of convincing iPads.
Wow.
I'm gonna put these all on Twitter, by the way,
in one big thread,
plus a few extras if they don't make it into the video.
So definitely hit the link below if you wanna see those,
but last but not least.
A cyclops riding a tractor listening to AirPods
in the style of the Simpsons.
I mean, come on.
Maybe it's not a perfect cyclops,
and it is interesting
that it's chosen over the ear headphones
for all of the headphones and not AirPod earbuds,
but I feel like there's nothing this can't do.
This is one of those AI tools that's so good
that it almost brings up more questions than it answers.
Like why does a tool like this
even exist in the first place?
Well, DALL-E 2 is a research project,
not a customer product,
and OpenAI's goal is to create good, safe general AI,
which is really hard.
Like there are a lot of really, really good
task-specific AI systems
that'll do things from like detecting cancer in x-rays
to self-driving cars that navigate the streets,
or just sharpening photos in Photoshop.
But the whole general AI thing,
which needs a ton of information
to be able to navigate a ton of different situations
is a whole nother challenge.
I mean, if you think like Tesla robot
walking around the earth completing tasks for you,
like that's the level we're talkin' about here.
And so being able to recognize objects and images
and associate them very quickly and accurately
is a big part of that.
Now are there things that DALL-E doesn't do?
Well, yes actually.
There are both some intentional
and some unintentional shortcomings
of DALL-E 2 as it exists right now.
So on the intended side,
the library of images that DALL-E references is massive,
but it doesn't have any images of adult content
or illegal activity or violence,
so it doesn't create images with that stuff in.
Makes sense.
That's probably why we got scissors
in the teddy bear's hand instead of a knife
because that's the next closest association
the AI was able to make for that surgery.
And you also can't ask for imagery
of specific identities of people.
So you can ask for man robbing a bank,
but you can't ask for Marques Brownlee robbing a bank.
As curious as I am about
what type of image that would spit out, you can't.
That would be dangerous for obvious reasons.
But also DALL-E 2 is known to have some quirks.
So one of 'em is it doesn't do very well
specifically with variable binding or,
what basically will happen when you ask
for relative position of objects in an image.
So if I ask for a red cube on top of a blue cube,
it might just give you a blue cube on top of a red cube.
And we actually saw this in one of the images I got back
for a blue apple in a bowl of oranges.
Well, right there, that's clearly an orange
in a bowl of blue apples, which is kinda funny.
And it also, for whatever reason,
doesn't do written text well.
So sometimes it can give you certain letters,
but if you ask it for a sign that says a certain word,
it'll almost never actually give you that.
There's actually a pretty hilarious Twitter thread
of someone asking DALL-E
for signs with certain things over and over again,
just to see what random text it spits out
which is also pretty funny,
but this is the type of stuff they'll be working on
for DALL-E, and for future versions, as you can imagine.
But it's funny, with every shortcoming they found,
there was also an equally awesome
accidental upside they discovered, too.
Like this diffusion method can also transform images.
So you can take an existing image
and run the model over and over
to push it more and more towards any prompt you want.
So you can take this plain jacket for example,
and slowly turn it into a Jackson Pollock painting.
Or take this picture of a cat
and slowly turn it into a samurai master.
Or take a picture of tech, a piece of tech,
and slowly un-modernize it over and over.
Like look at what it does to this iPhone.
It turns it back into an older and older phone.
It's modifying existing images
based on other existing concepts.
It's kinda sick.
So, is this going to be taking people's jobs?
Well, lucky for you,
if you want the exact answer to that question
that's literally the concept we attacked
with the new studio video.
So I'll link it below the like button
if you wanna watch it.
But we pit DALL-E 2 up against Tim,
who is the graphic designer here at the MKBHD Studio,
where their jobs are kind of basically the same thing.
It's to turn the words coming out of my mouth
into a good-looking image.
Spoiler alert, if you give Tim enough time,
he'll make something better.
But in 10 seconds, DALL-E is able to spit out
a bunch of different variations.
And while the images might be a bit fuzzy around the edges
or have weird text or fall apart
when you zoom in on faces or hands or objects,
this tool, as presently constructed,
is amazing for brainstorming ideas and concepts
and things that would normally take much longer to create.
It is truly an amazing side effect
of the development of this AI,
that it's able to make this tool,
where the images that it spits out
aren't necessarily supposed to be finished
final pieces of work,
but they are a great starting point for making stuff later.
And that's actually exactly what we did
with this video's thumbnail,
which started off as an image generated by DALL-E,
where it was told to make a robot hand drawing.
So I have no doubt that there will be versions of DALL-E
in the future that make even higher resolution
and more photorealistic images,
and then even better quick animations, and then video clips,
and then whole movies even,
all on our way to this general AI goal
that we're working towards.
What a time to be alive.
Thanks for watchin'.
Catch you guys in the next one.
Peace.