All right. Good afternoon, y’all.
Let's talk about blending reality and imagination.
But first, let's take a step back in time to 2001.
As an 11-year-old in India,
I became obsessed with computer graphics and visual effects.
Of course, at that age,
it meant making cheesy videos kind of like this.
But therein started a foundational theme in my life,
the quest to blend reality and imagination.
And that quest has stayed with me
and permeated across my decade-long career in tech,
working as a product manager at companies like Google
and as a content creator on platforms like YouTube and TikTok.
So today, let's deconstruct this quest to blend reality and imagination
and explore how it’s getting supercharged -- buzzword alert --
by artificial intelligence.
Let's start with the reality bit.
You probably heard about photogrammetry.
It's the art and science of measuring stuff in the real world
using photos and other sensors.
What required massive data centers and teams of experts in the 2000s
became increasingly democratized by the 2010s.
Then, of course, machine learning came along
and took things to a whole new level
with techniques like neural radiance fields, or NeRFs.
What you're seeing here is an AI model
creating a ground-up volumetric 3D representation
using 2D images alone.
But unlike older techniques for reality capture,
NeRFs do a really good job of encapsulating the sheer complexity
and nuance of reality.
The vibe, if you will.
Twelve months later, you can do all of this stuff
using the iPhone in your pocket,
using apps like Luma.
It's like 3D screenshots for the real world.
Capture anything once and reframe it infinitely in postproduction,
so you can start building that collection of spaces, places and objects
that you truly care about
and conjure them up in your future creations.
So that's the reality bit.
As NeRFs were popping off last year,
the AI summer was also in full effect,
with Midjourney, DALL-E 2, Stable Diffusion
all hitting the market around the same time.
But what I fell in love with was inpainting.
This technique allows you to take existing imagery
and augment it with whatever you like,
and the results are photorealistically fantastic.
It blew my mind
because stuff that would have taken me like three hours in classical workflows
I could pull off in just three minutes.
But I wanted more.
Enter ControlNet,
a game-changing technique by Stanford researchers
that allows you to use various input conditions
to guide and control the AI image generation process.
So in my case, I could take the depth information
and the texture detail from my 3D scans
and use it to literally reskin reality.
Now, this isn't just cool video.
There’s a lot of useful use cases, too.
For example, in this case I'm taking a 3D scan of my parents' drawing room,
as my mother likes to call it,
and reskinning it to different styles of Indian decor
and doing so while respecting the spatial context and the layout
of the interior space.
If you squint, I'm sure you can see
how this is going to transform architecture
and interior design forever.
You could take that 2016 scan of a Buddha statue
and reskin it to be gloriously golden
while pulling off these impossible camera moves
you just couldn't do any other way.
Or you could take that vacation footage from your trip to Tokyo
and bring these cherry blossoms to life in a whole new way.
And let me tell you, cherry blossoms look really good during the day,
but they look even better at night.
Oh, my God. They sure are glowing.
It's almost like this dreamlike quality
where you can use AI to accentuate the best aspects of the real world.
Natural landscapes look just as beautiful.
Like this waterfall that could be on another planet.
But of course, you could go over the hills and far away
to the French Alps from another dimension.
But it's not just static scenes.
You can do this stuff with video, too.
I can't wait till this technology is running at 30 frames per second
because it's going to transform augmented reality and 3D rendering.
I mean, how soon until we're channel-surfing realities
layered on top of the real world?
Of course, just like reality capture got democratized,
all these tools from last year are getting even easier.
So instead of me spending hours
weaving together a bunch of different tools,
tools like Runway and Kaiber let you do exactly the same stuff
with just a couple clicks.
Want to go from day to night? No problemo.
Want to get that retro 90s aesthetic from "Full House"?
You can do that too.
But it goes beyond reality capture.
Companies like Wonder Dynamics
are turning video into this immaculate form of performance capture
so you can embody fantastical creatures using the phone in your pocket.
This is stuff that James Cameron only dreamt about in the 2000s.
And now you could do it with your iPhone? That’s absolutely wild to me.
So when I look back at the past two decades
and this ill-tailored tapestry of tools that I've had to learn,
I feel a sense of optimism for what lies ahead
for the next generation of creators.
The 11-year-olds of today
don't have to worry about all of that crap.
All they need to do is have a creative vision
and a knack for working in concert with these AI models,
these AI models that are truly a distillation
of human knowledge and creativity.
And that's a future I'm excited about,
a future where you can blend reality and imagination
with your trusty AI copilot.
Thank you very much.
(Applause)