Throughout the history of computers
we've been striving to shorten the gap between us
and digital information,
the gap between our physical world
and the world in the screen
where our imagination can go wild.
And this gap has become shorter,
shorter, and even shorter,
and now this gap is shortened down
to less than a millimeter,
the thickness of a touch-screen glass,
and the power of computing
has become accessible to everyone.
But I wondered, what if there could be no boundary at all?
I started to imagine what this would look like.
First, I created this tool
which penetrates into the digital space,
so when you press it hard on the screen,
it transfers its physical body into pixels.
Designers can materialize their ideas
directly in 3D,
and surgeons can practice on virtual organs
underneath the screen.
So with this tool, this boundary has been broken.
But our two hands still remain outside the screen.
How can you reach inside and interact
with the digital information
using the full dexterity of our hands?
At Microsoft Applied Sciences,
along with my mentor Cati Boulanger,
I redesigned the computer
and turned a little space above the keyboard
into a digital workspace.
By combining a transparent display and depth cameras
for sensing your fingers and face,
now you can lift up your hands from the keyboard
and reach inside this 3D space
and grab pixels with your bare hands.
(Applause)
Because windows and files have a position in the real space,
selecting them is as easy as grabbing a book off your shelf.
Then you can flip through this book
while highlighting the lines, words
on the virtual touch pad below each floating window.
Architects can stretch or rotate the models
with their two hands directly.
So in these examples,
we are reaching into the digital world.
But how about reversing its role
and having the digital information reach us instead?
I'm sure many of us have had the experience
of buying and returning items online.
But now you don't have to worry about it.
What I got here is an online augmented fitting room.
This is a view that you get from
head-mounted or see-through display
when the system understands the geometry of your body.
Taking this idea further, I started to think,
instead of just seeing these pixels in our space,
how can we make it physical
so that we can touch and feel it?
What would such a future look like?
At MIT Media Lab, along with my advisor Hiroshi Ishii
and my collaborator Rehmi Post,
we created this one physical pixel.
Well, in this case, this spherical magnet
acts like a 3D pixel in our space,
which means that both computers and people
can move this object to anywhere
within this little 3D space.
What we did was essentially canceling gravity
and controlling the movement by combining
magnetic levitation and mechanical actuation
and sensing technologies.
And by digitally programming the object,
we are liberating the object from constraints
of time and space, which means that now,
human motions can be recorded and played back
and left permanently in the physical world.
So choreography can be taught physically over distance
and Michael Jordan's famous shooting can be replicated
over and over as a physical reality.
Students can use this as a tool
to learn about the complex concepts
such as planetary motion, physics,
and unlike computer screens or textbooks,
this is a real, tangible experience
that you can touch and feel, and it's very powerful.
And what's more exciting
than just turning what's currently in the computer physical
is to start imagining how programming the world
will alter even our daily physical activities.
(Laughter)
As you can see, the digital information
will not just show us something
but it will start directly acting upon us
as a part of our physical surroundings
without disconnecting ourselves from our world.
Today, we started by talking about the boundary,
but if we remove this boundary,
the only boundary left is our imagination.
Thank you.
(Applause)