hi thanks to coming to our session
so i'm Taka from MIT MediaLab, and currently
working at the University Tokyo
here I will present to you our new
interactive system
that we call inDepth which enables
force-based interaction
with objects beyond the physical barrier
in this presentation I talk about
a physical barrier
which used to be an obstacle, but now
with our system,
becomes a new interface.
Physical transparent barriers are common
in everyday scenarios,
like in aquariums or museums, that
separates between
contents and people. While these barriers
have important roles
it prevents us from reaching into the 3d
spatial contents
on the other side. Such barriers are also
common for 3d volumetric displays that
makes it hard to point at the specifics
3d spatial xyz coordinates beyond
the barriers for the input interaction.
We can point these contents in 3d but
can't point
'in-depth' - with 3d depth coordination.
So our research question is:
How can we give people access to reach
and
point at spatial contents beyond
a physical barrier
the key idea is that transforming the
physical
barrier itself into a force-based touch
interface
with inDepth system by directly
touching the barrier surface
people can freely manipulate the cursor
to any 3d positions at a depth
just by applying a force on the barriers
with your fingers
only by putting several force sensors
beneath the physical barrier
users can build their own inDepth
systems
so we utilize the physical barrier and
transform it
to an interface as another instance
in a bakery shop with the inDepth
system, you can imagine,
the customers can select their desired
items just by applying force
on the grass of the showcase customers
can press
lightly on the grass to select items
at the front
but they can also increase the force
intensity
to select items deeper in the back
and here's the design schematics of the
inDepth systems
there's a three component the physical
barrier object inside it
and force sensors the great advantage
in this configuration is the surface
and the items are kept unmodified. what you
need is
just deploying four sensors beneath the
barrier it's just that
these sensors translate force direction
and intensity into a position
in 3d space this system
also translates the input force signals
into
a various predetermined functions
for the hardware implementation we are
using a 6dof force sensor
specifically we are using the SCALE force
sensor toolkit
that was published at the UIST'19 due to
their ease of use
you can see each blck module underneath
the object
detects 3d force signals in real time
but together they form a network of
sensors
that enables interaction on rigid
objects
for more detail please see the work as
well
for the software implementation we
carefully designed the mathematical
algorithms as well
since human perception is not directly
proportional
to the amount of physical stimulus
it is not a good strategy
to directly convert the input force
intensity
onto the depth information therefore
we have developed a new conversion
algorithm based on Waber-Fechner's Law
where the depth is logarithmically
proportional
to the input force currently the
parameters are empirically determined
but they could be determined in more
systematic way
by future user study
for the applications we consider that
the inDepth could cover many different
application domains
to understand this broad application
space coherently
we have categorized various possible
scenarios into three domains
first type is the volume imagery which has
transparent barrier and 3d graphics
as a content second type is create
showcase type
which also have transparent barriers and
physical items inside
as stated as stated museums and
aquariums could be
classified into this category as well
the other type is the geometric content
which content
which contains an occluding surface with
graphics content
to see structures inside a gadget like a
screwdriver
inDepth could provide an intuitive
manipulation
so this is the demonstration in the
educational setting in astronomy class
the viewpoint and the depth of graphics
is controlled by applying force
with your fingers
for the this kind of volumetric display
applying rotational force onto the
surface can be directly translated
as a rotation of the graphical contents
the 3d coordinates of the physical world
and the rendered protein model
are aligned, so that people can
intuitively interact with the rendered
content
also in other medical scenario the
pressure
applied with your palm can control a
planar cursor
which is useful for seeing the cut
intersection of the rendered human skull
by pressing differently you can
intuitively
control the tilt and depths of the
intersections
as well
so there are many research opportunities
based on this
new interaction methods and now we are
planning to extend
our application scenarios as mentioned
before
it could be a really fascinating field
for real-world deployment
in aquariums or museums
also we recently found that inDepth
system has gained new opportunities in
the world
with COVID-19. now we have many
transparent barriers in the life
like at the reception in the shops these
barriers can be transformed
into new interfaces by attaching just
small
sense force sensors
okay this is the end of the prerecorded
session
let's move on to the our live session
thank you for watching
see you soon :)