Hi, everyone.
Thank you, Gemma, for introduce my presentation.
So my name is Motohiro Makiguchi from NTT Corporation in Japan.
So today my talk is Interactive 360-Degree Glasses-Free
Tabletop 3D Display.
Sorry.
So there are the contribution of our study.
So we propose a new interactive 3D display
system which has autostereoscopic and 360 degree
motion parallax for multiple users,
and the system has vertical motion parallax
for various height support, and a real-time interaction
with 3D data.
We implemented the system with 60 projectors that
is 1/5 of the conventional method,
and the display area is about six times larger.
So many of the researchers here may have experience
of tabletop collaborative work.
So discussion while surrounding a physical object
make it easy to share the subject
and to give new creative ideas.
The purpose of our study is to apply such tabletop
collaborative work through 3D digital data.
Using 3D digital data makes collaborative work
more speedy and more flexible.
So for that purpose, there are some requirements.
So one is autostereoscopic and 360 degree motion parallax
for multiple users.
So the key point is with no wearable devices
like 3D glasses or head-mounted displays.
So we think that point is important,
because keep eye contact or face to face communication
in the collaborative work is very important.
So second, we need vertical motion parallax
for various height.
For presenting correct 3D shape for the multiple users
at the same time, it is necessary to deal
with height differences.
And finality, the real-time interaction with 3D data.
We propose a system that meets these requirements
with the six--
1/5 number of projector and the six time
larger screens as conventional methods.
So this is our system overview.
The system consists of an optical unit,
tracking unit, and rendering unit.
I introduce for each unit.
So firstly is the optical unit.
So optical unit of our system has a projector array
and a spatially imaged iris plane.
So there are so many 3D screening system
proposed with autostereoscopic and motion parallax.
The basic principle is making multiple viewpoint images
for all around the display.
The conventional method roughly divided two type of the method.
So one is the time division method,
and the other is the space division method.
So one example, the time division method,
is using high speed video projector and high speed
rotating screen.
Because time division method needs high frame rate display,
so it is difficult to increase the frame rate of the 3D
images.
And the rotational mechanism is difficult to enlarge,
and users cannot touch the 3D object because there are unsafe
to use.
So is the spatial division method
is free from this problem of the limitation of the frame rate
and user safety.
So it's very unsafe--
safer, and can make more high frame rate systems.
So you see the proposed-- the system
using multiple projector and a narrow diffusion screen.
So this system uses 288 projectors and 200 millimeter
diameter system screens.
So then, increase of number of projectors
is difficult to implement, and--
to applications.
So we solved the problem of increasing projectors.
So we focused a digital perceptual mechanism
called linear blending.
So this is a comparison of the approach--
the conventional method and our approach.
The left figure shows the conventional method.
So its method is dense projector array and luminance
distribution of the screen has very sharp and separated.
On the other hand, our approach, called linear blending,
needs-- use a sparse projector array and a smooth luminance
distribution.
So the luminance distribution is needed to--
the adjacent viewpoint image are blended optically.
So the key point is, adjustment viewpoint images
are branded optically.
So by blending adjustment viewpoint images,
the intermediate viewpoint images
are perceptually interpolated according
to the optical blend luminance ratio.
This mechanism called the linear blending
is shown in the left figure.
So there are two edges whose disparity is less than fusion
limited angle-- so it's say, 6 to 8 arcmin--
are perceived as a single edge, and the position
of a single edge is determined by the blending luminance
ratio.
We deduced the number of projectors
for intermediate viewpoint point projector
by implementing this principle of linear branding.
To implement linear branding principle for actual 3D system,
we use a new screen that's called the spatially
imaged iris plane screen.
So this is consisting of a reflective layer,
Fresnel layer, and a diffusion layer.
The screen images the iris of the projector in the space.
We call this spatially imaged iris plane.
So the user can watch the projected images on the screen
when they enter in the spatially imaged iris plane,
but users cannot see the projected images outside
of the spatially imaged iris plane.
The adjacent spatially imaged iris planes are gradually
blended by diffusion layer, so this characteristic
is suitable for linear blending.
So in this screen, the main optics uses Fresnel lens.
So the Fresnel lens structure contributed
to large and thinner screens.
So for making a rotationally symmetric viewpoint
distribution, the lens axis of the Fresnel lens
has to be center of the screen, so the projectors are pointed
away from the optical axis.
In this setting, the spatially imaged iris plane
is deformed by lens distortion and the focal lens design
became very difficult.
We used ray tracing simulation and designed the Fresnel lens
so that the shape of the spatially imaged iris plane
is narrow in the horizontal direction
and long in the vertical direction.
We developed the screen with a diameter of 1,200 millimeters,
and evaluated the luminance distribution of the spatially
imaged iris plane.
This is the setting of evolution of luminance distribution.
We used a 2D luminance meter.
So this is the result of the evolution
of the luminance distribution.
The left figure shows the result without diffusion layer,
and the right figure shows with diffusion layer.
The shape of the spatially imaged iris plane
is narrow in the horizontal direction
and the long in the vertical direction, so we
provide the more wider vertical viewing areas.
And the diffusion layer expands the luminance distribution
to the horizontal direction about 6 degrees.
So we set the five projectors with 6 degree intervals
and evaluated the luminance overlap.
This figure shows the result. In this figure,
adjacent viewpoint images are blended optically,
and optical characteristic of the implemented screen
are suitable for linear blending.
This is our demonstration of a horizontal motion parallax
prototype.
The prototype uses 60 projectors with 6 degree intervals,
and the screen diameter is 1,200 millimeter.
As shown in this demonstration, our system
can produce 360 degree motion parallax
with 1/5 number of the projectors,
and the diameter of the screen is six times larger
than conventional method.
Then I introduce the tracking unit
for vertical motion parallax.
The tracking unit consisted a 360 degree camera and tracking
PC.
So the motion parallax has two direction.
In our system, the horizontal motion parallax
is reproduced by multiple projector array
and using linear branding, and the tracking unit
is used for reproducing the particle motion parallax.
So there are two approach to reproduce the vertical motion
parallax.
One is physical switching the height
of the projector with, for example, 2D projector array
or moving projector.
The another is virtually switching the height
of the viewpoint image.
Physically switching needs a huge number of projectors,
or moving projecting mechanism make system complicated,
so we focused the second approach, so the battery
virtual switching approach.
We used a single 360 degree camera and a OpenPose
to detect the user's head position around the table.
So the process is very simple.
First, the OpenPose detects the user's head position,
and second, the viewpoint image changes by the user's head
position, elevation angle.
This is a demonstration of head tracking.
Our system reproduces vertical motion parallax
with a single 360 degree camera and OpenPose.
As in this demo movies, so you can
see the changing of the height of virtual cameras.
Now we show the real-time interaction unit.
So the real-time interaction with a 3D object
is reproduced by rendering unit.
So rendering unit consist of an operation tablet, master PC,
and 60 client PCs.
Each client PC is connected one projector by HDMI,
so 60 viewpoint images rendered by client PC
are required to be timed synchronized.
Therefore, the master PC sends operational information
from the tablet PC, and synchronize the information
to the client PC for time synchronize, once per frame.
Output image of the client PC are
adjusted so that images from 60 projectors
are perfectly matched on the screen surface.
As the initial setting, we adjust
the keystone of all client PCs through the master PC.
The calibration step has three steps.
The first is hardware calibration.
So we make shift blends optical unit to adjust--
to attach the projectors.
So the shift lens and shift the projector's axis
to screen's frame.
And second, we make software combination automatically using
QR code and image recognization technique.
And finally, we said that we calibrated the software
calibration manually.
Client PCs output the viewpoint rendered
after correcting the keystone on the basis of the calibration
result. Content is created in Unity 3D
and installed on all client PCs in advance
as an initial setting.
So we used Unity 3D so it's very easy
to make new contents or 3D models.
This is a demonstration of a real-time interaction
with 3D objects.
We implemented a function to change
the materials of the 3D object from our tablet application.
We can change the material parameters,
including the color, gloss, and metallic, in real-time.
So this is-- there are limitations of our prototype.
So first, the screen quality is not enough,
because the hot spots and stray light have not
been sufficiently reduced.
So improving the screen quality is that of future work.
And increased number of users is a secondary imitation.
So user's shadow became a problem as the users increased.
So when users face across our tabletop display,
the shadow interferes the observation of each other's.
The third is tracking accuracy.
So user detection fails when the user
bends the body too far forward over the table
or has drawn out highlights.
So we want to improve these limitations as our future work.
So this is a summary of my presentation.
We proposed a new interactive 360-degree glasses-free
tabletop 3D display with autostereoscopic and motion
parallax, using linear blending and spatially imaged
iris plane screen.
And we make vertical motion parallax
using head tracking system with a 360-degree camera.
And finally, we make real-time interaction with 3D datas.
So thank you very much for your attention.
[AUDIENCE CLAPPING]
Any questions?
We have time for one or two quick ones.
So what limits the fusion angle that you could achieve?
So how many-- how fewer cameras could
you potentially-- projectors could you potentially use?
So your question is how we set the fusion limit angle?
Yes, so the fusion limit angle is
determined by previous studies, so it said 6 minutes
to 8 minutes angles.
So by these limitations, height of contents is limited--
imitated in our prototype.
Another question?