Hello everyone, my name is Jiannan Li
and I’m glad to share with you our work Route Tapestries:
Navigating 360 virtual tour videos
using slit-scan visualizations.
This is a joint effort between myself
and my collaborators Jiahe Lyu,
Mauricio Sousa,
Ravin Balakrishnan,
Anthony Tang,
and Tovi Grossman.
All of us are from the University of Toronto.
Many of us wonder if we can go to remote places
without physically traveling,
just like I open this door
and I’m in a izakaya in Ebisu, Tokyo.
One increasingly popular way to do this
is through 360 virtual tour videos,
which shows the surrounding view of a 360 camera
while it moves through an environment.
Such videos can also be useful
for virtual campus tours
or families making travel plans.
However, the current desktop interfaces
for navigating such videos are based on
the designs for normal field-of-view videos
and do not necessarily fit 360 content.
For example, if I want to find this coffee shop
with a quite distinct look
in a Tokyo virtual tour video,
it might take me some time using the previews
on the current YouTube interface.
The previews shown here are generated
through equirectangular projection
and it creates some level of visual distortion
because it projects a spherical video frame
to a rectangular video frame on a plane.
To facilitate the navigation
of 360 virtual tour videos,
we propose using Route Tapestries.
Route Tapestries
are strip-shaped video previews
showing continuously captured
scenes along the camera route.
In the Route Tapestry previews,
the target coffee shop is readily visible
without much visual distortion
and a viewer can find it relatively easily.
Let’s use one more example
to detail the experience of using Route Tapestries
for navigating 360 virtual tour videos.
Let’s say we want to find the moment
where I can see the entrance
to the Louvre museum
in a virtual tour video in Paris.
When my cursor hovers on the timeline,
two Route Tapestry previews are displayed above the cursor.
The top and bottom tapestries
shows the scenes
on the left and right side of the camera path.
A short yellow bar
marks the exact location of my cursor on the tapestry.
When I clicked on one of the tapestries,
the video playback jumps to the moment
where the camera passes by the scene I just clicked on.
and the camera automatically reorients itself
towards that scene.
Here, I am able to find the moment
in the video where I see the Louvre entrance.
Finally, we see the example
of finding a food bar in a campus tour video.
Overall, our Route Tapestry approach
is based on the observation
that there are clear, largely continuous boundaries
along the camera path
in a large number of virtual tour videos.
We design a method
for automatically generating route tapestries
using the slit-scan imaging technique.
More specifically,
we estimate the camera paths
and environment boundaries
using simultaneous localization and mapping.
We then join slices of pixels
from each part of the boundaries together
to form route tapestries.
A video player timeline using Route Tapestries
can operate in both the spatial and temporal mode.
In the spatial mode,
the tapestries for the left and right side of the camera path
always move at the same pace.
In the temporal mode,
the two tapestries are always synchronized
in video playback time
at the cursor position,
but they may move at difference paces.
Here we see an example
of a Route Tapestry timeline in the temporal mode.
Note the two tapestries move at different speed
because the scenes on the left and right sides
are at different distances from the camera.
Also the tapestries move very slowly
when the camera comes to a stop.
We have considered a number of
alternative navigation methods
when designing route tapestries,
including equirectangular previews
as used in the YouTube interface,
Little Planet previews
as used in prior research on 360 video editing,
And normal field of view thumbnails
showing the left and right side of the camera path
Our pilot studies have found that
the Little Planet and normal field-of-view thumbnails
do not appear to provide sufficient support
for navigating virtual tour videos
We also considered
two rows of normal FOV thumbnails.
This design is a visually discrete
alternative to Route Tapestries.
Rows of thumbnails
have been used in commercial video editing software
as timeline previews.
We call this technique
‘normal-field-of-view strips’.
To evaluate the performance
of Route Tapestries
for 360 virtual tour video navigation,
we conducted a user study
which compared Route Tapestries with two baselines
that had shown some promises.
These two baselines
are equirectangular previews,
and normal-field-of-view strips.
In the study,
the participants used the three interfaces
to find target buildings given to them
in three virtual tour videos.
Note for clarity,
we are showing the same video
with different interfaces being used.
However, in the actual study,
we used three different
but comparable videos
for the three interfaces.
We measured the participants’ task completion time
and failure rate.
The results show that
when using Route Tapestries
the participants missed
75 percent fewer targets
than normal field-of-view strip,
and they were able to complete the task
22% faster than equirectangular previews.
Our results highlight the benefits
of reducing visual distortion,
and maintaining visual continuity
for navigating 360 virtual tour videos.
Our future work
plans to explore Route Tapestries
for navigating a wider range of 360 videos.
For example,
we want to explore tapestries
consisting of multiple layers,
where objects at different distances
from the camera
can move at different paces,
like the parallax scrolling effect
in video games.
We are also interested in
adapting route tapestries
for head mounted display uses,
for example, placing Route Tapestries
in a circle surrounding the user.
To sum up,
Route Tapestries are continuous content previews
for 360 virtual tour videos,
they can improve navigation performances
as they have less visual distortion
and maintain visual continuity.
Route Tapestries can be automatically generated
from 360 virtual tour videos,
and our paper presents more technical details.
Finally, I would like to thank you
for your interest in this talk,
and thank my collaborators and funding agency.
If you are interested in more details
or more discussions,
feel free to ask me in the upcoming Q&A,
or drop us an email at the address shown in the slide.
Thank you very much!