Immersive analytics is a young, quickly evolving research field
that unites work in visualization, immersive
technologies and HCI. To help guide this new area, we present
17 key research challenges identified by a diverse group of
international experts and based on a virtual workshop that we
conducted at ACM CHI 2020.
In 2015, immersive analytics has been defined as: to explore the
applicability, and development of emerging user interface
technologies for creating more engaging and immersive
experiences and seamless workflows for data analysis
applications. Since then, the field has produced advances in a
range of areas. For example, empirical studies investigate
perception and interaction compared to more traditional
displays. Interaction techniques are being investigated for
tangible and haptic interfaces. Specific application areas are
explored, many of which require visualizations to be situated
within a real world environment. Design and authoring tools now
include tools for non programmers, and new commercial
platforms bring immersive analytics to end users.
As CHI 2020 couldn't happen face to face, we decided to go ahead
with virtual workshop. We had 28 participants from the HCI and
visualization communities. We met four times on Zoom over the
course of two weeks and across four different time zones. After
a series of inspiring lightning talks, and a synchronous group
work sessions, we split into work groups and we have
identified four topics and 17 challenges. The four topics
were: spatially situated data visualization, interacting with
immersive analytic systems, collaborative analysis, and user
scenarios and evaluation.
The first group of challenges relates to spatially situating
data visualizations in our real world surroundings. These
include finding ways to accurately link existing data
sources with relevant spatial locations, and making use of
semantic knowledge extracted from the environment. While
there is a growing body of evidence that spatial
interaction can improve sensemaking, more work is needed
to understand the benefits of spatially coupling data
encodings with a physical reference. There's also a need
to better understand the cognitive processes of analytics
using augmented reality for data analysis tasks. And finally, the
requirements of Immersive applications for detailed user
tracking and physiological data raises a number of ethical
concerns that must be addressed.
Interacting with immersive analytic systems is uniquely
challenging in part because of the combination of novel
multimodal input and output technologies. Consequently, the
challenges are: How can designers better map interaction
modality the most appropriate human sense, such as
proprioception? How to enable multi sensory output such as
vibrations as feedback for input? How to support multiple
kinds of transitions, for instance, from using desktops,
where users have expertise to immersive analytics
environments, which may be novel and familiar. And finally, how
to support complex analysis tasks, using rich multimodal
combinations.
While the possibility to facilitate collaborations is one
of the key benefits of immersive technology, there are still
challenges to solve in order to design efficient systems. Such
systems will of course need to support different forms of
collaboration: from close to loose, from formal presentation
to informal discussion. They should also support as much as
possible face to face communication cues, like facial
expressions, and hand gestures, but also ones that go beyond
what we can do in face to face like changing our own size, or
use laser pointers instead of fingers. Such collaboration
could be done from a wide range of immersive devices like AR/VR
headset, wall displays, or devices that have not been
invented yet. It is then necessary to explore how we can
translate communication cues done on one device for any other
one. But also to explore how we can integrate these new devices
during collaborative practices and how we can assess them.
One thing that emerged in our workshops was on suitable
scenarios for immersive analytics and developing
methodologies for evaluating them. We defined three grand
challenges as illustrated in the figure to the right. In
"Defining application scenarios for immersive analytics", we
addressed the challenge of identifying why certain
scenarios lend themselves to immersive analytics, while
others may not. The challenge "Understanding users and
contexts for evaluation of immersive analytics" addresses
questions surrounding who the end users of immersive analytic
systems are, and where they perform their work. Finally, we
consider instrumental to establish an evaluation
framework for immersive analytics for defining what
attributes we need to consider when evaluating system
performance and how these can be measured.
We hope these challenges will inspire the growing immersive
analytics community, encouraging new discussions and motivating
researchers who are new to the field. Immersive analytics
offers tremendous potential to extend and enhance our
experiences with data, and addressing these grand
challenges will enable us to realize new opportunities in
this innovative, interdisciplinary domain.