Hello everyone.
I'm Jiazhou Liu from Monash University.
Today, I'll be presenting
our "DataDancing" project in collaboration with Barrett Ens, Arnaud
Prouzeau, Jim Smiley, Isobel Nixon, Sarah Goodwin, and Tim Dwyer.
In this project, we explore the design space for whole-body
interactions and propose hybrid prototypes for designing visualisation
view management for 3D surfaces and spaces.
Recent studies have explored how users of immersive visualisation
systems arrange visualisation views in the space around them.Generally,
these systems focused on visualisation views
centred at eye level in absolute room
coordinates and interactions via hand-held controllers.
However, immersive technologies offer the opportunity to work with visualisation
views with natural embodied interactions using different body parts.
For example, foot interaction in general
HCI has been explored as an additional input channel
for complex tasks or when both hands are occupied.
The whole body can also be used as an additional modality
to support implicit tasks using proxemic interaction.
These novel interaction
design possibilities represent a significant shift from everything
we have learnt about interaction with flat screens.
In this paper, we first introduce a design space
for visualisation view management, presenting a framework that identifies
important aspects in whole-body interactions for designing view
management systems.
During this process, we investigate existing designs
from an extensive review of the literature.
We distilled a set of design dimensions using a bottom-up, open-coding approach
by categorising existing designs among these dimensions
to identify gaps and common usages.
This process results in eleven design dimensions
in two categories: visualisation view presentation and user interaction.
In the visualisation view presentation, the perspective refers
to the cognitive judgement of view locations from either
an egocentric perspective or an exocentric perspective.
Proximity refers to the distance
relationship between the user and visualisation views.
The view movability refers to whether each visualisation view has both position
and orientation fixed, free orientation pivoting, or free-movable.
These three dimensions are grouped as space perception.
The second group is the frame of reference,
including the dimension of the coordinate system, the curvature,
orientation, and the movability of the reference frame.
For user interaction, there are two groups as well.
In the interaction style group,
the degree of intent refers to the intent of the interaction,
either explicitly initiated by the user or implicitly mediated by the system.
Proxemics refers to the study of space and how we use it.
In the action group,
the input modality covers a wide range of interaction modalities.
And the target design dimension describes the target of the action.
This table illustrates how the existing view management systems
align with our design space.
For example, in the ImAxes and FIESTA systems, visualisation views
are exocentric to the virtual world or objects
in the world, such as walls.
The views are near the users allowing direct interaction with them.
All the views can be freely moved and oriented.
The main reference frame of the views is the fixed
3D virtual space, normally flat and vertical.
All interactions are explicit, and no proxemics are applied.
Users perform the action by hand and target the views directly.
Another example is the wearable conferencing space.
The views are displayed egocentrically and fixed to the user.
Users can only change the orientation of the views
which are close to them at eye level.
The reference frame is a one-dimensional curved and vertical line.
The reference frame can be moved with the users.
Users can perform explicit interactions on the reference frame
by changing the orientation of their whole body.
We then extrapolate
several view management prototype surfaces from these existing designs, each
demonstrating a different combination of interaction techniques and space use.
These surfaces include wall displays, floor displays, tabletop
displays, body-fixed large displays, and body-fixed cockpit displays.
We then conduct a user study that explores and evaluates
the usability of four combinations of these surfaces.
In order to simulate a real working environment
for visualisation view management tasks and to follow the design guidelines
for multiple views, we consider two fixed
semantic levels for the information contained within the views.
For example, in the top-left figure, a tabletop
display is used as a landmark for users to interact with directly,
while a wall display is used as a coordinated detailed view,
which contains more information for analysis and sensemaking.
We designed four such combinations and conducted a within-subject study
to evaluate the presentation and interactions with each display surface.
We recruited 12 participants and collected quantitative data, qualitative
feedback and rankings for the four prototypes
after completing a series of designed view management tasks,
such as finding the extreme values or trends over time from multiple views
by navigating, rearranging, highlighting, or sorting them.
The study uses a novel body-tracking system and foot
pressure sensors to track participants’ torso and feet positions.
From our study, we observe novel behaviours and collect
feedback from participants with a set of interface
design implications unique to visualisation view management.
Firstly, participants suggested having world-fixed exocentric display surfaces,
such as a wall display for collaboration use, while body-fixed
egocentric display surfaces, such as a cockpit display for private use.
Secondly, most participants preferred the curved surfaces, reporting that views
on the curved surfaces are close to them,
which offers easy and quick selection.
This finding aligns with our previous research
that semi-circle layouts perform better on visual analytics tasks
than other layouts and are more preferred by participants.
Lastly, participants have mixed feedback on the evaluated
novel interactions, such as the foot interactions.
On the one hand, participants were not familiar with and unconfident
with the foot interactions and reported neck
strain while looking down at the ground.
On the other hand, most participants felt
the foot interactions were intuitive and easier than expected.
Our qualitative results align with the related work
that users can perform high-precision interaction tasks with their hands.
In contrast, foot interactions can support secondary
navigation tasks such as panning and zooming.
Learning from the user study, we improve the interaction techniques
and propose two hybrid prototype interfaces that demonstrate
the interaction possibilities of our design space.
In the first prototype, Joe wants to find the day in December
when his favourite street has the most pedestrian volume.
Joe is standing in front of visualisations showing the pedestrian volume
in December.
The visualisations are in a small-multiples layout on a wall display.
This wall display can also be used in a collaborative environment.
He uses bimanual hand interaction to change the aspect ratio
or pan the whole wall display, so that he can see everything at eye level.
After scanning through the visualisations, he clicks on the bar
which represents the data for his favourite street on the chart.
Then, he drags visualisations from the world-fixed wall
display to a body-fixed cockpit display.
The cockpit display is a personal workspace that others cannot see.
Then, he uses the foot interaction to pan the cockpit display
and to change the geometry,
especially when his hands are occupied with other charts.
After comparing all selected charts in his private workspace, Joe finds the answer.
The second prototype uses proxemic interaction to interact
with views and switches between different surfaces.
Here, we have Patrick, who would like to walk close
to a wall display and see the visualisation clearly.
When Patrick moves closer to the wall, the wall display transforms
into a curved tabletop display, which allows him
to have the same distance to all visualisation views.
Patrick keeps moving,
and the geometry transforms from a semicircle to a full circle.
Then, the tabletop display becomes a body-fixed
cockpit display, which follows Patrick.
All the interactions in this prototype are implicitly using proxemics.
In summary, our DataDancing is the first to explore
the effects of foot interactions on floor
displays for visualisation view management in 3D space.
Future work involving the full 3D space may consider designing
effective foot gestures and using them as an additional
input channel for assisting other modalities.