Hi, my name is Florian Müller and I’d like to welcome you to my talk at this very special
CHI 2020.
Despite the difficult circumstances, I am glad you are here.
And for the next 15 minutes or so, I'd like to give you a brief summary of this work that
I have conducted with my co-authors Martin Schmitz, Daniel Schmitt, Sebastian Günther,
Markus Funk, and Max Mühlhäuser.
In this talk, we’re going to have a look at how we can interact with information on the go.
And for that, let’s start by looking at how we do that today.
While walking through the city, we see again and again people who stare at their smartphones
like zombies.
That makes us realize that there seems to be a strong need for constant access to and
interaction with information, anywhere and anytime.
And not only are people interacting with and staring at their phones while they are standing
and waiting somewhere, but also while they walk around.
However, just as distracted driving leads to dangerous situations for the driver and
others, distracted walking also causes people to endanger themselves and others.
As a possible solution, Head-mounted displays might help us to overcome these problems as
they allow us to keep the visual connection to the real world while interacting with digital
information.
However,
today's interaction techniques for Head-mounted Displays are not tailored to the specific requirements
of interaction while walking.
For example, Gesture-based interaction techniques as used, for example, with the Microsoft Hololens as you can see here
are quite nice because there is no secondary device that could be misplaced or lost.
However, these interaction techniques require the user‘s hands to be free for the interaction while especially
during mobile situations, the users' hands might be busy carrying things or interacting
with the real world in other ways.
Further, this style of interaction is not very expressive and thus limits the amount
of information that can be communicated from the user to the device in one interaction step.
As another possible approach, voice-based interfaces allow for easy and fast interactions
while walking.
However, voice-based interfaces have privacy implications in public settings and do not
work well in noisy environments such as crowded places or busy streets.
Looking at the related works, we see that research proposed novel interaction techniques
for interacting with Head-mounted displays in multiple areas:
First, there are mid-air gestures where one or both hands of the user provide input to
the system through gestures in front of the body.
Here, just as an example, some work by Colaço et al. who presented a sensing device to capture
more fine-grained gestures.
Second, there is on-body interaction where parts of the user's body, often the palm or
forearm, act as an interactive touch-enabled surface providing input and output on the surface of our body as you see here
in an example by Chris Harrison et al.
Third, there are head- or gaze-based interaction techniques, where the eye-gaze or head-pointing is used
to interact with information, as you can see here in this example by Kytö et al.
However, despite their usefulness, these interaction techniques are not tailored to the specific requirements
of interaction while walking.
Mid-Air gestures as well as on-body interfaces require the user's hands to be free for the
interaction and therefore cannot support a wide range of situations, where the users'
hands are busy.
Head- or gaze-based interaction techniques require the user to shift their gaze away
from their walking path, therby leading to dangerous situations.
Therefore, in this work, we explicitly focused on how we can interact with information using Head-mounted displays
while walking.
For this, we consider a system that displays multiple lanes parallel to the walking path
of the user.
Each lane represents an option the user can select.
If the user continues straight ahead, no option is selected and, therefore, no reaction by the system
is triggered.
If the user, however, wants to interact with the system, they can shift their walking path
sideways until they walk on the desired option lane.
And this change can be done without a need to change the walking speed, just as you would
change your walking path when avoiding an obstacle on a sidewalk.
The system highlights the lane the user is currently walking on and this highlighting
is applied to the entire lane.
This allows the user to interact with the system without looking down to the floor, as the current state of
the system is also visible while looking straight ahead in the peripheral view.
By walking on a lane for a certain time, the user can select one of the option lanes.
This works just as the concept of selection by dwell time in eye-gaze based interaction.
In this paper, we analyzed this concept in a controlled experiment and, based on the results, provide use cases
and example applications.
Further, we contribute a prototype implementation for the Microsoft Hololens that implements
this style of interaction. Let's have a look at the controlled experiment.
In the experiment, we investigated the influence of two factors on the accuracy and efficiency
of such interfaces.
First, we investigated the influence of the width of the lanes.
For this, we varied the number of lanes on a 1m wide interaction area where the lanes
always filled up the complete available space.
The experiment varied the number of lanes on this 1-meter width as 8, 12, and 16 (each
plus one non-active lane in the middle), resulting in lanes that were between 6 and 11cm in size.
Beyond the width of the lanes, the experiment also varied the time needed to walk on a lane
to select it as 1/3, 2/3, and 3/3 of a second.
At an average walking speed of around 1.5m per second, this results in half a meter to
1.5 meters of walking distance required to select a lane.
For the experiment, we varied these two independent variables - the number of lanes and the selection time - in a within-subjects design with
18 participants.
We measured the task-completion time as well as the accuracy of participants and used questionnaires
for further data collection.
Finally, we analyzed the data mainly using two-way repeated-measures ANOVAs.
As we did not want to rely on the at times unstable and inaccurate tracking of current-generation
Head-mounted displays, we opted against conducting the experiment with real devices.
Instead, we used two projectors for visual output and tracked the participants using
an HTV VIVE Tracking system.
In the experiment, the system projected the lanes to the floor in front of the user as you can see in this video.
The participants’ task was to shift their path to the highlighted target lane and after reaching the target lane stay
within the bounds of the target lane for the respective selection time of the condition.
Let’s have a short look at the results.
As I don’t want to throw too many numbers at you in this talk, I’m going to focus only on the accuracy.
If you are interested, there are many more results and many more numbers in the paper.
Okay, so a 2-way RM ANOVA showed significant influences of the number of lanes on the accuracy
of participants.
Post-hoc tests confirmed significantly higher accuracy rates for the 8 and 12 lane conditions
compared with the 16 lane conditions.
Further, the analysis also revealed a significant influence of the selection time on the accuracy
of participants.
Here, post-hoc tests showed significantly lower accuracy rates for the 1/3 of a second conditions
compared to both, the 2/3 and 3/3 of a second conditions.
While the results for the number of lanes are not really surprising (that smaller lanes
result in reduced accuracy rates as it was harder for participants to stay within the
bounds of the lane), the results for the selection time - I guess - are more surprising.
It is the 2/3 of a second conditions that showed the highest accuracy rates.
Too short selection times resulted in participants accidentally selecting intermediate lanes
while crossing them on the way to the target.
On the other hand, too long selection times made it harder for participants to stay within
the bounds of the lane, thereby increasing the chance of accidentally leaving the lane within the
selection time.
This effect becomes even more pronounced the thinner the lanes become as you can see with the dropping
of the accuracy rates at the 16 lanes condition with 3/3 of a second.
We attribute this effect to the fact that our head moves a little bit to the left and a little bit to
the right with every step we make, even further increasing the chance of accidentally leaving the lane within the selection time with every step you force the participant to make.
Based on the concept and based on the results of our controlled experiment, we built three example applications:
First, Assistant allows users to explore nearby services such as coffee shops or hotels or restaurants right
while walking through the city.
Second, Camera allows users to take pictures, apply filters, and share the results to social
media platforms.
Third, Music Walker allows users to walk through their playlist and select songs by leaving the
lane.
Further, users can continuously change the volume by walking on the respective lane for a certain amount of time.
To sum up, Walk the line provides a truly mobile input modality for interaction on the
go while walking.
The results of the controlled experiment confirmed that Walk The Line provides accurate and fast
interactions.
And further, the qualitative feedback from our participants showed that the concept provides
a joyful way for casual interactions.
And with this, I‘d like to close this talk.
Due to the current situation, I, unfortunately, cannot answer your questions right here or
tonight at the bar.
Instead, I’d like to ask you to drop me an e-mail.
And one last thing! there are more great #teamdarmstadt papers.
Go and check them out right now!
And with that, I'd like to say goodbye and thank you for your attention.
Stay safe and healthy and see you next year in Yokohama. Bye!