Our paper looks at mid-air interaction in casual postures.
We study a spherical input space positioned at the elbow level.
This input space is independent of body orientation and allows
users to reach all corners of the screen without lifting their elbow.
Usually, systems use a method that is not flexible enough.
By this method, the hand position is tracked on a coordinate system aligned with the display.
So your hand needs to be facing the display. And it's just not very comfortable if you're
not sitting straight in front of the display.
We first ran an observational study where we asked
participants to sit on a couch as they sit at home,
and gave them commands to do mid-air gestures. For instance: swipe right, left.
We found that even when they are in similar postures,
participants may do very different motions.
We also found that when participants changed their posture their motions didn't change
much.
So it seems that the mechanics of gestures are mostly due to personal preference, posture
did not play a big role.
So we propose an input space that causes minimum interference to the rested posture.
The input space is a spherical surface, and it has two main properties:
1) It is defined relative to the elbow joint, so you can be seated in any orientation.
2) The dimensions and geometry match the reach of the forearm when the elbow is rested.
The main difference between the spherical and planar
input spaces is the planar space defines a uniform mapping from
the world coordinates to the screen coordinates.
This mapping is not uniform in the spherical input space.
So if you are controlling a cursor, for instance, the cursor will move faster in the top of
the screen.
To compare these spaces, we designed a controlled study.
We had planar input spaces at elbow and shoulder levels,
and the spherical input space.
For each condition, our participants selected targets
for 5 minutes. We displayed the targets on a large screen.
We measured selection time and we also measured fatigue levels every 30s with the BORG scale.
Participants sat on this couch with their arm rested,
and we tracked their arms with a motion capture system.
We found no significant differences in performance between the input spaces.
The shoulder-level planar input space caused much higher fatigue, but at elbow-level, we
found no differences between spherical and planar.
But, if we plot performance on the screen space, we see a
pronounced weak spot in the spherical space, which is due to
non-uniform mapping I mentioned.
We designed another controlled study to test the performance of the spherical space. This
time, we had participants in different postures on a couch.
Each posture imposes a different constraint, so we wanted to know how interaction is affected
by it.
This is a trial of study. We varied target size, target position, and posture.
This time we implemented interaction using a smartwatch. Any arm wearable with orientation
sensors can be used.
Our results show that the performance is fairly stable across postures, but there are some
problematic spots that vary depending on the posture.
One interesting pattern is the center targets taking longer than the edge targets.
We think this may be caused by the spherical space being asymmetrical relative to the shoulder
line. That's because the reachable range is larger on the body side.
As a result, the center of the input space is offset from the shoulder line. And that
may confuse users.
In summary,
We found the spherical mapping works well in restful postures and can be easily implemented
with a smartwatch.
However, we found that unlike a simpler plannar mapping, it has some spatial bottlenecks,
which you may have to design your application around.