Hello, my name is Francisco Perella Holfeld from the University of British Columbia.
Today I'm presenting our paper titled Evaluating Design guidelines for hand proximate user interfaces.
Head mounted displays are becoming more and more accessible as an everyday computing device, but they still lack an expressive,
efficient and comfortable interaction modality that doesn't require users to have to hold the device in hand or have a display directly in front of them.
Borrowing from the interaction modality of smartphones, researchers have explored the use of the hand as an input and output modality.
Our previous work on hand proximate user interfaces or HPUI builds on this literature and drives the design space where the user can interact with
content with only one hand, where the content is displayed on and around the hand and interacted with using thumb to finger interactions.
Multiple studies have been conducted on using the surface of the hand as an input method, taking advantage of proprioception for input accuracy.
Displaying elements on and around the surface of the hand to improve such interactions has also been investigated in depth,
with HPUI being one of the results of such research. However, how to design for such context can be a difficult question.
While many of these works have explored the design space itself, it's not obvious how one would develop an application if we were to imagine this interface
to be used for applications like social media. Can existing design principles be used to develop these apps to be used with HPUI? Not directly.
Consider the design space of HPUI with thumb to finger interactions.
Even if you hold the hand flat, it's not a rectangular planar to surface, definitely not when moving the fingers.
With each movement of the hand, the surface continuously deforms.
The fingers are all separate surfaces that move independently of each other, so large displays on the hand are broken up by their movement.
And for thumb to finger interactions, there's a limited surface on the hand available for comfortable use.
Considering the characteristics of this design space, it's not obvious how one might design an application for this interface. In this paper, we try to answer this research problem.
We do this by first developing an initial set of guidelines based on literature and experience, then validating and refining them with the results from a design study.
To gather insight on how to design for hpy, we conducted A participatory design study with 15 participants each with UI design backgrounds
Since it's a novel design space, we also provided a set of demo applications for them to try out in virtual reality.
Following the demos, they were directed to propose new designs. The participants worked in groups of three to each design their own HPIU.
For four different applications, after which they would present their designs to each other and give feedback.
These are the 16 sample designs that the participants were introduced to in virtual reality.
The designs they were shown were split into four applications, each with three main designs and one sub menu design.
The four application types were chosen to cover different aspects of most applications commonly seen today.
A media player for single screen interaction with a relatively limited number of.
Inputs a social media application for one-dimensional traversal and also viewing text and images. A maps application for two-dimensional traversal and interacting
with larger continuous elements and a home menu for open-ended designs not based on any preexisting application.
The participants were not informed of the preliminary guidelines. We based our examples on as to not affect their own designs.
For each application they designed, participants were given verbal instructions as well as palettes such as this one, where they are examples of elements
they can use for their designs, as well as screen shots of the examples they had used in virtual reality.
The designs and discussions of the participants yielded many insights that allowed us to further refine the guidelines.
To keep things simple, in this presentation, we're not going to be including the analysis itself. We encourage anyone interested to read through the paper for the details.
I'll talk about each guideline and mention the relevant details from the analysis where applicable. The first guideline states to keep the adapted HPUI layout consistent
with mobile and other mixed reality UI standards.
This is not a surprising 1. Users have gotten accustomed to certain design conventions, maintaining consistency
of UI layouts allows the user to transfer their knowledge of other systems to interact with HPUI.
This reduces the time needed for familiarization and the learning curve for advanced interactions. We also saw this in our study,
where most participants cited the smartphone as an inspiration even though they didn't directly try to emulate it.
The relative placements of the elements were based on the layouts commonly seen in the corresponding mobile applications.
Guideline 2 - most frequently used thumb to finger interactions must be placed where it is most comfortable to reach for a majority of users.
After trying the demo applications, the majority of participants consistently placed the main interaction elements for an application in the
index and middle fingers of the hand, often relegating non essential interactions to the ring and pinky fingers.
Citing that those last two fingers were less comfortable for frequent use, the spread of elements laid out on the fingers as seen in the top two figures
is strikingly similar to the results seen in previous works, which showed the index and middle fingers being considered more comfortable.
Guideline 3 - for content where legibility is of significance, they should be placed as static surfaces where possible to be able to interact with continuous interactive elements like maps.
Deformable surfaces were explored as an option by filial and authors. This was primarily because of the limited reach of the thumb when considering single handed interaction.
While we had initially assumed the deformable surfaces might be used more when presented here, participants mostly preferred not to use them
primarily because of the impact the deformations had on legibility.
The participants preferred legibility over the ease of directly selecting an element, particularly when there is a continuous surface connecting the fingers.
Most participants preferred having such continuous surfaces in the space around the hand, though some participants did prefer having
the deformable surface because it was easier to directly touch with the thumb, which is analogous to the interactions on smartphones.
In particular, the participants use such deformable surfaces for elements such as volume controls or a seek bar laid along the finger.
Guideline 4. Limit the number of discrete elements to the landmarks on the fingers. The proprioception and tactile cues
from thumb to finger interaction could provide better accuracy for input, but prior work on this has mostly observed that the touch accuracy
of the thumb onto the fingers is similar to interactions on touch screens. An advantage of thumb to finger interaction is the potential for eyes free interaction .
But such an advantage is hindered by small UI elements with no point of reference on the hand.
While this would need further investigation in the future based on our current observations as well as the results from the study,
we recommend placing elements in clear segments on the surface of each hand, such as the separations between the phalanges of the fingers.
For a greater degree of precision when interacting with HPUI elements.
Guideline 5 - Prefer hpy for micro gestures. Use clearly marked gesture widgets and if using gestures or voice clearly denote them with appropriate labels and tooltips.
Gestures in general have lower discoverability. Hence, having elements that are clearly marked as interactive
relieves the user from having to remember gestures, while the design space of HPUI allows for a rich set of micro gestures
It's preferable to have visual components that guide the users to learn gestures.
For example, we use the home button in the palm as opposed to using a closed fist gesture or we use widgets like the circular scroll displayed on the finger.
Another advantage of HPUI is the availability of tool tips. This also can be used to further assist the users when less discoverable interactions are needed.
Guideline 6. If gestures are to be used, use socially appropriate micro gestures. This is an obvious one and a fairly straightforward one.
Large or inappropriate gestures may not be ideal for public spaces. For example, one of the participants created a sub menu with an alternative gun mode interaction
where the user makes a pistol shape with their fingers and mimes shooting UI elements. Understandably, after a
brief discussion with other participants, they all concluded it would be socially inappropriate and public contexts.
Finally, we have guideline 7. The user should have full control over the size and positioning of HPUI displays around the hand.
We did not have this in our initial set of guidelines originally but it was born of participant feedback and designs
Multiple participants expressed a desire to customize placements to better fit their comfort. Some participants prefer holding the hand
vertically as opposed to horizontally, and similarly, some prefer having the displayed elements around the hand at different sides of the hand itself.
So having predefined positions would not be ideal. Some participants even incorporated this idea into their own designs.
For example, the design on top includes a micro gesture joystick on the lateral side of the proximal phalanx of the index finger to move windows around the hand.
the design under it uses the orientation of the hand to shift elements to any desired position. Some participants also suggested
the possibility of allowing these elements to be anchored to the head as opposed to the hand were applicable.
We plan to continue refining these guidelines through their implementation into functional applications and their evaluations.
These results will be used to inform new empirical investigations in the HPUI design space, evolving and verifying how HPUI can be best implemented for all contexts.
Thank you very much.