Presented by Emilie Arendttorp from Aalborg University
Recent improvements in hand-tracking technologies
and Virtual Reality (VR) have made it possible to design controller free
hand gesture-based interactions using VR systems such as the Oculus Quest 2 and the HoloLens 2.
However, Gesture-based applications are often designed by professional system developers
without any input from end-users. Authors have suggested involving
users in the gestural language creation process. The majority of physiological behavior research
into designing natural gesture interaction done with Western, Educated, Industrialised,
Rich and Democratic (WEIRD) participants. Leaving out a large group in the design of natural
gesture interactions, here among the global South. When developing new interactions with user
groups already highly familiar with technology, there is a risk of designing with legacy bias.
Legacy bias occurs when participants and informants transfer previous learned user
interactions and experiences into the design and evaluation of new interactions.
Gestures are used in every day non-verbal communication,
through motion of body parts, such as hands and fingers, with an associated information.
A gesture can be understood by itself, or complementing verbal communication, gestures are often used
to emphasise or clarify statements, for example in intercultural conversations.
Previously, Billinghurst and Buxton have defined four gesture types,
followed by Aigner with a more refined distinction of five gesture types:
Pointing, semaphoric, pantomimic, iconic, and manipulation gestures.
We postulate that the inclusion of diverse user groups, especially the ones with little
prior exposure to technologies, can diminish legacy bias if included in early-stage of design
and testing of gestures. One such group is the small San community of Donkerbos.
Donkerbos is remotely situated in the Central-Eastern part of Namibia,
in the middle of the Kalahari Desert, only reachable by foot, donkey, horse or 4x4 car.
The community lives on a farm land,
on which they have built their own houses with available materials,
such as metal sheets, only with few households having solar panels for electricity. There is
limited access to water via the community borehole which works through a solar panel in the day.
The settlement consists of more than 30 households,
usually comprising of a couple with 3 to 7 children.
Primary learners attend a government school approximately 5 km from the settlement.
Due to their limited interaction with digital technology they do not have
the same legacy bias as Western people.
The Donkerbos community was currently planning the construction of a campsite for tourists,
yet could not visualize it, so we decided to base our study
on creating a VR campsite planning application using gestures.
Our study consisted of four phases.
First an exploratory study was conducted in the field consisting of one gesture session and one
technology VR exploration session. The first gesture session functioned as
a method to observe if the gesture types of Aigner were valid in Donkerbos.
The gesture session was partner-based, following the suggestion of Morris et al., using charades
for the inspiration of the gesture sessions. One participant (gesturer) was showing gestures,
where the other participants responded to the gestures (gesture- receivers).
The gesture session consisted of three different assignments, drawing, direction and charades.
A total of 119 gestures were recorded during the three assignments of drawing,
direction and charades. The gestures found through this session, was used for inspiration
for gestures to be implemented in a prototype for the VR headset Oculus Quest 2.
The prototype included eight objects for the participants to create and move around,
using buttons. Eight buttons for objects, and five for movement.
Three static gestures got implemented based on the first gesture session.
The static gestures were stop, pointing and thumbs-up.
The developed prototype was brought to the field for re-design based on another gesture session,
a VR session and a co-design session meant to resolve ambiguities and open questions
in regard to gesture interpretations.
Six groups consecutively participated in the co-design session. All groups were active in
sharing ideas in regards to the application, its content and interaction techniques.
It was surprising to see how creative they were in combining buttons to solve tasks that others
chose to solve with gestures. This speaks to the openness of the session in enabling the
participants with very little VR experience to create and re-design VR as a technology.
We recognize that natural gestures used in human-human communication
are not necessarily directly transferable to interactions in VR .
Upon further discussion with the community it was agreed that the thumbs-up does not serve
its purpose in the VR and should only be used as a concluding gesture to close the programme.
On the other hand, the stop gesture was immediately adopted
in VR interactions. We conclude that natural gestures suggested by users
need to be implemented and evaluated to assess their usefulness in VR.
Through out this study it has become clear that creating a universal gesture library
is neither possible nor desirable. While user-centered gestures need to be validated
against technical constraints we also imply that further research
into for example gender-sensitive gestures should be carried out. Thus designing for
the pluriverse suggests the implementation of multiple gestures for the same interaction.
Ideally a combination of methodological steps and the engagement of diverse user groups
should be targeted to reduce biases.
At a time were digital technologies encroach, and determine transformations in social interactions,
such as the Meta- verse and AI, it is more important than ever to recognise the possible but
also necessary contributions from researchers and users currently situated "at the margins of HCI".
This study directly advances the HCI agenda for diversity, through empirical work as part of an
ongoing research and design partnership with an indigenous community in Southern Africa,
thereby challenging systemic bias and promoting design for the pluriverse.