I present to you the effects of body location and bio signal feedback
modality on performance and workload using Electromyography and virtual reality.
I am Jessica Sehrt and I am a Ph.D.
candidate at the Frankfurt University of Applied Sciences.
For understanding
and improving muscle control as interaction technique,
it is important to consider the physiological difference
between muscle tension caused by moving a limb
called isotonic contraction
and isometric contraction,
which must be consciously activated
without any movements.
Isometric contraction is, for instance, designed to prevent artifacts and signal
caused by movement or to prevent crosstalk of multiple
neighboring muscles.
Particularly assessing the isometric muscle activity
using electromyography sensors is indispensable for a wide range
of medical, assistive and interactive applications.
Such systems are desired, for instance, when isotonic muscle movement
is not even possible at all due to motor impairments
to control electric wheelchairs, exoskeletons
and remote robotic systems.
Such systems are desired to trigger events in hands-free interaction,
to prevent unintentional motion based input or can even support
the restoration of neural pathways when only the intention to move
a limb is trackable, for instance, after stroke.
The so called biofeedback loop
describes a system that displays the bio signal back at the user
during interactions with the physiological functions of the own body.
Closing the biofeedback loop increases body awareness,
often causing changes to one's own behavior,
thoughts, and even body functions, which are typically
not consciously perceived.
In particular closing the biofeedback loop using EMG
sensors is helpful where muscle control can support multiple functions
or information about the muscle activity Signal
is required to regain such control.
Karolus et al showed that visual and auditory EMG based
biofeedback can increase body awareness during physical activity.
Choi et al showed that a closed loop biofeedback via tactile stimulus
based on user's heart rate was effective.
And Palermo et al presented an augmented reality environment
to provide visual feedback of EMG activity to amputees when training
for using an energy hand prosthesis.
As not all
humans activate and use their muscles in the same way.
We hypothesized that there are differences
in the user's input performance between different areas of the body.
Therefore, we conducted an exploratory VR study with 18 participants
and within subject design with the independent variable body
location based on a standardized Fitts Law target selection task
using EMG and a head mounted display as the pointing device,
as well as subjective assessments te measure
performance and workload.
We used the input of the hand at the VR controller as control condition
and compared it to the EMG input at the biceps,
the muscle of the upper front arm, to the Triceps Brachii
caput laterale, the muscle of the upper back arm,
to the Gastrocnemius caput medialis, the muscle of the inner calf,
to the Temporalis anterior,
the muscle of the temple and to the Flexor carpi radialis,
the most distinguishable muscle of the forearm.
Objective
performance measures of the study on the left
with the average throughput results from the physical task between the body
locations show the highest throughput by the VR controller,
which was to be expected as it was installed as control condition.
During experiment execution, we found that the participants
were not able to contract the temporal muscle isometrically.
After exclusion of the temple the quantitative results on the throughput
suggest that stationary, isometric muscle contractions do not significantly
differ in terms of the input performance between the muscle groups tested.
Importantly, there were no interaction effects
with the index of difficulty
as shown by the regression slopes on the right with target selection
time of each body location as a function of the effect of index of difficulty.
This finding indicates that the participants tend to point equally well
during conditions with all body locations independent from the level
of difficulty of the task.
The workload
score of the Raw NASA TLX questionnaire and its subscales
from the study show regarding the main score that the triceps
received a significantly higher workload rating than the muscle of the temple.
The VR controller received significantly lower ratings for perceived performance
than the triceps and also significantly lower ratings for effort than the biceps,
which was to be expected as it was installed as control condition.
Importantly, qualitative results revealed that the participants
probably did not activate temple muscle isometrically and instead
used isotonic movement, despite our instructions.
Comments were “…temple
is only good to control because you can press the jaw to activate the temple
muscle” or “temple muscle activation needs movement of the eyebrow and forehead
if you do not want to involve the pressing of the jaw”.
EMG biofeedback can help users
to focus on their own muscle tension,
however, which sensory cues
are suitable for rendering EMG signals
and closing a biofeedback loop as currently unknown.
As humans have a
limited set of resources available for mental processes, we hypothesized that
task performance based on biofeedback from multiple
cues can cause difficulties.
Thus, we investigated three feedback modalities
which were visual, auditory and tactile.
We conducted a three way full factorial within subject
designed study in VR and with 40 participants.
Due to the low variance of the throughput in the first study, its observability
during the experiment to ensure asymmetric contractions and its relevance
in related literature, we use the biceps
brachii at the front of the upper arm
for EMG input in our second study.
The throughput performance measures
for each biofeedback modality
show that the highest throughput was achieved using visual
and tactile feedback, as well as when no feedback was rendered.
A main effect
for the auditory feedback indicates that the average throughput
was significantly lower when auditory cues were present
compared to when not.
Target selection time as a function of the IDe shows that
the target selection time does not depend on auditory, tactile or visual cues.
The results of the Raw NASA TLX questionnaire
showed significant main and interaction effects on the feedback modality.
The subjective quantitative results also indicate
that input performance can be improved by presenting muscle tension
as simultaneous tactile and visual feedback
alongside the objective results.
The results also show that there is no single modality
that systematically improves the target selection time or workload.
Interestingly, while the main effect indicates that audio has a negative impact
on throughput, the subjective frustration was lower when audio was on.
Participants appreciated
informativeness and usability in their biofeedback modalities.
Importantly, the results show that some modalities can produce
stress due to the obtrusiveness, through repetitive patterns
or even by information overload while using too many modalities.
Negative side effects were caused by fatigue/exhaustion of the muscles
or through the system in general,
and an inconsistency between the signal and the biofeedback modality.
Interesting positive side effects where that some participants
noticed an interplay of learning and fatigue effect,
and that habituation supported the learning.
The conclusion from study one is:
isometrically activated muscles are more difficult
to control than movement dependent isotonic contractions,
and no difference in input performance was observed
among the different body locations with
isometric muscle contractions.
The conclusion from Study two is: interactions with EMG
based systems and VR designed to register isometric muscle contractions
can be improved with combined visual and tactile, multi-modal biofeedback.
We highlight the necessity of discrimination between
isotonic muscle contractions and isometric muscle contractions.
For further investigations, our approaches can be found on
GitHub.
Thank you very much.