My name is Zhu Wang, I am a researcher at New York University. I am excited to present
our work, VRGaitAnalytics: Visualizing Dual Task Cost for VR Gait Assessment. It is a
walking balance assessment system that can help experimenters investigate individuals'
gait performance during walking and obstacle crossing, and provide spatial data visualization
to better understand and interpret the experimental data.
Human balance is the ability to keep the body stable and prevent falling when standing or
moving. It is essential for our health and quality of life due to its necessity in sports,
exercise, and all kinds of daily-life activities.
According to the data from CDC in the US, falls are the top public health problem for
older Americans, it could be more severe and costly than we thought. Over 25% Americans
older than 65 experience at least one fall each year. Fall-related injuries bring one
to the emergency room every 11 seconds and cause one death every 19 minutes. The direct
cost of the medical attention for fall injuries reached 50 billion US Dollars in 2015, and
expected to reach $68 billion by 2020.
Falls typically happen when changing environmental conditions, tripping over an obstacle, and
multitasking. Changing environmental conditions brings multi-sensory cues, and causes changes
in multi-sensory input, including visual, touch, vestibular, and auditory input. Tripping
over an obstacle and multitasking are another two common causes of falls, and particularly
challenging for aging adults. They would be vulnerable to falls when stepping onto an
obstacle such as a curb if they did not notice or could not clear it safely. They may also
have difficulty walking safely when engaging in a secondary task such as memorizing words,
or looking at phones.
Our goals include understanding of the relationship between balance and the stimuli of visual,
auditory and cognitive load. With all the causes of falls taken into account, we designed
and implemented a balance diagnostic tool to
access balance with quantitative balance parameters and qualitative observations, examine the
response to multisensory stimuli, investigate balance strategies such as obstacle crossing
and avoiding a collision and also make it accessible to clinics or even home use.
Our day-to-day life requires performing concurrent cognitive tasks while walking, such as walking
and texting, walking and navigating on a busy street, are all examples of dual-tasking that
combine walking with cognitive abilities. Performing two tasks concurrently might require
more than the participant’s total attention capacity, which impacts the performance of
either task or both. This is called the "dual-task interference cost"
The system has a dual-task paradigm where participants need to memorize numbers while
walking with or without negotiating obstacles.
The system uses HTC Vive Pro to present VR environments with spatial audio, the headset
combines a wireless adapter to make the VR headset untethered, Vive trackers mounted
on the top of the feet to get foot kinematics. The VR system cannot provide somatosensory
input of the foot to simulate the impact when participants hit the obstacle, so the feet
are represented by graphic foot models in VR for the participants to visually detect
their own feet and identify if their feet cleared the obstacle. The system also provides
auditory feedback to indicate the success or failure of the obstacle crossing. We measure
foot dimensions and foot tracker placement to align the foot models to their feet.
The system provides 3 major stimuli. Visual, auditory, cognitive. Each of them has low
and high intensity. The stimuli are independent variables which can impact balance, experimenters
can control the stimuli and present participants the permutations of the stimuli. The motor
tasks include normal walking, and crossing virtual obstacles during walking. For the
obstacle crossing, the system can generate anticipated obstacles generated with specific
height at the start of the scene. So the participant can have enough time to plan the obstacle
negotiation strategy. The system can also generate unanticipated obstacles with the
selected height but at a random time. Participants will not be able to predict how far or when
an obstacle will appear in front of them. Experimenters can observe how participants
react to specific conditions, measure the balance and investigate the underlying causes
under different conditions.
VRGaitAnalytics uses a spatial data visualization module to instantaneously provide VR playback
and outcome measures to indicate gait performance.
In clinics, experimenters and participants may have the demand to record and playback
trials, so the system provides motion playback, allowing experimenters and participants to
observe and interpret their balance performance. The VR playback allows them to inspect motion
at any moment of the trial. They can freely change their viewing position and angles.
The VR user views the replay as an observer, the video shows how the system replays a trial.
The avatar was driven by recorded data, and the VR user can watch the playback as an observer.
When watching their own performance, participants can learn how to plan a successful obstacle
crossing and perform the task better. In addition, VR playback does not have confidentiality
issues compared to video recording.
The system can visualize the motion trajectory for each body segment. The trajectories are
rendered behind the moving segment and could be useful for better visualization and motion
interpretation.
Overlapping the VR playback of the two trials and replaying them side-by-side simultaneously
can make it easier to detect between-trials differences, such as which trial has a lower
foot clearance, longer step length, shorter step width, etc. So we implement the trial
comparison module to load two trials' playback data and replay them in the same time frame
and the same place
Experimenters have full control over the display of the two trials. The two trials have different
color coding for their motion trajectories, and different transparency renderings for
the segments. Experimenters can replay the trials from the beginning to the end simultaneously,
or jump to a specific frame. For each trial, there is a corresponding frame slider for
users to pick any frame and align the starting point of the trial with the other trial. For
example, users can use the frame sliders to align two trials based on the moment of their
second stride of the left foot, and replay the two trials simultaneously from that moment.
The measurement module calculates metrics in real-time such as walking velocity, step
length, step width, the success rate of obstacle crossing. Experimenters and participants can
select what metric to display, and view the data plot of the metrics in VR or on screen.
The outcome measure and its visualization can give them feedback right away instead
of waiting for an offline report.
Our preliminary analysis for the step length and step width in obstacle crossing show that
step length decreased but the step width increased under high cognitive load compared to no cognitive
load.
The result supports previous studies where healthy young adults demonstrated a shorter
step length and a wider step width when walking and performing another cognitive task. This
difference is caused by dual task cost。
This project is the collaboration across NYU COURANT, STEINHARDT, and Tandon. We are grateful
for the funding from the NYU Aging incubator. Thanks to all of them. And Thank you for coming
to our virtual talk.