Hi, I am Ninad here and today I will be presenting about Virtepex:
A system for virtual, remote telephysical examinations.
This work was done in collaboration with my co-authors Kevin Desai, Balakrishnan Prabhakaran and Thiru Annaswamy.
During an in-person strength assessment, a provider usually grades
the strength of joints into discrete levels
by physically interacting with the patient
However, there is an increasing need to provide accessible care to those unable to make in-person visits
Over the past two years, telehealth adoption has accelerated due to the pandemic.
However, current standards are often found lacking.
Our needs assessment study indicated that physicians were willing to adopt
novel telehealth procedures and desire the ability
to monitor specific joints and obtain high-level but principled feedback on range of motion, and strength.
So we asked ourselves, can we do better than the current standard of video call in providing principled feedback.
Virtepex uses a marker-less force estimation pipeline to estimate the strength of an observed upper body arm movement
like for example, the elbow flexion.
It also enables an asynchronous mode of evaluation for non-urgent, routine care.
In terms of ease of use and strength estimation detail,
Virtepex lies between the two extremes ends of rehab
monitoring systems, with a simple video call on one end and complex haptics and sensor-based systems on the other end.
Our primary motivation was to enable a easy to use, but objective feedback mechanism for physicians, in performing a remote strength assessment procedure.
Here we briefly go over the workflow for a Virtepex-guided assessment.
For a specific upper-body joint movement like elbow flexion, we track the joint angles using the Microsoft Kinect depth camera.
It provides real time information about 3D position of different human body joints and also supports tracking the hand gestures of a user.
Via the open/closed state of a hand gesture, we infer the start and end of a user activity session
which gives us the range of information for the specified joint along with the total time it took to perform the movement.
Using these two values we estimate the force required to perform the movement via an inverse dynamics solver.
The estimated force is used a proxy for strength and transmitted to a virutal object like a bowling ball
for interactive visual feedback to the physician.
The virtual environment was developed using the Unity game engine for simulating physically plausible motion of the virtual objects.
In the asynchronous mode, the physician has an option to compete with a pre-computed force estimate of a patient
and evaluate how well the patient performs against their own movement.
Virtepex is developed following participatory design principles.
During its design phase, we evaluated the system with 3 experts from Physical Medicine & Rehab department.
The design process began with a needs assessment study for discussing current telehealth methods,
and some of the challenges faced by physicians in assessing strength remotely.
This study formed the core part of the initial guidelines.
Following the study, we enhanced the prototype by
modeling the assessment like an virtual exercise game and providing an option for having asynchronous evaluations.
We validated the implementation of the force estimation pipeline with a simple experiment.
A positive, linear correlation was found between the estimated force values and the range of motion.
The experiment involved repeated motions of elbow flexion
with varying degrees of range of motion, but keeping time for the movement same.
Since the estimated force value essentially depends on two things, the range of motion and the time for the movement
Now, we take a look at some of the in-game interactions of the system.
We begin with a demo of the tutorial scene which was included to help familiarize users with VR and hand gesture based controls
As we can see, the ball changes its color to signal to the users about the current inferred state.
Next we take a look at PinHit scene which involved virtual objects like the bowling ball seen in the video
Virtepex has ability to track four upper body joint movements
involving the elbow, wrist and the shoulder.
The estimated range of motion is also displayed along side other important metrics
ike hand state and the specific movement being tracked.
You can notice that for the shoulder joint, we track two types of movements, shoulder abduction and flexion.
whereas we only consider the flexion movement for the wrist and the elbow.
Finally we see a demo from the asynchronous mode of Virtepex
where we simulate a collision between two bowling balls corresponding to two different users.
In this mode, an opponent or the secondary user has already performed the joint movement and recorded their force values with the system.
Using these recorded values, the main user can compete against and test themselves
for example, against a benchmark set by the physician.
Such a mode also has applications in long term rehab monitoring where a patient can track progress against their own personal benchmarks.
We evaluated Virtepex by conducting experiments with 10 experts
from the physical medicine and rehab department of a local Veterans Affairs hospital.
The main goal of the study was to measure the effectiveness of the system
as tool for evaluating patients remotely and in an asynchronous manner.
The experts interacted with the different scenes inside the virtual environment
and tried to evaluate the strength of a pre-recorded user.
Then, they scored along the four themes of ease of use, quality of experience
comparison with in-person and comparison with telehealth
In the expert evaluation, Virtepex fared favorably in comparison to the standard telehealth procedure
due to its objective nature of feedback and overall ease of use
Improved ease of use is critical since that is a key factor in real world adoption.
As expected, an in-person interaction was still considered better than Virtepex
presumably due to the much stronger signal for a patients strength via the physical isometric test.
However, it is important to note that beating,
or replacing the in-person interaction was not the primary design goal of Virtepex.
There were a few limitations with our system that were noticed during the evaluation phase.
We note that Virtepex cannot obtain high precision force estimates
due to its modeling assumptions and for such use cases, on-body sensors might be better.
On a related note, the range of motion and time computation are dependent on accurate hand gesture inference
and hence, errors in inferring the hand state can lead to noise in the downstream force estimation.
As a part of the future work, we plan to leverage the webcams found in laptops instead of the Kinect camera.
With the recent progress of deep learning models on the task of monocular human pose estimation,
webcams in laptops can potentially be used for real time motion inference and providing accurate 3D positions for upper body joints.
Lastly, I would like to acknowledge Keerthana, Richard and Sara
for their help with the project.
This concludes my presentation for our Virtepex paper.
Thank you for listening!