Hi Ubicomp and ISWC audiences. I'm Chongyang the final year PhD at UCL. I will present our work
about leveraging activity recognition to enable protective behavior detection in continuous
data. This work was done in collaboration with my fellows, Yuan Gao and Akhil Mathur,
and also with my supervisors Amenda NicLane and Nadia. People suffering from chronic pain
tend to avoid movements that they fear. As you can see, in this video, different movement strategies
were adopted by this participant, like stiffness and hesitation. We refer to these as protective
behavior as people think it can reduce the chance of increase pain or injury. In fact, such behavior
leads to more pain and more debilitation. Physical rehabilitation is the mainstream
treatment for people with chronic pain, with the aims to reduce their use of protective behavior
and also help overcome fear. It is also aimed to build up their physical abilities.
During the rehabilitation, physios would tailor or adjust support and feedback
according to the presence of protective behavior, like they would suggest simplification of the
feared movement. They will also remind them to take a breath when the muscles are too tense. As
the rehabilitation is extending to people's daily life outside the clinic, technology should be able
to automatically detect protective behavior in order to personalize the physical rehabilitation.
Here it is a complete data stream of a patient performing a series of activities like stand
on one leg and stand-to-sit. And this is the majority voted groundtruth for the protective
behavior. The task is to detect the presence of protective behavior in this continuous data.
In the past, the backbones for processing movement data have developed a lot in the past few years.
At first, recurrent networks were directly applied to data matrix globally. Later on,
networks working on local movement vectors were proposed. Then, the rise of the graph
convolutional network showed its effectiveness in modeling skeleton-like movement data.
However, directly using those backbones in such continuous data does not promise a good
performance. As we can see that, the precision recall area under the curve for the binary task
is all below the chance level. Thereon, this paper proposes to improve the modeling performances
from two perspectives. First factor we can tell from the data side is that is the data
imbalance. As we can see, the positive class of protective behavior only take up for around 21%
of the data. Toward this we combined two successful plugins for loss function engineering,
that were originally proposed for object detection in computer vision.
The first is the focal loss and the second is a class balance term. We adapted those
two for the categorical cross entropy loss used in this paper. By alleviating the data imbalance
during training, the performance improved, but the PR-AUC is still below the chance level.
By further looking into the data, an informative evidence for the protective
behavior had been found is that the activity background. As a result, we
propose to contextualize the protective behavior detection, namely PBD, with activity recognition,
namely the HAR. And here is the architecture we found to be better for the job via our
preliminary experiments with many different learning frameworks candidates. Now the
performance is further improved with a PR-AUC better than the chance level. And the best
performance is achieved by alleviating the class imbalance therein. This paper also discussed the
impact of sensors number on the performance by testing on the input graphs with different sizes.
The performance dropped, but still better than the chance level. We believe with better modeling
methods and the hardware designing the system shall be more efficient for people's daily life.
To sum up, the ubiquitous community working on body movement data that collected with
wearable devices or from visual systems has been focused a lot on the task of activity recognition
in the past. Thereon, the research is now moving widely towards the understanding of
movement behavior for purposes of healthcare and wellbeing. In our paper, we bridged the activity
recognition and behavior detection, which co-occur in our real life with one altering the other.
There are still some limitations about our paper, for example,
the activity recognition was not improved with behavior detection. And, we look forward to seeing
the brilliant minds all over the world for better solution in this direction. Cheers!