Hello everyone, I'm Xin Qin from Institute of Computing Technology, Chinese Academy of
Sciences.
I'd like to present this research titled “Cross-Dataset Activity Recognition via
Adaptive Spatial-Temporal Transfer Learning”.
This work is jointly done by Xin Qin, Yiqang Chen, Jindong Wang, and Chaohui Yu.
Human activity recognition aims at recognizing activities by training models on the large
quantity of sensor data.
Since it is difficult to acquire abundant labeled data, it is necessary to utilize the
labeled dataset to help annotate the unlabeled one.
We often refer to it as “cross-dataset HAR”. However, data from different datasets usually follows
different distributions.
Directly training model on the labeled data may result in poor performance.
So how to build well-performed model for cross-dataset HAR?
Transfer learning is an effective paradigm for solving this problem, “What to Transfer”
and “How to Transfer” are two key points.
For “What to Transfer”, although several methods tried the source domain selection,
most of them work in deep neural networks which are computationally constrained.
As for how to transfer, existing work fails to consider the temporal relationship between
features, which may result in negative transfer.
First, to solve what to transfer problem, we show the spatially-adaptive feature learning.
We adopt the idea of adaptive distribution factor from an existing work.
We treat the divergence of marginal and conditional distributions as the global distance and local
distance.
And dynamically evaluate their importance. Different from the existing work using A-distance which
needs to build a binary classifier, we use MMD as basic distance metric to estimate μ.
By calculating the overall distance, we can achieve source domain selection.
As for how to transfer, we learn both spatial and temporal features.
We focus on the temporal feature learning by extending the work of Geodesic Flow Kernel
(GFK).
In general, GFK learns the transferable features gradually by integrating an indefinite number
of subspaces between two domains and gradually transfer the source domain to the subspace
of the target domain.
Along with this process, the features are more likely to be in the target subspace.
However, GFK treats all temporal factors as equally important.
Taking inspirations from Markov property, we believe the latter transfer has better
influence.
Thus, we extend GFK with the temporal factor to weight the importance of transformation.
Adaptively combine the two main learning components, we call the proposed approach ASTTL.
It learns the transfer prediction function for cross-dataset HAR.
We make some preprocess and simulation experiments on four public HAR datasets.
We construct cross-dataset scenarios by taking these datasets as target in turn, the other
three as source candidates and then build transfer learning model for the selected tasks.
Firstly, we evaluate the performance on source domain selection by calculating the domain
distance between the target and each source domain.
Taking the classification accuracy on the target domain after transfer learning as ground-truth
for domain similarity.
Results of ASTTL are consistent with the ground-truth, which is superior to the comparison methods.
We evaluate the classification performance by comparing with several methods.
And our method achieved the best performance on four transfer learning tasks, with an average
accuracy improvement of 8.7%.
Then, we compare the full ASTTL with two variations.
Results show that both temporal feature and spatial feature learning make important contribution
to the performance.
Besides, we make fine-grained evaluation.
Results show ASTTL can make good classification on each activity.
Sensitivity analysis shows ASTTL is robust in human activity recognition with respect
to a large range of parameter choices.
ASTTL reaches a steady performance within fewer than 20 iterations, we also analyze
the time complexity in our paper.
Furthermore, based on the proposed method, we design a system to make application in
the real environment.
It is composed of two components: the software on a smartphone for activity data collection,
and the software on a server for cross-dataset transfer learning.
Evaluation on the lab collected data shows that application achieves satisfying performance
in real activity recognition scenarios.
The survey on 24 users shows that most of users are satisfied with the system.
So to summarize, our main contributions are the followings:
We present an Adaptive Spatial-Temporal Transfer Learning approach, it learns the spatially
and temporally adaptive transferable features in a unified framework.
And it can achieve accurate source domain selection and activity transfer in cross-dataset
HAR problems.
It can be applied in the real-life activity recognition scenarios.
For more information, please refer to our paper.
Thank you!