Hello, my name is Ian and now I am going to present our work on improving human activity
recognition through self-training with unlabeled data using a novel method called "SelfHAR".
This is a collaboration between different researchers from the University of Cambridge.
Human activity recognition (HAR) is the task of identifying what the user is doing,
such as whether the user is walking or running from sensor data.
These sensors are commonly found in mobile devices, such as accelerometers and gyroscopes.
However, there are major challenges in HAR, one of which is the inherent
difficulty in collecting labeled HAR data. Compared to other data modalities,
it is almost infeasible to label sensor signals after they have been collected,
so it requires a very high level of participation from the users during data collection.
Also, there is a large number of variations between different setups, such as different
devices and sensors, different placements of sensors, and different sets of activities.
So, the main objective of the work is to improve the performance of deep learning HAR models
by using large, unlabeled datasets collected in
free-living environments to complement labeled ones with semi-supervised learning techniques.
And now I am going to present our work and method, SelfHAR. The main idea of SelfHAR is
to combine teacher-student self-training and multi-task self-supervised learning,
and it consists of four major steps: First of all, we train a teacher model
using conventional fully-supervised training on the labeled dataset.
And then we run the unlabeled dataset through the teacher model to obtain pseudo-labels,
and then we perform sample filtering to select the most useful samples to form the self-labeled
dataset. And then we combine this self-labeled dataset with self-supervised learning techniques
to generate multi-task transformation samples and labels, by augmenting the
data samples with signal transformation. This forms the multi-task self-labeled dataset,
which consists of the augmented data, the transformation labels, which indicate which
transformation has been applied, and the HAR labels. And finally, we first pre-train the
student model in a multi-task setup involving both self-training and self-supervised learning,
and then we fine-tune the student model using the labeled dataset at the end.
And now I am going to present some results. The first experiment that we wanted to conduct was to
evaluate our method against other semi-supervised learning techniques. In this setting, we make use
of the full labeled dataset and the full unlabeled dataset, except for the fully supervised baseline.
In this table, the first column corresponds to the datasets, the second column corresponds
to the fully-supervised baseline, the third one corresponds to transformation discrimination only
pipeline which is the previous state-of-the-art. The fourth one corresponds to self-training only,
which is an ablation study that we isolate the self-training element,
and the last column is our SelfHAR pipeline. Here we can see that out of all cases, compared to the
fully-supervised baseline, our SelfHAR method is able to out-perform it in all cases. And also,
compared to the previous state-of-the-art, our method was able to out-perform it in 5 out of
6 datasets. Please note that out of all these pipelines, the final model has the exact same
complexity and architecture, which means that there is no extra cost incurred at run time.
And we also wanted to evaluate how well our pipeline would do in limited labeled
data scenarios, where we limit the number of labeled samples per activity class to 2, 5, 10,
50, and 100. Here, the green plot corresponds to our SelfHAR pipeline and we can see that compared
to the fully supervised baseline, our method out-performed it in all cases by a large margin.
For example, in the HHAR dataset, our method is able to achieve similar performance using
almost 10 times less labeled data. Compared to the previous state-of-the-art, our method is also able
to out-perform it in most cases, especially when there are more than 10 labeled samples per class.
So to conclude, we proposed SelfHAR, which is a method based on combining self-training and
self-supervised training. We have shown that SelfHAR is performant, where it outperformed
previous state-of-the-art semi-supervised training methods in the majority of cases.
And we have shown that SelfHAR is data-efficient, where it achieved the same level of performance
using up to 10 times less labeled data compared to supervised training, which
indicated the ability of leveraging unlabeled datasets in increasing the diversity of data.
Here are some references that I used in this presentation.
For more information, please feel free to visit any of the following websites,
or get in touch using the email address at the top. Thank you very much.