Hi, my name is Marius Bock and I am excited to present our paper "Improving Deep Learning
for Human Activity Recognition using shallow LSTMs" which was joint work with colleagues
Alexander Hölzemann, Michael Möller and Kristof Van Laerhoven.
In our work we revisited the popular DeepConvLSTM architecture by Ordonez and Roggen.
To this date the architecture is a state-of-the-art method for applying Deep Learning on Human
Activity Recognition.
In our paper we suggest to simplify the original architecture by removing the second layer
of the LSTM, thus employing a 1-layered instead of a 2-layered LSTM.
The LSTM is parameter-wise by far the largest part of the architecture.
Omitting one LSTM layer thus resulted in a decrease of up to 62% in parameters and of
up to 48% in runtime during experiments.
Now, you might ask yourself at what cost does this change come?
How much does the simplification hurt the predictive performance of the model?
Well, to answer this question we compared the performance of our architecture to the
original DeepConvLSTM architecture using 5 publicly available Human Activity Recognition
datasets as input.
These datasets were the Opportunity dataset, the Wetlab dataset, the RealWorld HAR dataset,
the SBHAR dataset and the HHAR dataset.
The datasets differ in the amount of participants, number and type of activities and number of
features being employed.
To test both architectures we also varied the number of hidden units employed within
the LSTM layers.
We employed either 128, 256, 512 or 1024 hidden units, which left us with a total of 8 architecture
variations to be evaluated.
We used Leave-One-Subject-Out Cross-Validation and used the same set of hyperparameters across
all experiments.
The hyperparameters were obtained based on tuning results from the Wetlab dataset.
Each variation was tested using a set of 5 random seeds and thus ran 5 times.
The final results for each variation was then the average recall, precision and F1-score
across the 5 runs.
An exception was made for the Opportunity dataset where we chose to use the same preprocessing
and train-test split as Ordonez and Roggen and thus did not apply Leave-One-Subject-Out
Cross-Validation.
So, to come back to the initial question: how much of a decrease in predictive performance
does one need to account for when using our simpler architecture.
The answer is: None - the predictive performance actually increases!
As one can see with sample results obtained from the RealWorld HAR dataset, using our
simpler architecture, performance significantly increases as far as precision, recall and
F1-score is concerned.
Overall, our architecture continuously outperforms the original the original DeepConvLSTM architecture
for the Opportunity, Wetlab, RealWorld HAR and HHAR dataset, with recognition performance
going up as much as 11.7% in F1-score.
For the SBHAR dataset our architecture is only on par when employing 512 and 1024 hidden
units.
Now, let's have a look at a statement which was written in a recent survey paper by Chen
et al. on Deep Learning for Human Activity Recognition.
It says that "the depth of an effective LSTM-based RNN needs to be at least two when processing
sequential data".
This popular rule-of-thumb is based on findings presented by Karparthy et al., which visualised
and investigated how LSTMs learn within the setting of Natural Language Processing.
Nevertheless, during our experiments we investigated an opposite trend and had architectures using
a 1-layered LSTM consistently outperform 2-layered ones for 4 out of 5 datasets.
The goal of our paper is thus to challenge this rule of thumb that it might not be applicable
to sensor-based Human Activity Recognition.
Now, to conclude this presentation, let's talk about how we made sure to exclude overfitting
as being the reason for the witnessed predictive performance increase.
In order to spot the degree of overfitting, we closely monitored the generalisation gap,
which is the difference between training and validation accuracy.
The larger the gap, the more likely the network overfitted on the training data.
We also made sure to use datasets, which have both large and small feature dimensions and
varied the type of activities, which are to be predicted, by including complex, transitional
and simpler/ periodical activities.
All in all, we saw both architectures suffer equally from overfitting across all datasets.
Lastly, our directions for future research are two-fold.
First, we aim to validate our assumptions on larger Human Activity Recognition datasets,
for example extracted by using IMUTube.
As Human Activity Recognition datasets are relatively small in size compared to other
Deep Learning datasets from other research areas, validating our claims on larger datasets
would further rule out overfitting as being a reason for the witnessed performance increase.
Second, we aim to apply the same analysis as Karparthy et al. using Human Activity Recognition
datasets as input.
Doing so we hope to better understand and visualise the trend that we witnessed that
1-layered LSTMs might be a better suited option, when dealing with sensor-based data.
Here is a list of all the papers which were mentioned in this presentation.
And, yeah, this concludes our presentation.
Thank you for your attention.
If you are interested in our implementation and/ or results, our architecture as well
as everything to reproduce our results are available on GitHub.
So, just scan the code or type in the URL below and it will take you right there.
Feel free to also contact via e-mail if you have any questions about our implementation.
And, yeah, thank you for your attention!