Hello everyone, I am Elena Di Lascio
it's my pleasure to present our work entitled A Multi-Sensor Approach to Automatically
Recognize Breaks and Work Activities of Knowledge Workers in Academia.
Work-related stress is becoming
a serious issue for knowledge workers who often have high responsibilities for planning
and managing their work.
Recently is growing interest in using personal informatics systems that
leverage information automatically inferred from sensor measurements embedded in personal
devices in order to send timely notifications or to display insightful information to workers
with the main goal of making workers happier and more productive.
In this context, the ability
to automatically recognize when the user is engaged in a work activity or taking
a break is a fundamental primitive that personal informatics systems need to implement.
For instance,
the system can recommend the user to take a break when prolonged work is assessed,
or notify the user when it's time to go back to work.
Given the complex nature of work and break activities,
their automatic recognition is a challenging task. Breaks and work activities
can have similar characteristics and using a single source of information or a single
sensor might cause misclassification. For instance, being at ones' desk doesn't necessarily
imply that the user is working on a work activity, or people might take a break and work in the
same location. users might not use the laptop during work activities, for example while
reading a paper or reading a book, but also when they are engaged in conversation during
breaks.
So in order to address this challenge, we propose to use multiple sensors collected
with multiple devices such as laptop, phone, and wristband in order to capture different
characteristics of people during work and break activities. In particular, we monitor physiological activation
using electrodermal activity and blood volume pulse, user's movements using accelerometer, phone usage
with a custom application and laptop usage using RescueTime monitoring tools. We then propose
to use multiple devices equipped with timers for collecting self-reported activities. We
design and develop a situated self-report device called Devo, a mobile application, a Desktop
application and we also allow the workers to use a paper diary. With these devices, we collected
after intensive data cleaning, 449 labelled activities from nine knowledge workers
in Academia.
I now introduce our observation on the use of multiple devices to log
self-reported activities.
To evaluate participants' preferences in choosing specific devices
to log self-reports, we use the report frequency metric as suggested in previous work. It represents
the ratio between the number of self-reports and the number of days of data collection,
we notice that most of the participants use more than one device to collect their activities.
This hints to the need to account for personal preferences, so using multiple devices to collect self-reports
in the workplace seems to be a suitable solution. We also noticed that participants
selected more often Devo to log their activities. And also participants reported that Devo was
funnier and easier to use
compared to the other devices.
I briefly described the procedure we proposed for the automatic recognition of work and break activities, more details are
in the paper. We started with the cleaning and pre-processing of our sensor data, applying
channel-specific approaches for filtering and decomposition techniques. Then we segmented
the traces using five minutes windows and we extracted representative features. We then implemented
user-independent classification pipeline, we use imputation strategies, resampling techniques
in order to account for the imbalance nature of our dataset, we then apply standard scaling
filter methods feature selection, and we test different classifiers and notice that
the gradient boosting classifier was the one achieving the best performance.
I'm now briefly
presenting some of our key findings. So first of all, we analyze the impact of different
combination of sensors. And our best results is obtained when using data from all the devices,
achieving a f1 score 69% for breaks detection and 94% for work detection. We then compare our
approach with rule-based classifier, we implemented a time-based classifier which labels as break
every five-minute window that belongs to lunchtime. And we also implemented a laptop inactivity based
classifier inspired by previous work, which labels a window as a break
whenever laptop activity is not present. So our results show that our approach outperforms
the rule-based classifiers especially for the identification of breaks.
Furthermore, by analyzing the distribution of the features in the two classes, we observed that during
breaks people tend to be more physiologically aroused, more physically active, use more
phone and less the laptop
To conclude this talk, in this work, we
demonstrated it is feasible to automatically recognize work and break activities
using sensor data collected from multiple personal devices. We also noticed that workers
tend to use multiple devices to log self-reports so a multi-device strategy seems able to
accommodate users' needs.
We believe that our findings can open up new possibilities in
in the design and development of personal informatics systems for supporting knowledge workers.
For more information, check out our paper and we thank you for your attention!