Hi everyone, I am Xuhai. It's my great pleasure to present our work:
Leveraging Collaborative-Filtering for Personalized Behavior Modeling - A Case
Study of Depression Detection among College Students.
Depression is a common but significant health challenge.
The percentage of the population with depression has been increasing in the past few years,
especially for people aged from 16 to 25.
More than 10 percent of young adults in the US had at least one depression in the year.
There is a growing realization that data collected from everyday devices can help us to understand
the relationship between people's daily behaviors and symptoms of depression.
However, most of the prior work on algorithms follows a one-size-fits-all approach,
building one model for all users to detect depression.
As an old saying goes, no two snowflakes are the same. Similarly, no two persons are the same.
Such an approach de-emphasizes the important fact that individuals behave differently,
leading to lower model performance and limited population-level interpretability.
Training a personalized model on a user’s own data is a straightforward solution,
but requires a large amount of data, especially ground truth labels.
However, mobile sensing studies, especially those focusing on mental-health-related issues,
have a lot of passively sensed data, but usually very few labels.
Our novel insight is that labels drawn from relevant people can help to make
up for fewer labels, using a method inspired by collaborative filtering.
A user-item matrix is a typical setup for a collaborative-filtering-based recommendation
system with user ratings for items like goods or movies.
New item ratings can be suggested based on similar users' ratings of content a user has not seen.
Our approach is analogous. Instead of items, each column is a feature calculated from passively
sensed user behavior at a specific time. The final “item” is a label, in our case the
user’s score on a depression symptom scale. When a new user comes in, we can leverage
the same user similarity score developed for collaborative filtering to identify a specific
group of users to predict a “rating”, which is depression score in our case.
In the next few slides, I will introduce our pipeline.
We collected data from a wide range of sensors, such as Bluetooth,
screen, GPS, and extracted a number of features from the raw data.
Each behavior feature forms a time series which we define as
a user behavior profile. We put each feature into one user-behavior matrix.
Then, we calculate a behavior relevance matrix. People who behave similarly and
who behave oppositely will have high relevance. Thus, the matrix uses the square of the Pearson
correlation between two users' behavior sequences. We then conduct a feature selection process on
the training set. For each feature,
an inner leave-one-user-out loop is used to find the most important and stable features.
When a new user comes in, we first calculate their relevance scores against the users in
the training set. For each selected feature, we choose the training users who have high
behavior relevance scores. This leads to a personalized training set for each new user.
We assign label score -1 for non-depression and 1 for depression.
Then, we use the relevance scores as the weight to aggregate the training users’ label scores.
A score greater than 0 leads to depression, otherwise it leads to non-depression.
This gives us an intermediate classification output for each feature.
Finally, we use majority voting to merge these outputs into the final classification.
We evaluate our method on two independent datasets.
The ground truth was collected via a well-known depression survey BDI-II.
Our goal is to detect whether a student has at least mild depressive symptoms.
The two datasets were collected from Carnegie Mellon University and University of Washington.
Each dataset has around 100 users, and based on the BDI-II outcomes,
around 40% of them reported at least mild depression at the end of a semester or a quarter.
We apply our method to the datasets, and the figure shows the accuracy and
F1 score of a simple majority vote baseline, a state-of-the-art algorithm, and our method.
On the left, our method achieves over 5% higher accuracy score and F1 score on the CMU dataset.
We replicate our pipeline on the UW dataset,
and achieve about 3% advantage compared to the state-of-the-art, as shown on the right.
These results indicate the effectiveness of our method.
Moreover, our method goes beyond just having better prediction performance.
In addition to building an accurate model,
we need to know more about the contexts to better interpret an individual's behaviors.
To enable personalized interpretation, we extract personalized behavior rules that can capture
the behavior differences between users with depression and users most different from them,
which means users in the training user group that had a negative correlation.
We discretize features and employ association rule mining to find frequent behavior rules.
We identify contrasting rules that have identical X and different Y.
The example here shows a pair of contrasting rules, where the X indicates users spent a small
amount of time at social places, but for Y, a user with high depression had more disturbed sleep,
while users in the contrasting group slept better.
We plan to seek collaboration with domain experts to further study how
these personalized interpretations can be used to support personalized therapy design.
To summarize, in this paper, we present a new approach to leverage behavior
relevance among users for personalized classification on mobile sensing data.
We apply our method to depression detection among college students.
Compared to the state-of-the-art, our method has significantly better performance.
Moreover, we demonstrate that our method can generate personalized behavior rules
for students with depressive symptoms.
With that, I'd like to end my talk. Thanks to my wonderful team for doing
the great work together. And thank you all for listening. I am happy to answer any questions.