hi everybody thank you for being here i am Koustuv Saha and along with my co-primary
author Ted Grover, we are delighted to present our work on building person-centered prediction
models of psychological constructs with social media contextualized by multimodal sensing
we first thank our co-authors Steve Vedant Pranshu Gonzalo Pablo Gloria Aaron and Munmun
Now before i begin i would like to set the stage for social media as a passive sensor
social media platforms enable us to share our thoughts and connect with others
and this has enabled researchers to leverage social media as a passive sensor
to examine and predict human behaviors and well-being and social media data is
self-recorded in a self-motivated fashion in one's naturalistic setting and this data can
be collected unobtrusively to obtain longitudinal and historical data beyond study periods social
media alternatively functions as a verbal sensor as it can capture people's linguistic expressions
so while we saw some of the potentials of social media data it also comes with limitations like
even though social media data can be collected passively it is dependent to be actively
curated by the individuals social media data is sparse and not everyone is equally active on it
and again people's social media behaviors have offline underpinnings and complementary multimodal
sensing can contextualize offline behaviors which could address the limitations of social media
and help us obtain a holistic understanding about an individual and in this work we leverage
multimodal offline sensing to contextualize person-centered predictions so typically we build
social media based prediction models on the entire data set that we have however social media use and
language may tremendously vary across individuals now while this challenge can be overcome through
personalized models social media data also suffers from sparsity issues and therefore in this work we
balance the trade-offs between two personalized and two generalized models by contextualizing
on offline behaviors as captured by multimodal passive sensing so if you see the bottom figure
on the contextualized approach we build separate models of behaviorally similar individuals
to predict the psychological constructs these models can effectively capture between
individual homogeneity and within individual heterogeneity in our behaviors and my co-primary
author ted will take from here on and explain the methods results and implications of our work
thank you Koustuv in our work we collected data from 754 information workers
as part of the onboarding process participants were required to complete ground troop surveys
that gathered scores for various psychological construct measures
passive sensing data from these participants was collected continuously for a year-long period
offline and physical activity data was collected through a variety of different sensor modalities
and social media data from participants facebook accounts was also collected
from the passive data collection we generate a variety of different behavioral features
for the physical sensor data we compute features for the mean standard deviation and regularity
in daily behavior build our contextualized models we use these offline sensor features to cluster
participants based on differences in their offline behavior we tested three different types of
clustering algorithms and evaluated these clusters based on three different metrics of cluster fit
across all clustering fits tested we found that hierarchical clustering with three clusters was
most optimal we evaluate all modeling approaches using a five-fold cross-validation framework
and evaluate performance in terms of pooled pearson's correlation and smap across all training
folds we test several different machine learning algorithms and also withhold a separate testing
set of 50 participants to evaluate the robustness of any performance trends on unseen data
the figure here provides a summary of predictive performance results and the
differences in performance for generalized versus contextualized modeling approaches
we found that the contextualized approach yielded increased performance in predictions of
personality traits anxiety and sleep quality but decreased performance for cognitive ability these
patterns of results were also consistent for the held out testing data set the figure here presents
pooled pearson's correlation results comparing different modeling approaches including models
built solely from sensor data as well as sensor and sensors social media feature sets combined
the contextualized modeling approach shown as dark blue in this figure performed significantly better
than these alternative modeling approaches for most personality traits anxiety and sleep quality
overall there are a few key implications of our work that should help future research in this
domain first there are important trade-offs between generalized and contextualized models
more data in a generalized modeling approach may not always be best as contextualized models
built on smaller participants pools per cluster perform better for some constructs second our work
has implications for when to invest in building more personalized models contextualized modeling
approaches may be better suited for constructs that have a stronger theoretical association
with physical behavior finally contextualized modeling approaches may also allow for more
precise insights into multiple online and offline behaviors associated with psychological constructs
this concludes our presentation thank you for watching and feel
free to reach out to either Koustuv or myself with any questions you may have