Hi everyone, my name is Tony Liu and today I'll be presenting our joint work
titled "Machine learning for relationship estimation: considering population
heterogeneity."
So why do we want to predict social
relationships? Well, these relationships -- communicating
with friends, family members, and so on -- are central to human life and
well-being, and disruptions in social relationships
have been associated with an increased risk
in mood disorders. In addition, the type of relationships individuals engage with
can be indicators of the amount of social support that they might receive.
So one of the goals of our study is to predict these social relationships from
passive phone data. In addition, we also explore
age variation and how this interacts with communication patterns within our sample.
So participants in our study installed a mobile phone sensing app on their
personal devices that collected both passive phone data and administered EMA
surveys. These surveys asked about the type of
contacts people are calling or texting as well as
as well as what we call semantic location.
This is taking locations that the app has detected participants being present
at throughout the day and giving them some sort of
semantic meaning. So what we have pictured here in the
screenshot is a marker of a movie theater that the participant visited,
for example, which is given the semantic label
of a place of entertainment.
So why might this semantic location information be useful in predicting
relationship roles? Well we thought that communication from
different contacts could be contextualized through this location
information. For example, a call from a bar might
provide a different signal to the relationship type than a call
from a grocery store. The call from a bar might be a friend
while the call from the grocery store might
more likely be a family member. So, visualizing this data what we have here
is four weeks of location logs for a representative participant.
We can see that there is sort of regularity in the locations that they
visit such as a home to work pattern that we would
typically see. We then combine this location data with our
communication events, with the dots representing the different kind of
contacts that this person is talking to or texting.
These contact labels are the target for our prediction,
relationship prediction task. We can see with this visualization that there are
some patterns emerging, such as the tendency for this participant to make
task related communications at home indicated by the blue dots and the
tendency for this participant to make friend-related communications at work
indicated by these pink dots.
So our final features for relationship prediction include demographics, such as
age and gender, communication information, such as
texting frequency and call duration, as well as the location information that
we just discussed. So let's look at some results. Throughout
the study we used autoML which automates both hyperparameter tuning and ensemble
selection. We run autoML across different
combinations of all of our feature sets that we just discussed,
and one key takeaway here is that including location data
improves the performance of our models. We see that models trained with location
information alongside the other feature blocks gives us the highest F1
scores.
Another takeaway here is that age and
gender potentially interacts with communication.
The model that's trained only with age and gender doesn't actually do any
better than baseline, but age and gender when combined with
communication features give us a performance
boost over the model that just uses communication features alone.
This interaction between demographics and communication led us to explore
age heterogeneity in our sample.
What we found is that as age increases, there is a general preference for
calling over texting as well as a general decrease
in late night communication. Though these findings don't really seem
that surprising, what's interesting is that there are differences across
relationship roles when we look at age and communication together. So what we
have pictured here is late night texting frequency on the y-axis,
participant age on the x-axis. And though the overall trend is that older
individuals text less frequently at night, as you would expect,
this trend varies depending on the type of contact.
We see on the left hand plot that texts to family members are actually
relatively stable within our sample beyond the age of 30
whereas texts to task related contacts on the right
falls off quite severely.
This again shows that there is heterogeneity in
communication across both age and contact types within our sample
population, something to keep in mind for us when
we were building our predictive models. In fact, when you look at our paper we
performed another prediction task where we train models on subsets of our
population split by age to see how well they generalize
to other age groups. And what we found with this exercise is that models
trained on younger individuals generalized particularly poorly to older
individuals.
So what are the takeaways from our work? The first is that contextualizing
communication with location data improves performance. And the second
broader takeaway is that phone sensing studies should be validated on more
diverse samples. Here we have examined how age interacts
with communication, but there are other sources of variation
in these studies that need to be considered as well,
such as employment status, baseline mental wellness, and so on.
We know that a lot of these studies are limited in participant recruitment for
practical reasons, and in fact our group does a lot of
convenience sampling as well, but it's particularly important to keep
in mind that building predictive models on these more uniform samples
doesn't necessarily translate to generalizability to the wider population.
Thank you for your time, and I'm looking forward to any questions and discussion.