Hello! My name is Varun Mishra, and today I will be presenting our work on receptivity
for mHealth interventions. This work was done in collaboration with Florian Kunzler, and
colleagues at Dartmouth, ETH Zurich, and University of St. Gallen.
Recently, there has been a lot of focus and interest in leveraging ubiquitous technologies
to deliver health and well-being interventions. One intervention framework that is particularly
popular is Just-in-time adaptive interventions. JITAIs have been deployed in several studies,
and have shown to be effective for a variety of health outcomes. Fundamentally, JITAI aims
to deliver the right type or amount of support, at the right time.
So what is a “right time?” There are two key concepts when it comes to timing. First
is a state-of-vulnerability, i.e., period of heightened susceptibility to a negative
health outcomes. For example, if the goal is to provide stress-relieving interventions,
then, ideally, the interventions should be delivered when the user is stressed or is
about to get stressed. Here, stress is the vulnerable condition. Delivering stress-relieving
interventions at a time when the user is relaxed might not be beneficial.
The next concept is state-of-receptivity, i.e., periods when users are able and willing
to receive, process, and utilize the support provided. For example, Delivering a stress-relieving
intervention when a person is giving a talk might not be a good time, even if the person
is stressed.
Hence, detecting -- both -- vulnerability and receptivity is important for an intervention’s
effectiveness. In our work, we focus on the receptivity component.
The goal of our work was to evaluate how various factors associated with receptivity to interventions.
We started with intrinsic factors, i.e., information that was expected to stay constant over a
period of time, like age or personality
Next, we evaluated how contextual factors, like location, activity, or device use, related
with receptivity.
We also evaluated the relationship between receptivity and intervention effectiveness.
I.e., did being more receptive lead to better goal completion, and vice-versa.
Finally we evaluated the feasibility of building machine learning models that could leverage
the various factors and detect receptive moments.
So, we conducted a 6-week long study with 189 participants in switzerland, where we
deployed a chatbot-based digital coach -- called Ally -- that provided physical activity interventions
aimed to improve daily step goals. We built ios and android apps to deliver the interventions
and to passively collect sensor data.
Before starting the analysis, however, we had to first operationalize receptivity.
Ally initiated each conversation with a “greeting” message, like Good Morning, or How is it going?
We defined several metrics to measure “receptivity” around these initiating messages.
First, is just-in-time response, i.e., did the participant reply to Ally within a short
period of time. In our analysis, we used a window of 10 minutes.
Next, is response delay, i.e., the time taken to respond to the initiating message.
Third, is conversation engagement, i.e., did the participant engage in a conversation with
Ally.
We conducted very detailed analyses and gained several interesting insights. While we list
some key points in this talk, we encourage you to read the paper for more thorough discussions
and comparison across the different dimensions of receptivity.
For intrinsic factors, we found that in-general older people were more receptive to the interventions.
We also found that several personality traits showed associations with receptivity metrics.
One interesting example was that neurotic individuals were more likely to have a higher
just-in-time response rate, but it didn’t translate to higher conversation engagement.
In the paper we discuss the potential implications of these findings on intervention design.
Next, we found that several contextual factors like activity type, location type, and device
interaction, showed significant associations with receptivity.
For example, people were more likely to be receptive when they were on foot. Participants
were less likely to be receptive in locations other than their home or work; or that they
took longer to respond when at work.
Next, we found some interesting associations between receptivity and intervention outcome.
We found that if participants were more receptive in a day they were more likely to achieve
their step goals by the end of the day.
Similarly, if participants completing their step goals on a particular day, they were
more likely to be receptivity the next day. Thus suggesting a cyclic relationship between
goal completion and receptivity. In the paper, we provide an in-depth discussion about the
importance of this finding.
Finally, we build some machine learning models to detect receptivity, and found that even
with a fairly simple classifier with minimal feature engineering we achieved improvements
in F1-scores of up to 72% when compared to a biased random classifier.
Our results are very promising, and show that it is indeed possible to detect receptive
moments for intervention delivery.
As next steps, we are currently deploying these detection models in other studies to
see if the models are effective in the real-world.
We are also building better models that move beyond just momentary features and consider
more temporal information.
Thank you for listening, and please feel free to reach out to us for more details.