Hi, I’m Samantha from the Augmented Human Lab
at the University of Auckland, New Zealand
I’m excited to present Prompto at Ubicomp 2021.
We most often face prospective memory lapses;
forgetting to perform intended actions in the future,
such as leaving an item behind.
We could address these lapses through memory training on mobile applications.
However, these applications mainly provide
time-based prompts to promote regular training.
A well-timed prompt may not be effective if
we are not cognitively available to engage
in the training.
In this work, we focus on providing prompts
when users are cognitively available.
We explore the notion that
when we are calm and relaxed,
we are potentially more receptive.
Prompto was designed to detect a low cognitive load state
through physiological signals of
electrodermal activity (EDA) and heart-rate variability (HRV).
This is estimated to be when EDA values
are low or near baseline
and when HRV is increased.
When this cognitive load state is inferred,
it sends a prompt which asks you to start
a memory training session.
In a training session, Prompto guides you to apply a memory technique to help you remember
an intended task better.
This technique is called implementation intentions or the `When-then’ technique and is known
to strengthen prospective memory.
It involves two steps. First, to formulate and verbalise a “when-then” sentence“
such as "When I leave home after breakfast tomorrow, then I will bring the documents.”
And second, to visualise yourself performing the action.
We demonstrate Prompto through this video.
We conducted a study on older adults’ receptivity to these prompts at low and high cognitive
load conditions.
Giving them 4 prompts per condition.
We mapped out their responses to the prompts that could either be starting practice of
the memory technique, asking it to try later or ignoring it.
We found that users started significantly more practice sessions in low cognitive load
compared to high cognitive load.
And they were significantly more responsive to prompts in low cognitive load compared
to high cognitive load.
We also conducted a field deployment with older adults who used Prompto over 7 days.
This was to understand if the benefits of prompting at a low cognitive load state could
still hold.
It was also to identify the challenges and requirements to prompt based on physiological
signals in-the-wild.
Users received at most 2 prompts a day that were at least 5 hours apart.
Although the response rate during deployment was lower than the response rate in the receptivity
study, users were still likely to respond to its prompts and to start practice sessions.
Most users appreciated the concept and found Prompto helpful.
​​In terms of the wider implications of the work, there are other cognitive functions
that rely on working memory and cognitive load such as learning.
We encourage more research on prompting based on cognitive states;
developing systems that determine if you are in the right state of mind before you start
an intensive task. We have shown a thorough exploration of what
is possible with and needed for sensing cognitive load states.
Imagine what more is possible in combination with other contexts such as time and mobile
usage activity.
There are several limitations and future works.
Different sensor placements and forms could be explored.
There are of course many other methods for cognitive load detection.
A noise filtering algorithm could be put in place to reduce motion artefacts.
And lastly, there are known effects of age on physiological readings and different physiological
readings for each person.
Machine learning models could be implemented for more robust detection.
To summarise, our main contributions are Prompto, a conversational system that measures physiological
signals and prompts for memory training sessions during low cognitive load;
A study on older adults’ receptivity showing that users were significantly more responsive
to prompts and engaged in more memory training sessions in low cognitive load compared to
high cognitive load;
An in-the-wild deployment revealing that most users appreciated the concept, found Prompto
helpful and were likely to respond to its prompts.
And here is my awesome team behind this project.
We are looking forward to future collaborations related to this project and other projects.
You can also check out the paper and our websites for more details.
Feel free to contact us for further discussions.
I hope you enjoyed this presentation. Thank you.