Hello everyone, my name is Alicja and today I’d like to tell you about
a collaborative project between Scaled Insights, a Behavioural AI start-up, the University of Leeds, and Health Education
England (an organisation that is responsible for delivering training to National Health Service staff in the UK).
In this talk I’ll cover why we need AI-enabled solutions for continuous medical education,
why motivation is a key factor in that, and how we believe personalization can offer an effective
and scalable solution.
Lifelong learning is a requirement in many professions. For example: engineers, lawyers, accountants,
and many more. It also applies to all health and care professionals whose
continuous professional development can literally save lives.
We hear a lot about how technology could revolutionise healthcare. But that technical knowledge and
skills has to be acquired. And that digital disruption can affect every single health
and care worker. This has been argued in an expert report called the Topol Review, which
recommends that in order to make the best gains from the opportunities that technology
offers, upskilling is needed. But this is not easy. The challenge facing
Health Education England is that you not only need to motivate health and care professionals
(who are extremely busy at the best of times), but you also need to do it at scale, since NHS counts
over 1 million staff, I believe making this HEE one of the biggest education and
training providers in the world.
HEE already offers an extensive range of e-learning resources via their platform e-Learning for
Healthcare. Since the hard work of creating content
has already been accomplished by HEE, we believe that to better motivate the NHS workforce
to complete e-learning modules we can use personalisation and adaptation on top of the
existing platform. In that sense, personalisation really is the cherry on the top that
makes the e-learning provision more effective.
And while personalisation and adaptation in learning environments can apply to very different
things, for now we are focusing on motivation, as we believe this is the key to encouraging health and care workers to
access e-learning resources. But how do we do that?
We focus on personalizing motivational messages, or nudges. A nudge is an intervention which
aims to steer the user towards a certain behaviour without restricting their choice. Examples
of nudges are a speed sign, or an app notification. They encourage you towards a certain behaviour
(for example, reducing speed or checking your inbox), but they don’t force you to carry out that action.
We believe this is crucial in the lifelong learning setting.
Another requirement is that the nudges should be a ‘cherry on the top’ and easily added
to the existing HEE platform. In our experiments, the nudges are presented together with a description
of a e-learning module using wireframes of the current system.
But before we do any personalisation we need to know several things: which nudges to use to be effective,
and how user characteristics might interact with the effectiveness of nudges. To explore
these questions we conducted a study with medical and healthcare students, where we
considered several different nudges and three user characteristics (interest, demographics,
and personality).
You will find the details of the data collection in our paper. To briefly summarise, we recruited 107
students in Medicine or Healthcare to complete an online survey. Most
of the participants were quite young and studying Medicine, and most of them were also frequent users of technology
for learning.
I will now briefly summarise the study design.
Within the survey we asked whether a participant would choose a particular e-learning resource.
We picked six resources on the same broad topic (child and adolescent mental health)
and we added nudges to five of them. I will show you the resources and the nudges shortly.
For each resource we collected a score of likeliness to click and asked
for an explanation for the participant’s choice.
We also investigated whether these learner characteristics mediated the likeliness to
click. Demographics questions and MindTime questionnaire were included at the beginning
of the survey and we asked about user interest with each presented resource.
These are the wireframes for the six resources we used in the study.
Each resource is presented using: a title, placement in topical taxonomy and rating (which was
kept constant for all resources), a short text description, an image, and for five resources a textual nudge.
We used a taxonomy of choice architecture techniques to select five nudges which were
applicable and implementable in this scenario. The paper summaries each nudge. To explain them quickly:
the Feedback nudge gives the learner an idea about their progress, the External
Information nudge provides a statistic relevant to the topic, the Opinion Leader nudge shows
logos of organisations which have endorsed this particular resource, the Benefit nudge highlights points that can
be awarded and being able to expand the learner’s professional portfolio, finally the Self-commitment nudge
allows the learner to save the module for later.
I will now summarise our findings. Our study showed that not all nudges are equal.
Even without any personalisation, we found that the Opinion Leader nudge was most effective.
This is useful to know when we have a new user in the system
and no user model available, or when the user does not consent to personalised content.
In this case we can just use the most effective nudge as a default.
We also found that among the three user characteristics, interest was by far most closely linked with
likeliness to pick a resource. This suggests that using a content-based recommender system
could encourage NHS staff to access modules relevant to their interests. However, demographics
and personality still played a role, although to a lesser degree. We believe that these
characteristics would be especially useful for personalisation when user interest is
not applicable. For example, most health and care workers should know the basics about
preventing and recognizing sepsis. In that scenario, interest doesn’t really play a role.
Instead, personalisation could be driven by personality or demographics.
Finally, a qualitative analysis of the feedback on each resource revealed that the HOW really
matters. Participants pointed out that they are more likely to disengage if the presentation
is incongruent. For example, a silly picture which doesn’t match the topic of the resource. What seemed
to drive engagement was informative descriptions and eliciting emotion (for example, by using a picture
of a child with whom you’d empathise). Finally, as with many online environments in the era
of information overload, keeping the presentation simple was highlighted as positive as well.
These are the lessons we have learnt in the current study. In the
near future we will conduct a similar study with NHS staff at a large hospital in the UK. We are
very interested to see which of these patterns will repeat, and which might possibly differ.
I would like to finish this talk with a motivational message, personalised to the HAAPIE workshop
crowd. During this pandemic the e-Learning of Healthcare
website has seen a huge amount of traffic. Thanks to a huge effort by HEE, they were able to quickly
release an e-learning module on Covid, which has been accessed over 1.5 million times to
date. Even outside of healthcare, a lot of learning has been forced to move online.
With this massive shift to e-learning, perhaps now is the time to contribute (whether it's in healthcare
education and others) and showcase the promise of personalisation for learner motivation.
Thank you and I’m happy to take questions.