The case study I will introduce is about monitoring cognitive performance with
a serious game: a longitudinal case study on online cognitive assessment.
My name is Jacqueline Urakami and I am from the Tokyo Institute of Technology. This study was
conducted in cooperation with You Zhi Hu and Mark Chignell from the University of Toronto.
Cognitive functions are related to the ability to maintain independence and to perform activities
of daily living. A rapidly aging society and with this an increase in diseases associated
with cognitive deficits placing a growing demand on health and long-term care providers
that make it necessary to continuously monitor cognitive performance.
We developed a serious game for valid and reliable assessment of cognitive performance
based on the Whack-a-mole game. Serious games are games that have an underlying purpose
other than entertainment and so far there is missing evidence from long-term studies especially
in the effectiveness of using such games.
The research questions addressed in our study were: RQ1 Will players be willing to play
our whack-a-mole game repeatedly over the course of five weeks and continue to enjoy
the game? RQ2: Can our serious game be used for repeated measurement of cognitive performance
carried out online by the user and without facilitation? and RQ3: What challenges arise
from conducting a study with an international research team working from different continents?
Nineteen students played whack-a-mole games online over the course of five weeks. The rules of each
whack-a-mole game changed slightly depending on the cognitive performance that was measured.
The study started with a one week preparation phase where participants received written
instructions and could try the games in a demo version. Data collection took place over the
course of four weeks where participants had to play all seven games once a week. Online
meetings were scheduled so participants could ask questions and report problems. Participants had
to prepare a short report at the end of the data collection phase to reflect on their experience.
Only 8 out of 19 participants completed all games and 7 participants completed at least
3 sets of games. We noticed server end problems especially when many participants played the game
at the same time. Also throughout this study a variety of user end problems occurred such as
technical problems with sending data to the server, loading moles for the game or connection
problems. Some participants mentioned that they had problems understanding the instructions
and felt bored with the game interaction. We also noticed a lack of motivation and enjoyment
especially throughout the course of the four weeks of data collection.
Answering our research questions we found a gradual decrease of motivation and acceptance and
belief that incentives and gamification features are necessary to motivate players. Overall,
we found a fairly good compliance with playing the game online, but adding video instructions might
support users better than giving written instructions. Also providing offline versions
of the game might help to avoid some of the user end problems our participants experienced. When
it comes to working as an international research team we felt that it's really important to have
regular contact with participants, react quickly to complaints and requests and to have regular and
frequent communication between all participating parties to keep everybody up to date.
Based on our experience I would like to highlight three problems that might be relevant for others
as well who conduct long-term studies online. First of all, avoiding dropout by giving regular
performance feedback. Adding incentives and providing personal monitoring to participants to
give them the opportunity to report on problems. Second, dealing with technical problems by
giving quick feedback, checking individual user settings, and providing alternative solutions
such as downloadable versions of the game. And third, avoiding player fatigue by adding
variations in interaction methods, diversifying game-play and incorporating learning opportunities
to keep playing the game interesting and keep players motivated to continue over the long term.
For future studies we want to add an emotional component such
as fearful faces. The Braintagger games are freely available for research and clinical use.