This work is a systematic review of affective gaming
I'm going to go somewhat fast through the slides
So feel free to pause or rewind if you miss something
So what is affective gaming?
Affective games are those which use physiological signals to influence the gameplay experience
We looked at a variety of both direct and indirect physiological signals as part of the review
Direct physiological signals are ones which can be directly manipulated by a person
So for example, I can breathe and make a particular facial expression on command
Which is why they are categorized as direct
And indirect signals are ones which are mediated by some other interaction
For example, to make myself sweat more I need to do something else to make that happen
Like jumping jacks
Or to lower my heart rate I might take a few deep breaths to calm myself down
That in turn lowers my heart rate
The original research question was to understand if, how, and why
Using these physiological signals in games
actually enhances player experience
However, after conducting the review
it turned out there was still a significant gap between research and evaluation
so the paper ended up giving an overview to newcomers in the field
by creating a way to categorize and group the literature
and to understand how and why certain signals and sensors are being used more than others
We followed the PRISMA guidelines on conducting systematic reviews
You can read the methods section of the paper for a more detailed account
We designed the affective game loop to classify the ways
in which physiology can be mapped to in-game variables
First, you have a player
The player uses some form of physiological device as input to the game
Could be a sensor tracking the player's heart rate, or facial expressions via webcam
Then this is fed into the game
The game then processes this sensor input
maps it to some in-game variable, and feeds it back to the player
So this is the way physiology can be *integrated* into the game experience
But we still need a way to describe a simple overlay of the players' physiology
Over the game without affecting it in any way
We call this physiological projection
which means the physiological signal is simply projected back to the player
without entering the loop and affecting the game in any way.
Now let's zoom in and understand the various ways in which
the physiological signals can be mapped to in-game variables.
So we can either have an input mapping that is mechanical, environmental, difficulty
or some combination of these three.
A mechanical mapping is where the input is used to control game actions or mechanics
Usually by the player-character
Environmental mappings are where the input it used to affect mood or context of the game
like audio/visual effects, lighting, or weather
Difficulty mappings are those where the input is used to influence the difficulty of the game
which might impact the non-player characters' abilities
player abilities, or background contexts specifically relating to game difficulty.
Mechanical mappings we classify as action-based
And difficulty/environmental are context.
So now let's bring it back to the affective game loop
We can have indirect and/or directly controlled physiological signals
which the player uses as input to the game
The game then takes this input and maps it to in-game variables
which can either be action or context-based
and then this feedback is sent to the player.
Physiological projection can be mapped to any signal in the form of visual, audio, or haptic feedback
An example of this might be an audio signal of the player's beating heart in the background of the game
So here's a simple example
This game is called "Breath"
Essentially, targets are placed along a curve on the screen
and the player needs to blow into a breathing apparatus to move the bird up/down
through inhaling and exhaling, to collect the targets.
So for this loop we have the player using a spirometer
which is a direct physiological input
That is then fed into the game
where it is mapped to the up/down movement of the bird
which is a mechanical, or action-based mapping.
In this graph you can see each of the physiological signals
and how often they were used as a particular input mapping in the literature
So for example heart activity was most often used to affect difficulty in games
whereas breathing and facial expressions were mapped most often to game mechanics
like player speed or movement.
The trend we see with this graph
is that more direct signals like breathing and facial expressions
are paired with more mechanical in-game mappings
whereas more indirect ones like sweat and heart rate
are most often paired with difficulty and environmental mappings.
Taking a step back we will look at the papers' usage of these signals
irrespective of the affective game loop categories
This graph shows the raw number of systems which use each signal.
We can see that heart rate was the most frequently used
This might be because heart rate monitors are cheaper
or more widely available than other sensors
or maybe because the readability of these signals to the average player
is higher than another indirect signal like sweat
The purpose of the contribution included anything from increasing player engagement
to helping people with a specific issue
like therapy games, or serious games used to help with rehab.
Half of papers were looking at general enjoyment
The mental health domain seemed quite promising
As papers were generally focused on desgining a game to help children with ADHD focus
Increasing empathy in children with Autism Spectrum Disorder
or people with anxiety, to help them decrease their stress.
Generally, the papers achieved the goal they set out to achieve
Only 10% of studies did not achieve their desired outcome.
To summarize, we made the affective game loop
to help contextualize the field.
The loop might be useful for newcomers to the field to understand
which domain their work falls in, devices others have used
and how others map the physiological signals they use to in-game variables
Another purpose is to link up parts of the field that might not be communicating.
So for example those in the mental health domain
publish at separate venues from those in the general engagement one
and this systematic review will help each field understand what work
is being published in the other.
Generally, researchers who have explored the topic
have found success in using physiological input to enhance the player experience.
However, formal evaluations are rare
Suggesting that the field is still in its infancy.
For the field to advance, it might be useful to have some gold standard physiological devices
with built-in integration to game making platforms like Unity
with customizability for particular purposes offered as well
in order to speed up the initial making and testing of these devices.
This paper is the first step to facilitate this,
providing a launching point for researchers in this field
to understand what has been done before
To get a sense of how their work fits into the existing ecosystem of affective gaming.