Hi, I am Shaghayegh Roohi, presenting on behalf of
a team of Aalto university and Rovio entertainment researchers.
This paper is about predicting game level difficulty and churn.
We present a model that is able to do that without players.
We do this by testing the game using AI players
and predicting human pass and churn rates from AI data.
The AI model is tested and validated on
a dataset of over 90 thousand players and over 100 game levels.
In an interactive game it's impossible to predict all possible behaviors in advance.
For example, players may invent new uses for game objects.
In rocket jumping, for instance, players use the recoil of a weapon for locomotion.
This unpredictability leads to an iterative process of prototyping and testing.
Using human playtesters to test a game is time and money consuming,
which is why play testing using AI players is an active research topic.
This paper utilizes a data set of pass and churn rates from Angry Birds Dream Blast.
This data can be easily collected from all players.
Pass rate is a measure of difficulty in the range of zero to one.
Zero means that players never pass a level
and one means all players pass the level on first try.
Churn rate is a measure of engagement, again in range of zero to one.
At the first glance, the data shows that pass rate is a predictor of churn rate, but
the relation is non-linear and changes dynamically over the levels.
Our paper proposes a simulation model that explains this.
Several researchers have applied AI agents at their games
to estimate the difficulty of game levels or detect design flaws.
They usually exploit human data or heuristics to create human-like agents.
However, we concentrate on deep reinforcement learning
that doesn't need any hand-coded heuristics or human player data.
Some work predicts an individual player's churn probability based on their in-game data.
We focus on the prediction of average churn ratio per game level,
not the probability that a particular player churns.
Previous work has also investigated the relation of game challenge and engagement.
For example, flow theory states that the balance between
challenge and skill needs to be optimized.
On the other hand, Lomas found out in their educational game,
lower challenge leads to higher player engagement.
We model the relation between level pass and churn rates by combining
Deep reinforcement learning game playing agents,
and a simulation of how the player population evolves over the game levels,
based on simulated skill, persistence and boredom.
Previous work has not investigated the combination of these two parts.
Combining these components, our simulations produce human-like data,
where the relation of pass and churn rates changes correctly over game levels.
Our game-playing AI uses a standard deep reinforcement learning agent
that is a neural network that outputs game actions
based on the current observed game state.
The agent interacts with the game through actions
in this case determining where to tap.
this causes the agent to receive an
observation of the resulting game state and a reward.
The observation is fed through the neural network to determine the next action,
and the learning process optimizes the neural network
to output actions that yield high rewards.
This is the trained neural network playing the game.
We compare two models. In the baseline model,
pass and churn rates are predicted through simple linear regression models,
and AI gameplay statistics are used as the input features.
Not surprisingly this doesn't yield good results.
However, we show that one can take the simple baseline password predictor,
Normalize the predictions over all the levels to get a level difficulty measure
and then inject this to a simple population simulation block
that determines which player churns.
This then produces both more accurate churn and pass rate predictions,
and the population for the next level.
The process is then repeated for the next levels.
The player population simulation implements
simple computational models of individual player differences.
Each player in the population is defined as a tuple of attributes
that determine the player's behavior based on simple rules.
First, a player can pass the level if
player's skill level is higher than the level difficulty.
If the number of player's attempts to pass a level
is higher than player's persistence, the player will churn.
This way we can model how less persistent players are likely to churn early.
Even if a player passes a level, they may get bored and churn with some probability.
This models how game engagement tends to decline over time.
Furthermore players can learn from their attempts to pass a level.
Full details are provided in the paper.
The population simulation is straightforward to implement as code.
Numerical results on 5 folds cross validation show that our model acts
similarly to the baseline model in term of pass rate prediction.
However, our extended model improves the churn rate prediction of the baseline model,
with an effect size of about 2 standard deviations.
In summary we propose a novel simulation approach
for predicting churn and pass rates of game levels.
At first, AI gameplay statistics are collected from each game level.
The DRL agent only needs to be trained once per level.
Level difficulties are estimated using these statistics.
Based on the estimated level difficulties,
the evolution of player population over levels are simulated.
Our approach allows preliminary testing and validation of level designs without real players.
Naturally the model needs to be updated with real data
whenever new levels are released to players.
Thank you for your attention.