Hi, everyone.
I'm Sabir, and this is Erica.
We're from Northeastern University.
So here we are to present our paper that I already
mentioned, so let's start.
So in this paper, we propose a new methodology
for player modeling, and we call it
Interactive Behavior Analytics Method or, in short, IBA.
This method facilitates knowledge discovery
from game data using human experts for context
to our analysis.
In this presentation, we'll first describe the methodology
and then demonstrate the use of the methodology using a case
study.
So now we move into the details of the idea methodology.
So here you can see a diagram that shows
the overall methodology of IBA.
On the high-level overview, this methodology
is comprised of two visualization
systems, a labeling mechanism, and some abstraction
algorithms, so the first step here
is to format the game data to fit
into a visualization system, which is called StratMapper
So StratMapper is a visualization system
and labeling mechanism for game data.
At the center of the application you
can see a two-dimensional game map.
Here it is shown for DotA.
You can zoom in and out.
Below it is a timeline which you can also
zoom in and out, visualizing all the events in the games.
A tooltip appears here with detailed information
up and hovering over the game icons.
The timeline allows users to select
a temporal period of interest.
You can move around to filter events and users on the game
map and display only those that appear during those selected
time period.
So in summary, this visualization
helps users to visualize the game as a whole
and to identify player activities and strategies.
So a StratMapper visualization is
used to label strategy-complex events
using the labeling mechanism, so these labels
are decided by human experts.
So now we come to the point how the labels are decided.
So developing labels are similar to developing codes
for qualitative thematic analysis.
First, the members of the research team
familiarizes themselves with the game to be labeled.
Once the researchers are familiar and comfortable
with the game space, they discuss the types
of strategies and actions taken during gameplay
to identify reoccurring patterns and themes within the game.
From there, they then identify a set
of labels for decision-making, strategic and tactical
behaviors.
So once the labels are identified,
team members can individually label
the game using StratMapper.
So as we have already mentioned, the StratMapper
has a labeling mechanism.
In this slide, we show our test label,
so this is done by selecting a particular time
period in the timeline, and muting irrelevant units
and events in the game, and then adding
the label names and other related information.
So here at this label appears, you can see.
So there we show two other labels--
that is Team Fight and Farm-- that we
have used in this research.
So after the labeling is done, the label data
is then processed and fed into another visual system
that is named Glyph.
So Glyph is a visualization system
that allows easy understanding as well as investigation
of player strategies.
So Glyph is comprised of three visual representations, a state
graph, a sequence graph, and a group graph.
In the state graph, these states are the game states,
and they are represented as nodes and the transition
as links.
In the sequence graph, each node represents a full sequence
of state actions for a player.
You can select the node, and then you
can see the sequences in the state graph
and the complete sequence of the states below the state graph.
Similarly, upon selecting a node in the group graph,
you can see the players in the group
and also the team behavior in the state graph.
So one thing to note here is that the distance
between the sequence nodes in the sequence graph
shows how similar the sequences are for the players.
So this Glyph is used to extract and analyze
the gameplay strategies, which we will discuss later.
Sometimes the resulting patterns may be incomprehensible,
or the labels may be incomplete.
And as a result, the team may discuss their interpretations
of labels, and they can revise the labels accordingly.
So as a result, this becomes an iterative process,
and that is the main idea behind the idea methodology.
So the next phase of our presentation
is actually focusing on the analysis
from Glyph visualization, and Erica will lead that.
So to demonstrate how IBA can be used to analyze and understand
game data, we're going to go over one of the case studies
we did that we talk about in the paper.
We talk about two case studies in the paper,
and we go into much more detail there.
But unfortunately, due to time, I'm
just going to give a very high-level example from one
of them, which used the game Defense of the Ancients
2 or DotA 2.
So just really quickly, for anyone
who's not familiar with this game,
you play it on this game map.
You have two teams, the Radiant and the Dire,
and they're trying to get to each other's base
and destroy the base.
And there's these towers which are represented
by these triangles that will fire at enemy players
if they get too close to them.
So following the IBA methodology,
two researchers examined forums, tutorials,
and gameplay commentary about DotA,
and they developed a list of 24 labels.
And that was simplified to nine labels
for the purposes of this paper, and this was that full list.
For the purposes of this presentation,
I'm only going to talk about two of these labels
in this demonstration, and that is Gank and Push.
So really quickly, again, a gank is
when you have two opponents playing
against each other in a lane, and the ally or the allies
of one of those opponents enters the lane
from the side to flank the enemy,
attack, and hopefully kill them.
And then a push is when a player or a group of players really
quickly moved down a lane very aggressively, usually taking
out towers, creeps, and any enemies
they find in their way to try to make
as much progress towards their goal as possible.
So these labels were applied in StratMapper,
using what you saw, and that was done to two randomly selected
games from a professional DotA league.
The labeling process took about 10 hours,
and we calculated an inter-rater reliability score of 0.72
with 78% agreement.
And that was done using Cohen's kappa.
After the games were labeled, the player sequences
were exported to Glyph, and we got this visualization,
which you've already seen.
So this is the labeled DotA 2 data displayed in Glyph
with each sequence in the sequence graph
representing an individual player.
No two players had the same sequence,
so they each got their own node in that graph.
So specifically, these are all the players
based on the team and the game, with the Radiant
on the left, the Dire or on the right, and then Game 1 on top,
and Game 2 on the bottom.
So one of the things you might notice
is that the sequences for Game 1 are
very different from each other, while the sequences for Game 2
are very similar, which you can tell
from how far and close they are from each other.
And this is because these games progressed really differently.
So Game 2 was really back and forth.
Both teams had time they spent in a winning and a losing
position, so in other words, that
means that the players in Game 2 both played offensively
and defensively at different points in the game,
regardless of which team they were on,
so they have very similar sequences
because they had very similar behaviors.
Game 1, however, was really one-sided.
The Dire team got a really early lead,
and the Radiant team never caught up.
And this resulted in notably different gameplay behaviors,
and then you can see that here their sequences are
much more different from each other than Game 2.
And this is the game we're going to look at a little bit more,
but just really quickly you might
notice that some of the sequences
are actually close together between the teams.
And those four sequences that you
see there are actually support players, so they're
playing heroes that have supporting skills.
And it was interesting to see that they played similarly,
despite the fact that their teams were either winning
or losing all game.
So we can see these behavioral differences
a bit more if we look at highlights
of the individual sequences.
So Batrider and Slark were the top-lane heroes
from the first game.
And if we look at the highlighters in Batrider's
sequence, we can see that he never transitioned to the push
state, and he also never transitioned from a gank
to a kill.
I know there's a link there, but I assure you
it's going the other way.
These indicate more passive and defensive gameplay.
Pushing is really risky.
You put yourself way out in enemy territory
and open yourself up for attack, so if you're
in a really disadvantages position,
you're probably not going to do it much.
And similarly, if you've fallen behind, killing
your opponent's probably going to be a lot more difficult.
So a gank can be risky, and even if you go for it,
you're probably not going to pursue it
to the point of trying to kill them no matter what.
So that's what we saw here.
Looking at Slark's sequence, we see more offensive behavior.
He has numerous transitions to the Push state,
and he successfully transitions from a gank to a kill.
And this offensive behavior implies
that Slark was in a more advantageous position
that he was able to take advantage of it
by just continuously threatening his opponent
with aggressive behavior, forcing
them to stay on the defensive.
So this illustrates that IBA can let
us see not only the gameplay context, such as who
is winning, but how the behaviors and the strategies
of players in those winning and losing positions
can change because of that context.
So wrapping up, IBA is a new methodology
that provides a generalizable context to where
means of analyzing game data.
It uses two visualization systems and a human-in-the-loop
to develop labels in context through an iterative process,
and this allows designers to better understand how
the dynamic environment of their game influences and impacts
strategic decision-making.
And, in turn, this could aid in catching bugs
during development or even refining gameplay
towards an intended design.
Thank you.
[APPLAUSE]
Hi, Eric Harpston, Carnegie Mellon University.
Super cool system.
I'm wondering to what extent--
so say I want to play this to a new game.
How much do I need to do?
Or how much can I just take your system, and it works?
Like do I just need that background map, and all
of the things go where they go?
Or how committed to DotA is it right now,
and how much would it need to change to afford a new game?
Yeah, thank you.
That's a great question.
So the basic part you need to do is
defining the labels for your game,
and then just format your game replays
into the format for StratMapper.
And the rest of their process are automotive.
Thank you.
And just to add, we only showed DotA in the presentation,
but in the paper we actually talk
about a second case study that's on a completely different game.
So it is pretty generalizable.
Hello.
[INAUDIBLE] University.
Thanks for a very cool presentation
and your meticulous work.
I'm really interested in knowing what
kinds of emergent strategies you can think up
from just tagging the game in terms of these different kinds
of actions.
Would there be more latent strategic choices
that could be mapped from the data that could be
even generalized across games?
Thank you.
Yeah, that's a great question.
That's sort of the big question we're
kind of exploring with this work,
and I definitely-- we've used this for two games,
and Glyph has been used for some more.
And we've definitely found that you can sort of find similar
high-level, general strategic ideas that
map across different things, so, yeah, that's
very much what we're exploring or at least a part of it.
Do you have more questions?
[INAUDIBLE]
Hi, Eric again.
I had a second question.
To what extent-- and this, again, might just be--
so just a quick clarification, the StratMapper and Glyph
are kind of two existing systems that
are pulling together in this?
Or is one of the new, one of them not?
That's just a quick guess, no.
So Glyph was a pre-existing, but we
built the StratMapper [INAUDIBLE] this.
So to what extent is this system committed to fundamentally
specialized games?
Which space is really important in most games,
but in some games you care more about a topology
than you necessarily do about like literal space,
and so is there a potential limitation
in the kinds of games this could be applied to?
Yeah, I think, at present, we are mostly
focused on two-dimensional games that have two-dimensional maps,
so, yeah, maybe we may explore more 3D
or if there is no map in a game, that kind of things.
Yeah, that we can look for in the future.
[INAUDIBLE]
We need to [INAUDIBLE] on your email [INAUDIBLE]..
Thank you.
Thanks.