Hi, I'm Tommy Thompson and this is AI and Games: a series on research and applications
of artificial intelligence in video games.
Over the last couple of years of making these videos, I've looked at a variety of video
games that have proven useful testbeds for AI research, with the likes of Ms. Pac-Man,
Super Mario Bros. and more recently StarCraft.
But in this video I want to look at a genre that is still relatively new whilst presenting
exciting opportunities for AI research: multiplayer online battle arena's.
The MOBA genre is undoubtedly one of the most popular in gaming today, but what does it
any of this mean for AI research?
I'm going to give an overview of MOBA's as a genre, what aspects of their design can
prove interesting to AI research and look at some projects that are now bearing fruit
both in academia and in corporate research labs.
Alright, let's start with me butchering my explanation of what MOBA's are and why they're
proving so interesting.
Multiplayer online battle arena's are an offshoot of real time strategy games, originating with
the Aeon of Strife map for Blizzards StarCraft, followed by the Defence of the Ancients mod
for WarCraft III: Reign of Chaos and its subsequent expanions the Frozen Throne.
In MOBA's players take charge of a single character or hero as part of a team.
Teams of players are taxed with the destruction of the opposing teams structures, with the
critical structure found in the centre of their base on one side of the map.
Maps in MOBA's are typically broken up into lanes, with structures such as towers or turrets
for each team to defend their own and their opponents.
Players must work to destroy these structures alongside not only fellow players, but simple
AI-controlled non-player characters - often referred to as creeps - that continually spawn
from each teams base and move forward often unwittingly to their eventual demise.
What makes this interesting and challenging for both human and AI player alike is the
need for management of resources, control of our chosen hero unit, alongside consideration
of the macro and micro layers of strategy.
This is in contrast to RTS games that typically remove absolute control at the micro-level
given that AI often handles the behaviour of individual units for a given task.
This allows for players to focus on the macro or higher level strategies in play.
However in MOBA's, player need to manage the control of their own hero unit, but also consider
how their actions will influence the strategies in play at multiple levels of abstraction.
This requires teamwork and communication between players, whilst also ensuring we effectively
manage our hero's skill tree, and resources through use of items.
MOBA's are rather strict in their map design and general formula, with maps often adhering
to the three lane structure.
The variety of heroes on offer are what govern the larger meta strategy of the game.
Despite this, there is still sufficient breadth in this design for numerous games in the genre
to emerge.
While MOBA's are typified by two spiritual successors of Defence of the Ancients, namely
DotA2 by Valve and Riot's League of Legends, there are host of other interpretations of
this space such as the likes of SMITE by Hi-Rez Studios, Blizzard's own Heroes of the Storm,
Giganti...
no wait, um...
Battlebo... oh...., Infinite Cr... nope, Paragon....?
Nah.
Y'know what, nevermind, let's talk MOBA research!
But why all the fuss?
Well MOBA games are veeeeeeery popular these days.
They've effectively pushed RTS games and other popular genres aside to become the largest
eSport in the world in terms of prize money awarded.
Everyone is in on this jazz, betting companies, sponsorship deals and when you're looking
to get your hands on say $20million prize pools at the invitationals, hell, I would
even give it a stab where it not for the fact I'm in my 30's, which by MOBA standards means
I've been dead for almost ten years.
So where there's a lot of money, there's a need to provide a great experience for players,
spectators, commentators and more.
There are a variety of ways in which AI can potentially impact areas of MOBA development,
consumption and engagement.
Whilst much of this is still more fantasy than reality there are some very real problems
that AI could be tackling, this includes:
Hero Drafting: where AI could recommend the selection or blocking of specific characters
in a tournament context.
Commentary for different parts of the match.
This is an incredibly demanding and taxing problem to explore and comes in a variety
of forms, such as analysis of the hero draft phase as well as building up the key plays
of the match.
This last part is incredibly taxing, given this requires recognition of what's happening
at the point in the game, isolating the key play that occurs, managing the camera for
maximum dramatic effect and potentially even hypecasting the whole event as it occurs.
For some context, let's check this moment from the 2017 international and note how in-depth
knowledge of the game meta, managing the camera and how context and showmanship from the commentators
elevates this into an epic moment for the crowds.
Much of
this comes down to human expert-knowledge from match commentators such as Toby 'TobiWan'
Dawson here as well as teams of people filtering the footage, in real-time and by hand.
That's not an easy task!
AI could very well look into speeding up and assisting in this process.
Outside of commentary, AI tools can still be used for pre and post-match analysis for
teams at a competitive level: examining their movement patterns, their overall strategy
with hero selection during play as well as item usage such as wards which temporarily
expose parts of the map hidden by the fog of war.
But of course in addition to all of this is the continued development of the games themselves.
League of Legends and DOTA2 go through an ongoing process of balancing characters and
items within the meta of the game.
There is potential for AI tools to be able to establish the major changes at strategic
level of a given patch, to isolate exploits that need to be patched out and potentially
predict changes that will be made to the game based on current in-game performances.
On top of this is managing the communities that play and how well they interact with
one another.
MOBAs sadly have a poor reputation for their community and toxic behaviours, but what I
AI can catch this as it is happening - an issue already being tackled by the likes of
SpiritAI and their text message monitoring platform Ally.
Filtering language and reporting players automatically based on their in-game behaviour.
Many of these issues are relevant to other games and even physical sports such as football
and basketball, but MOBAs are a perfect platform for this given they provide one thing that
many machine-learning driven AI systems need: data.
DOTA2 and League of Legends provide complete replay data of matches and there are a significant
number of tools and communities out there who write parsing tools to enable hoobyists
and researchers to parse that data for their own interests.
And there's plenty of data to go around: with over 1000 years of DOTA gameplay occurring
daily, with an estimated 430,000 years worth of games played by April of 2014 according
to
So while this research field is still relatively new, I'm going to walk through some recent
work in the last couple years and give you a flavour of just how far it's come, but also
how much more work is to be done.
First up, hero drafting: the process of selecting and potentially blocking heroes for the opposing
team.
Can we predict the hero a team will select?
That'd be useful not just for match commentary but also in-game strategy.
Early research in this area arose courtesy of Adam Summerville and Ben Steenhuisen from
UC Santa Cruz and DatDota respectively working alongside Michael Cook - whose work has previously
been highlighted on AI and Games with his ANGELINA system.
This project looked at using a stack of DOTA2 matches from the DatDOTA site around the time
of the Frankfurt Major in 2015 and applied Bayes Nets and a LSTM neural network and trained
them against the draft phase, isolating choice number, team, choice type and preceding choices.
The resulting Bayes Net wasn't terribly accurate, but the LSTM wasn't far off the actual commentators
comments made during those matches.
Next up, player performance!
What can we learn about players from their gameplay and how can this be put to good use?
There's a variety of research emerging in this field that utilise the reply data to
full effect.
One notable example is work by Rohit Bhattacharya and Azwad Sabik of John Hopkins University,
who developed a prototype player-personalised recommendation system for hero and playstyle
recommendations in 2015.
This work analysed 500 matches in top-tier competition and clustered data of winning
teams using Principle Component Analysis, Locally Linear Embedding and Affinity Propagation.
The resulting system can look at a snapshot of the current state of the game and recommend
the most successful heroes for that period of game, as well as give advice on goals and
objectives players should focus on, all of which based on expert gameplay data.
Meanwhile, a team of researchers led by Anders Drachen of the University of York in the UK,
began formative research in analysing zone changes and distribution of team members in
matches of DOTA2 using a process known as spatio temporal analysis.
The resulting system, once again using a form of time series clustering over 196 matches
across four skill tiers, enabled the research to highlight significant differences in strategic
play between expert and novice players.
This highlighted how expert players movement patterns and behaviour are not only more refined
than novices but have unique identifiers during early-mid and mid-late game.
Moving into esport analytics, Anders Drachen appears again alongside Matthias Scubert of
Ludwig-Maximilians University in Munich and Tobias Mahlmann of Lund University in Sweden.
This research looks at building the foundations for future esport analytics through encounter
analysis: meaning reading the in-game data to isolate periods of play where multiple
heroes from opposing teams are in range to affect one another.
With a decisive encounter established as one where heroes get killed.
Using the spatio temporal analysis process, this system assessed over 400 DOTA2 replays
to establish a variety of different encounters, breakdown where encounters largely occur throughout
a match even begin to predict success in encounters based on other in-game data and previous performances.
Alongside this is work by Pu Yang, Brent Harrison and David Roberts at North Carolina State
university who published work in 2014 on identifying combat patterns in DOTA2.
These patterns are built from graph models of in-game behaviour during specific windows
of play, the interactions that take place and their outcomes.
Through use of rule-based classifiers, a series of rules and strategies are formulated by
the system with a confidence rating of whether they should be employed at a given point.
Whilst still far from complete, analysis against new matches outside of the training set proved
to be valid and accurate in over 80% of test cases.
This is far from an exhaustive overivew, but it highlights that there's a lot of work happening
in this field and it's been going for a few years now.
Outside of these topics, there has also been in areas such as item-analysis and in-game
economies, dynamic difficulty adjustment for solo players, online ranking algorithms for
ensuring players don't matchmake with dickheads and of course, writing bots that can actually
play MOBAs!
Now I deliberately left this until last, but arguably the most well known research happening
in MOBA's right now is in creating bots that can play the game themselves!
While there is work in this area happening in academia, the biggest and most notable
to-date is from the corporate world.
Headlines were made back in August 2017 as a new AI bot was paraded in front of fans
and professionals at The International: the biggest DOTA2 event in the world - hosted
by game developer Valve - and acts as the closing event of the annual Dota pro circuit.
The bot itself was developed by Open AI: a non-profit company co-founded by AI scaremonger,
elitist libertarian, modern-day sensationalist slave-driver and all-round asshat Elon Musk.
At the time of this video Open AI is still tight lipped on how the bot works.
But it is reliant on a custom interface based on the public DOTA2 Bot API that reflects
human observations and in-game actions and a configuration that allows it to use machine
learning techniques and gradually learn the game through self-play.
This is hosted on a cloud system to enable massive processing power as well as speed
it up even further by running the game almost entirely on GPU's.
The learning bot is also capable of being coached by being fed good strategies either
for specific circumstances or to bootstrap base behaviours.
The bot was gradually trained and improved over a period of 5 months in preparation for
the International, with the bot's performance gradually improving over time.
In early June it still struggled to beat players with a DOTA2 Matchmaking rating or MMR of
1.5K (which is equivalent to less than 15% of all players) to ultimately defeating professional
players such Arteezy (MMR of 10K), Sumail (8.3K) and former-world champion Dendi (7.3K),
with this battle being streamed across the internet for the world to see.
Is this the future of AI research?
Is this a significant technical achievement?
Yes... and no.
Firstly, we need to remember DOTA2 like any MOBAs is a team game - 5v5 in this instance.
Being capable of reacting to, anticipating and outperforming a human in 1v1 capacity
- even against professional players - was an inevitability.
Training machine learning algorithms to work in reactive and dynamics contexts such as
1v1 combat in a MOBA is a relatively attainable goal.
Don't get me wrong, it's not easy by any stretch and full credit to the team for constructing
it, but this feels more like a PR stunt that it is a significant breakthrough for AI.
Call me bullish but I'm not all that impressed just yet.
Open AI hosted a LAN event for other pro players at the International to take their shot at
the bot.
They have since conceded it was defeated, but failed to give actual numbers.
Despite this, over 50 players have come forward detailing their experiences.
Like any Machine Learning bot, you can't insure it is infallible.
As such, players found a handful of strategies the bot couldn't anticipate for and exploited
them to the best of their ability.
League of Legends and DOTA2 will continue to not only hold the attention of game players
and esports communities around the world in the coming years, but also become increasingly
prominent in AI research.
Hopefully having watched this video you can not understand why I would anticipate a continued
growth of research in this field, but what it is about this particular genre of games
that proves so enticing as a testbed for Artificial Intelligence.
And that's it for this video, I hope you've enjoyed it and don't forgot to like, subscribe
- click the bell, jump through a ring of fire or whatever the fuck else YouTube want in
order to ensure you keep up to date with my videos here on AI and Games!
If you're a MOBA player or researcher, get some discussion rolling down in the comments:
what are you excited about seeing AI tackle within this space?
If you're keen to learn more about some of the research highlighted in this video, you
can check out the full list of referenced academic papers in the video description.
My thanks as always to the sponsors of this series over on the crowdfunding platform patreon
whose names are on screen now.
One of the extra goodies patrons get as thanks for ensuring this shows continued survival,
is the chance to vote on topics they're excited to hear about most.
Last year I put together the biggest vote ever, and in return was tasked with covering
three major topics: the Total War franchise, StarCraft and MOBA's.
After seven case study videos in the last five months - a new personal record - I'm
happy to say I've now satisfied that remit!
In turn I hope everyone has found these topics interesting!
Next up I'm taking the opportunity to make some videos on topics I want to cover, but
also launching a new poll that will guide the series content through to the end of the
2019.
Some of the potential topics are listed on screen now.
Get on over to the community tab here on my channel and vote on what you wanna see.
Meanwhile patrons will not only get the final say on what comes next, but also get sneak
peeks and extra content on my upcoming videos.
As always, if you wanna find out more, head on over to patreon.com/ai_and_games.