Hi I'm Tommy Thompson and this is AI and Games: a series on research and applications of Artificial
Intelligence in video games.
Since the beginning of this channel I've explored a variety of AI techniques to solve specific
problems within video game design that require both immediate, reactive behaviour in conjunction
with long-term and strategic decision making.
First-person shooters are certainly a common genre from which interesting AI can emerge
given its predicated on fast, reactive and engaging gameplay.
But if we're to consider the need for effective systems that balance both short and long term
decision making in games, there is no greater problem space to explore than Real-Time Strategy
(RTS) games.
Both the RTS and FPS genres have continued to innovate the state of the art in AAA video
gaming in an effort to satisfy their design.
But RTS games have much larger considerations for the long-term impact of quick and reactive
decisions, as well as the need to manage resources at varying levels of abstraction and complexity
over multi-hour gameplay sessions.
When considering the challenges and innovations faced in AI for strategy games, as a player,
developer, or researcher there's one franchise you simply cannot go without discussing, Creative
Assembly's Total War.
Total War has grown from strength to strength since the launch of the franchise in the year
2000.
As each game becomes more ambitious, the challenges faced by the AI systems continue to increase
in scope and complexity.
This has resulted in a development team that continues to push the state of the art in
game AI unlike any other in video game history.
But the journey to reach more recent entries at the time of this video, such as Total War:
Attila and Warhammer has had more than a few bumps in the road along the way.
So in this first in a series, I will be looking at not just how the core tenets of the AI
of Total War, but how each entry in the series aims to solidify, tweak or completely rewrite
parts of these systems.
In addition, I'll look at the response to these changes from the fanbase, the modding
community that has thrived within Total War and how The Creative Assembly adopted groundbreaking
academic research to help innovate from the very first entry all the way to more recent
releases.
So let us begin our march to total war.
But first let's get into the Real-Time Strategy genre and understand where Total War sits
within this space.
Real-Time Strategy games are focussed primarily on the control of territory and assets: whereby
two or more opposing forces will appear in the same map in an effort to assert dominance
by securing resources or locations within the existing defined area.
Often in order for this to be achieved, each force will build collections of structures
and units that either improve the resource collection within a given area, defend existing
structures and land or scout and secure areas of the map by force.
The latter often requiring the destruction of opposing forces units in the process.
This process typically requires players to explore a previously hidden environment by
piercing what is referred to as the 'Fog of War' in order to establish enemy and resource
locations.
In addition, over time each faction is expected to build upon the structures and units previously
established by enhancing their existing capabilities and resource acquisition.
While arguably not the first RTS game ever made, Westwood's 1992 release Dune II: The
Building of a Dynasty acted as the archetype for future games in this genre such as Westwood's
own Command and Conquer series, but also Age of Empires, Total Annihilation, Dungeon Keeper,
Homeworld and Blizzard's Warcraft and Starcraft franchises.
The genre has continued to evolve and branch off since the late 1990's, with the likes
of real-time tactics games like Dawn of War and Star Wars: Empire at War emphasising tactics
over resource management.
Meanwhile games such as 1994's UFO: Enemy Unknown have explored a balance between the
real-time and turn-based decision making, with it still proving popular to this day
with the franchise being successfully rebooted as XCOM in 2012.
Arguably the largest deviation is the the rise of the Multiplayer Online Battle Arena
or MOBA genre - popularised by the Warcraft III mod Defense of the Ancients and typified
by games such as Dota 2 and League of Legends - where players assume the role of hero units
on the ground to define and assist the larger strategies playing out during a match.
AI applications within real-time strategy games have existed since the inception of
the genre, with players responsible for managing the movement of faction units by telling them
where to go, but without direct control of their actual movement.
As such, players are expected to interact with these systems in order to yield desired
outcomes.
In addition, AI is critical to the single-player aspects of the game, given that players play
off against opposing commanders responsible for managing their own units and construction
strategies.
The role of the AI player is one that is tremendously challenging and is considered to be one of
the larger challenges facing academic research.
This has led to the forming of numerous academic benchmarks and projects, including the Starcraft
AI Competition hosted at the AAAI conference on Artificial Intelligence and Interactive
Digital Entertainment.
This is a video in and of itself, but don't worry: I'll be talking about that on AI and
Games in the near future.
So what makes Total War so special and challenging for an AI system? Creative Assembly's strategy
series is interesting in that it combines multiple modes of strategic gameplay into
a single game: with players managing turn-based resource management and political strategy
alongside real-time combat with large scale armies.
In addition, the combat juggles between micro and macro levels: requiring not only AI control
of individual units, but more abstract control of unit types and managing the combat formations
and troop deployments on the battlefield at runtime.
The first entry in the franchise, Shogun: Total War, balances the combat simulations
that strive for realism and authenticity, alongside the political strategy that is aimed
to give context and stakes to each conflict.
The original Total War is set in 1530, during the Sengoku Jidai period of feudal Japan:
a time largely popularised in contemporary fiction by the works of Japanese film director
Akira Kurosawa, with films such as Kagemusha, Seven Samurai and Ran proving influential
on the design and development of the game - with clips of the latter being used as part
of the games cinematics.
Both the player and opposing AI assume the role of the Daimyo: local-lords who control
provinces of Japan with a need to conduct both diplomatic strategy alongside military
movements.
When rival factions are drawn into conflict, players take control of the Taisho or general
and move hundreds if not thousands of troops across the battlefield.
Sengoku Jidai made for an ideal period of history for the game, given the politics and
even the economics of the period was built around the logistics of fielding armies in
defence of the Daimyo and his ambitions.
Total War deviates from many traditional real-time strategy games in that it removes mechanics
such as Fog of War from combat gameplay as well as the need for resource management outside
of unit counts from combat decision making.
Given the nature of the game itself, the AI player is comprised of two distinct AI systems:
the Diplomacy AI and the combat AI.
The Diplomacy AI manages the turn-based strategy of the game and is responsible for moving
armies around the map, conducting diplomacy (be it by sending envoys or assassins to forge
alliances or eliminate opposition) and building the agriculture and infrastructure of owned
provinces.
Meanwhile the combat AI dictates combat unit formations, strategies and attack patterns.
Much like human players, this is only responsible for managing the strategy of specific groups
of units, given the units themselves are already controlled by AI techniques.
So to get to grips with the AI of Total War, I'm going to explore the AI systems from lowest
to highest level: from individual troop control systems all the way strategic systems that
seek to conquer all of feudal Japan.
Total War combat is driven by units: groups of specific troop types that can be deployed
in formation ranging from melee types to archers to cavalry.
These combat units are not only expected to keep formation, but to move around and conduct
combat as a unit.
Movement is often a tricky task, especially when asked to navigate through or around a
variety of terrains such as mountains and forests.
To achieve this, Total War adopts artificial neural networks for managing individual units.
Neural nets are a fast and effective means to achieve quick and reactive responses to
a pre-defined objective.
We typically train them using machine learning methods but they can also be tweaked by hand
if the network is small enough.
What makes this so effective is that once they are trained, the processing time to make
decisions is lightning fast.
In addition, a well-trained neural net is able to generalise its decision making process:
meaning it can recognise the similarities in numerous individual circumstances and in
each case give a similar answer.
Despite this, neural networks are typically not great when being given multiple objectives
to complete at once, especially when they oppose one another.
In this instance, the troops have different neural nets they can trigger in depending
on the objective they're trying to solve, whether it's moving formation, avoiding oncoming
enemy fire or taking position to attack themselves, be it up close or from afar.
Each of these networks is pre-defined and is not tweaked or optimised during gameplay,
so it's not learning to be better at the game as you're playing it.
These Unit AI systems are in effect regardless of whether a human or AI is in control of
the army, as each requires these neural net systems to manage the individual troop behaviours.
The combat level is the AI system that controls opposing forces akin to players.
This AI needs to decide the group formations and actions of units.
Moving segments of its army together in order to achieve a specific tactic or micromanage
movement of units given the terrain or weather that is currently in play.
To achieve this, Shogun employs a logic system that allows for specific decisions to be made
based upon the current state of the world.
These decisions are derived from 'The Art of War': a collection of strategies by Chinese
military strategist Sun Tzu that dates back to around 500 BC.
The use of a logic system is well suited to adapting the texts, given that in many instances
a set of conditions are in play that should result in a particular response from the system.
These rules considers the type, condition and size of both our units and our opponents,
the current terrain, the weather and state of troop morale.
It's been stated that there are 220 rules embedded in the combat AI system across three
distinct versions, each with their slightly different sets of behaviours.
A driving force behind this was to ensure that combat was as realistic as possible,
deviating away from the tactics that had driven many real time strategy games of the era.
As Mike Simpson, Development Director of Creative Assembly stated:
"Most [games] use simple scripts of canned behaviour that fire when you bump in to them,
and very simple swarming behaviours.
They’re limited, and are “gamey” rather than real world.
What I mean by that is that the tactics you use to beat them are something that you have
to learn for each game or sometimes each scenario/level, and bear no relation to reality.
What we strive for is a game where real world tactics actually work.
It’s not the easiest path to take, but it is the most rewarding."
The final layer of Shogun's AI systems is the diplomacy system.
This is in control of all decision making on the main campaign map: managing army movement,
conducting diplomancy between factions and building infrastructure within provinces.
This uses a mixture of classic state-machine decision making alongside the use of genetic
algorithms in order to customise the tactics made by each Daimyo.
Genetic algorithms are a form of machine learning where we create large quantities of candidate
solutions to a given problem that learn to solve it by breeding the best ones with each
other.
These candidates aren't code, they're an encoding of how the AI behaves typically as a string
of numbers, which is then translated into the AI itself to dictate how it behaves.
These allows for us to create a collection of diverse solutions for a given problem:
they don't all behave in the exact same way and have their own slight characteristics.
As a result, each Daimyo has a unique combat behaviour.
From a players perspective, this gives them each some sort of personality: allowing for
Daimyo's that are more willing to conduct diplomacy to others that will take a more
aggressive stance in expanding their territory: attacking both AI and human player alike.
With these three systems, the core tenets of the AI of Total War are in place: to manage
individual units whilst retaining cohesive behaviour, to group units of troops together
in a manner that is responsive, reactive and challenging towards players and to create
strategic opponents with personality.
Despite being built back in the year 2000, the core principles of combat and strategy,
combined with the systems built in Shogun have carried the Total War franchise forward
into new and varied theatres of war in many a subsequent release.
Despite these innovations behind the scenes - even from the very beginning - the AI has
never proven infallible and as such, with every release there is a drive to tighten
it up and improve it as best as possible.
The first changes came as the series migrated from feudal Japan in 2002, with the launch
of Medieval Total War.
Medieval pulls the game farther back in time to the late 8th century all the way to the
14th century, with players taking charge of a variety of factions at different periods
of history: with the likes of the Viking era in Britain - where much of the country is
fractured following its abandonment by the Roman Empire - all the way to the late-medieval
period, where much of Europe has taken shape as actual countries.
Medieval: Total War is in many respects a refinement of the original game and is built
on an improved version of the original Shogun engine that among other things allowed for
larger battles than its predecessor.
There are no major innovations between Shogun and Medieval for the underlying AI tech, just
a number of optimisations and improvements.
The unit AI systems were improved to prevent bottlenecks in tight spaces such as bridges.
In addition, the suite of cbomat AI systems was expanded from three up to nine allowing
for greater versatility.
All in all this resulted in a game that Rock Paper Shotgun referred to as near-perfection.
Rob Zacny says...
"I’m still not sure a more balanced Total War game has ever materialized.
The Risk-style map is easy for the AI to manage, and the different starting positions of each
kingdom and empire allows for some true AI superpowers to form and challenge players
late in the game.
Medieval is a triumph of simplicity, and it took a decade for Total War to come close
to matching it."
All in all, this is just the beginning of the franchise and it's safe to say that the
road to war is not a smooth one.
With each subsequent release in the franchise, the challenges have become larger and the
scope has increased significantly.
While war never changes, the AI behind it most certainly does.
I'll return in part 2 with an exploration of the modding community that arose from 2004's
Rome: Total War, how the release of Medieval II: Total War drove a need for the core systems
to be re-written and rebuilt in Empire: Total War and how this seismic change impacted the
franchise all the way to Total War: Shogun 2.