In 2017's Halo Wars 2, players take command of the UNSC Spirit of Fire once more as they
face off against a new enemy - The Banished: a ruthless band of mercenaries led by the
Brute warlord Atriox. Stranded in deep space with no means of help and outdated combat
technology, Captain James Cutter and his crew must survive a war of attrition against one
of the most ruthless warriors in Covenant history.
Developed by 343 Industries and Creative Assembly and released for both PC and Xbox One, Halo
Wars 2 was designed from the ground up to present Atriox and his subordinates as exciting
new additions to the Halo universe. But for this to work in a real time strategy game,
they need to be intelligent and have a sense of personality. I'm Tommy Thompson and in
this episode of AI and Games we're going to explore the AI Commander system that enables
for Atriox to not only exhibit strategic intelligence, but be capable of defeating players on various
difficulty levels *without cheating*.
I've previously discussed on the show that sometimes developers allow AI characters and
other gameplay systems to cheat to better facilitate the experience. As shown in episode
36, the companion AI in Ghost Recon Wildlands are often invisible to the enemy, spot targets
for the player without seeing them and cheat sync shots to ensure targets go down as expected.
But while companions such as the Ghost Squad and Elizabeth Comstock exploit knowledge of
the world for their own purposes, no genre adopts this principle more, than real-time-strategy.
RTS games notoriously cut corners to improve the experience: from having build behaviours
that don't require them to acquire the resources needed (hello Warcraft 3), to enemies that
ignore the fog of war and always know where you are. One reason why some RTS games cheat
is that it's an incredibly difficult problem to write competent AI for. You need to factor
at least two levels of intelligent decisions: the micro scale where you're moving individual
units, and the macro scale where you're coordinating the overall army. One need only look at my
episodes on Total War back in 2018 - where the evolution and scale of the AI systems
involved at both micro and macro is simply staggering. This isn't an easy problem to
solve, but Halo Wars 2 employed a fresh angle.
The designers intended for Atriox and his commanders to exhibit strategic play without
resorting to cheating. They should make intelligent moves on both macro and micro scale, adopt
specific play styles and adapt to various game modes and difficulty levels. But more
critically, each AI commander should exhibit human like behaviours: helping teach players
to play the gmae, but also inject personality into these characters and bring them to life
within the Halo universe.
Now you might think with the recent success of Google DeepMind training human-like StarCraft
players with AlphaStar that employing deep learning is the way forward. But this isn't
a cost-effective and pragmatic approach for a video game. AlphaStar isn't designed to
be an interesting opponent for varying skill levels, its built to win at the highest levels
of competition. Plus, there's the element of cost. As cool as it all is, AlphaStars
training process cost tens of millions of dollars to train and that's simply impractical
for even the biggest AAA studios.
So while 343 and Creative Assembly didn't train a machine learning AI based on human
players, they employed a similar principle. Their AI commanders show human-like traits
in combat, because they're based on strategies by high-ranking human RTS players, with one
specific player a critical part of the process.
Balint Marczin, more commonly known in RTS circles as Nakamura, is a competitive RTS
player, streamer and caster and at one time ranked as one of the top 10 players in the
world in Halo Wars for both Xbox and PC. But in addition, Marczin is a game designer at
Creative Assembly, with his work to-date including both Halo Wars 2 and Total War Warhammer 2.
His job was to research the macro and micro layer strategies human players exhibit within
the game and work with the programming team to build a system that would enable for AI
to replicate it in various scenarios.
This resulted in five macro strategies that define a particular style of play being encoded
into the AI system:
- Rush: AI players that attempt to get a large army on the field as quickly as possible,
destroying and capturing expansion bases as it moves across the map.
- Boom: AI players that quickly capture multiple bases and prepare for a war of attrition.
- Map Control: a more aggressive form of Boom that focuses on capturing and maintaining
capture points across the map.
- Fast Tech: where the AI moves quickly to a minimum of tier 2 units production, in an
effort to surprise their enemy with heavier units.
- Turtle: A defensive straegy that captures bases, stays on tier 1 units for longer, deploys
turrets and upgrades expansion bases.
Meanwhile on the micro layer there are 10 unique behaviours the AI can execute and it
allows for a lot of minute control of the individual units. This includes common behaviours
such as focussing fire on individual units, holding ground at a specific location or ordering
special abilities such as cloaks, shields, grenade throws and healing. But there are
also micro tactics commonly exhibited by human players such as the pull-back and kite. In
pullback, the AI commander will monitor a squad of units in combat and cycle out those
that receive heavy damage from attack range, while kiting is a tactic that keeps enemy
units just beyond attack range, allowing for either long-range attack or opportunities
to quickly deliver damage before backing of once more.
So at any point in time, the AI system in game will read the current game state and
feed it into both a macro strategy planner and a micro behaviour selector, meaning the
system can make intelligent decisions about a long-term strategy to execute, but also
run micro behaviours to help ensure that strategy is being maintained in the long-run. While
the micro layer is focussed purely on movement commands, the macro layer is built into multiple
layers that focus on construction, tech ladders, leader powers and the management of armies.
But one of the real successes of this two-tier system, is that it allows for both macro and
micro layers to be customisable based on the in-game scenario. In certain campaign missions
or in multiplayer skirmish matches, the AI system has complete control of both layers,
allowing it to make its own decisions and conduct strategy its own way. But there are
times that the game designers wanted to have more control over the system. Campaign missions
that need more scripted behaviour, or a reduced competency at macro behaviour in Blitz firefight,
or completing disabling the strategic elements to become purely reactive or having zero intelligence
whatsoever like in some campaign or tutorial missions. All of these game modes, all facilitated
by the same AI system.
So with all these systems in play, how do you create unique personalities or customise
the AI to operate at different difficulty levels? The trick lies in the decision making
at macro level, which is broken up into three sub-systems: the alert manager, the strategic
decision maker and the strategy manager. The alert manager is how the system reacts to
local information whilst making strategic decisions, hence it can interject and influence
decisions being made to respond to dynamic changes in the world, such as creating new
units and adding defensive structures such as turrets. The strategic decision maker is
reliant on the strategy manager: a system whose decisions are influenced by one or more
strategy tables.
At the beginning of a match, the strategy manager will read in a strategy table. Each
table describes the strategy types that are being used by this AI character and defines
their style or personality for the game. While this defines which of the macro strategies
and micro behaviours it includes, the strategy table loaded into memory is influenced by
the game mode and we'll talk about that in a second. The strategy table also includes
build orders, tech and leader power unlock orders and preferences and the type of sub-missions
this AI will frequently use - ranging from attacking enemy structures and defensive tactics
to scouting the map and foraging for resources.
By defining all of this in a table, the game becomes more heavily data driven. The programming
team have built all of the strategies and behaviours for the AI to execute as well as
a XML driven format for designers to feed strategy tables into the system. This gives
designers the freedom to develop new strategies - and even unique executions of the same strategy
- without having to worry about the programming side of things. This in-turn crafts the personality
and behaviour of individual leaders. This isn't just for the Banished characters, but
also UNSC leaders that appear in Skirmish modes such as Professor Anders, Jerome-092
and Captain Cutter himself. Plus returning to the system overview, the strategic decision
maker extends the life of a given strategy table, allowing the system to dynamically
add new behaviours to a chosen strategy in order to flesh it out and show growth over
time.
Now the last thing to address is difficulty: since a given leader will play more or less
intelligently based on the difficulty level selected. To address that, there are two approaches:
in campaign there are multiple strategy tables for a given leader - one for each difficulty
level - which gives level designers a lot more control over each playthrough. While
in skirmish, a strategy table is built for Legendary difficulty and then scaled down
based on the selected difficulty. This limits the number of uber units, the max number of
turrets deployed, restrictions to tech and leader power upgrades, and even limitations
on the types of micro behaviours the leader can execute.
While this creates plenty of flexibility for programmers and designers, this is all an
incredibly difficult balancing act to pull off. At launch, the game had 94 strategy tables
built: 28 for campaign as well as 66 for skirmish. These require the design team to be stay on
top of the meta of the game as more DLC and updates roll out. Halo Wars 2 had a fairly
extensive set of DLC release post launch, including new characters as well as an entirely
new campaign. This added another 15 campaign tables and 111 skirmish tables bringing it
up to a total of 220. This has seen a lot of work on design side to manage these AI
personalities after launch, with a lot of the skirmish AIs on the lower difficulties
not just heavily tweaked but actually dumbed down after launch based on player feedback.
This resulted in AI players who upgrades, leader power usage and re-building of destroyed
structures much more limited as well as forcing them to fan out a little more. Given lower-level
AI leaders would frequently turtle to a point that it was too difficult to beat them for
more novice players.
Halo Wars 2 strove to be a polished and accessible entry to the franchise for both seasoned RTS
players as well as those new to the genre, with a cast of characters that evoked their
personalities when in combat against the player. What's really interesting about this is that
the creators of the Total War franchise sought an entirely new approach to building AI for
a strategy game, rather than adapting existing systems from their own work; enabling programmers
and designers to work together to achieve the best outcome for a variety of situations
and players.
Thanks for watching this episode of AI and Games on Halo Wars 2. As we wrap up I would
like to give special thanks to Creative Assembly - the developers of Halo Wars 2 - who provided
not only feedback but the in-engine development footage shown throughout this episode. With
thanks in particular to Balint Marczin (Game Designer), Dr Derek Fagan (AI Gameplay Programmer)
- whose Game AI North talk from 2017 is the basis of the video - and Lucy Boxall (Head
of Corporate Communications) who made this all possible. Thank you all so much for your
time, it's greatly appreciated.
Halo Wars 2 was a topic voted for by my crowdfunding supporters who have a say in what we explore
in the series. To have a say in what is made next, plus support me to actually make these
videos, join the AI and Games Patreon like these lovely people already have by checking
the links on screen and in the description. Have a good one folks, I'll be back.