In 2016 Tom Clancy's The Division brought players together in the war-torn streets of
Manhattan fighting against a variety of enemy factions, as well as one another. Ubisoft's
RPG-shooter carries a diverse range of enemies to keep the player on their toes. But unlike
other games covered on the show to-date, The Division needs to deal with the practical
issues of being an online co-operative game and how to manage all sorts of AI systems
for thousands of players across the world.
I'm Tommy Thompson and welcome to AI and Games: a series on research and applications of Artificial
Intelligence in video games. Over the next two videos I'm going to be looking at how
developers Massive Entertainment sought to create a variety of interesting and challenging
encounters throughout the world of the Division; how they designed their enemy opponents and
manage their behaviours as thousands of players rush to battle online every day in the mean
streets of New York City.
After an outbreak of weaponised smallpox, New York city breaks down into chaos and is
quarantined from the outside world until the contamination is contained and order is restored.
Players assume the role of an agent within the Strategic Homeland Division: a sleeper
government unit that is activated in periods of national emergency in order to help first
responders bring order to the region. Players can play individually or in groups of up to
four completing a series of story missions and open world events in a region of Manhattan
that at launch spanned over six square kilometres in size ranging from 21st street and 9th all
the way to 55th and 3rd avenue.
To give players some variety and challenge, regions of the map are controlled by five
rival factions: including the flamethrower-loving Cleaners, escaped convicts from Rikers Island,
as well as the private military company the Last Man Battalion. In addition the most infected
region of the city - The Dark Zone - has been cordoned off and is the games major PvP arena
comprised of not only enemy AI but also fellow players who can go rogue at any time and begin
to attack you.
As players explore the city of New York, they will interact with non-player characters (or
NPCs) in one of three scenarios:
As roamers in the open world: such as a wild dog, pockets of civilians, friendly JTF soldiers
near safe houses or hostile mobs out on patrol. Placed encounters: These are instances where
players will find enemy mobs out in the game world, often deliberately positioned by the
in-game systems with help from designers to force players into combat or to make an alternate
route. Narrative missions: specific designer-crafted
missions that are part of the main story of The Division. These are lengthy plot-driven
sequences with an emphasis on glueing together multiple individual firefights.
In each instance, the AI placed in the world has specific objectives that are aligned with
one of two critical design goals: To add to player immersion by enriching the
in-game world and its design. To provide puzzles to solve in combat.
The latter was the critical part of the game: given it aims to encourage players towards
scouting areas prior to combat and establishing a base level of strategy. Once fighting breaks
out, players need to continually re-evaluate their combat priorities and spatial awareness
in order to come out intact. This requires enemy NPCs to carry interesting behavioural
traits for players to recognise, but also some consistency in their basic behaviours,
with a variety of subsystems that handle threat detection, dynamic objective handling as well
as being able to react to players themselves.
All NPC behaviours are achieved through use of a behaviour tree system, where subsections
of the tree are activated to run specific behaviours that are seeded with certain objectives
or variables. This can include the likes of target destinations for navigation, areas
to explore potential threats, enemies to attack, cover to move into and so-on. I'm going to
look at the design and structure of these for enemy NPCs, but if you want to learn more
about how behaviour trees work: check out my videos on the AI of Halo 2 and 3 which
cover some of this in more detail.
The behaviour tree architecture was reliant on a richly layered system, which was capable
of providing multiple paths to each behaviour. This is because of the range of ways in which
one specific action could arise and required the system to be as flexible as possible.
The top layer of the tree breaks down the three critical ways in which a NPC would react,
including direct orders from in-game systems, to maintaining active combat or initiating
combat if necessary. This moves down into the conditions layer, which identifies what
behaviours can be executed based on the current conditions of the world and the character
itself, all the way down to the lower layer which handles pretty much every variation
of combat that is possible in the game.
What's critical here is that the middle layer is built such that it can link to a variety
of different actions on the lower layer. This is to prevent behaviours being duplicated
through the bottom half of the tree based on very specific conditions. As such all the
different NPCs in the game (which I'll come back to in a minute), as well as tools such
as the Seeker Mine - which is technically an AI character - can have similar sub-trees
or behaviours to suit their purposes which are copied around. As you can imagine, this
would result in a behaviour tree architecture that is rather complex - and you're right
- what you see here is simply massive in scale and this required heavy collaboration between
programming and design teams in order to get this working. Given the scale of the problem
and the challenges faced, the programming team developed numerous tools for editing
and eventually debugging each of the NPCs and built these into the Snowdrop engine that
The Division is built upon. These debugging tools are extensive, given they allow for
the developers to identify what is going wrong with a particular character in real-time in
an online instance and I'll come back to talking about this later.
So now that we know a little more about the AI architecture, let's take a look at the
design of the NPCs themselves.
At launch the Division had 36 enemy NPCs types spread across five factions. Archetypes help
establish the type of behaviour of a given character, while factions govern attributes
that are specific to all NPCs within them. To enable a bit of gameplay diversity, there
are large number of archetypes in The Division:
Assault: Basic shooting and cover AI. Rusher: Rush directly towards target.
Tank: Takes and deals large amount of damage. Sniper: Long-range and accurate assault.
Thrower: Throw grenades and area-of-effect items.
Support: Assist teammates in combat (e.g. healing)
Heavy: Suppress players. Leader: Provide buffs to nearby players.
and Controller: Fights via Proxy (turret)
There is also Special NPCs that don't adhere to archetypes - but these are only used in
end-game and boss battles.
Each of these archetypes are designed to give clear feedback of the immediate dangers to
the player and how to strategise their attack patterns. In addition, many of these archetypes
compliment one another such that one can exploit each others attributes in order to achieving
their goals. For example, while snipers maintain their distance from players to deal large
amounts of damage, rushers deal damage up close and immediately try to close the distance
between themselves and the player. Meanwhile Tanks and Heavies force players to stay in
cover, while throwers will attempt to flush players out. As such this forces players to
use different tactics throughout a given battle based on the enemy unit composition. This
is driven by a desire to force players to continually evaluate three key gameplay attributes;
Target prioritisation and ordering - Who should I attack right now?
Repositioning and exposure - Am I in the best cover given the current scenario? Often resulting
in players moving to better tactic positions and keeping them from coming out of cover.
Rewarding endurance - Provide long protracted battles that test a players ability to maintain
their posture at a given level of intensity.
As mentioned earlier, the 36 NPC types were spread across the major NPC factions: Rioters,
Cleaners, Rikers and the Last Man Battalion that is split into tiers 1 and 2. Depending
on where you are within the New York map, you will encounter one or more of these factions
in combat. Each faction is designed to scale difficulty as you progress through the game,
progressing in the order I listed them. But given there so many individual NPC types,
how do you manage the difficulty progression in a manner that is easy for designers to
iterate upon, but also effective enough to be pervasive across entire groups of NPCs.
To achieve this, Massive built an AI profile system that modifies a number of exposed attributes
about each of the NPCs. This enables them to become more or less of a threat to the
player accordingly. This are eight different attributes that are influenced here:
- Reaction time to player actions. - Movement speed.
- Group behaviour - Cover usage
- Skill usage - Stagger resistance.
- Suppression resistance. - Health
- Damage Accuracy
The profiles that are built can be consistent across either a specific NPC type or a faction
that NPCs exists within. So for example the Rioters - who are the first faction in the
game players will encounter - are restricted to using a 'Dumb' profile where all of these
attributes are fairly low. However, all Heavy's, Support's and Turrets have dedicated profiles
for those archetypes that are consistent across all factions. Meanwhile the Last Man Battalion
have their own Elite profile: which makes them far more threatening to players than
all others.
The last and final modifier that has an influence on an AI's behaviour is it's role and rank
within the game itself. Characters range from regular grunts to veterans, elites and finally
the named NPC rank, which is reserved for only the most high-priority targets. As a
NPCs role increases, this has an influence on base stats such as weapon damage, armour
and shot accuracy, with elite and named NPCs having access to unique weapons and mods.
They even get custom laser sights that have absolutely zero effect on their gameplay ability,
but it puts players on edge as a result.
This suite of management systems enables Tom Clancy's The Division to build up a rigorous
and diverse collection of non-player characters that help bring Manhattan to life. Whilst
these systems fundamentally use the same core AI architecture, the use of archetypes, factions,
profiles and ranks enable a lot of diversity in not just the behaviours executed, but also
the level of challenge that these characters present for thousands of players across the
world. But therein lies the bigger problem: how do you maintain performance for these
kinds of systems in a game designed to run online for players all over the world? The
Division has hosted over 10 million players since launch with a peak of over 1 million
concurrent players. That's a lot of non-player characters that need to created, maintained
and controlled over thousands of individual game instances. In part 2 of my case study
of The Division, we're going to look at how Massive Entertainment pulled this off and
explore the online integration. The design decisions that were made to ensure the game
runs as smoothly as possible and the problems and limitations this presented for the AI
systems. In addition, we'll also look at the tools that were built to enable testing of
the AI behaviours in online instances, as well as how AI players were used to test the
consistency and stability of story missions on test servers.
In the meantime, don't forget that AI and Games is a crowdfunded show on Patreon and
is supported thanks to the people whose names are on screen now. You can support the continued
development of this series via the links on screen now and in the description. Thanks
for watching and I'll be back soon with part 2.