Horizon Zero Dawn stands as one of the most critically acclaimed of Sony's roster of Playstation
4 exclusives. As the hunter Aloy, players venture across the post-apocalyptic landscapes
of the future to uncover the mysteries of her past and how the world fell in the years
before. Humanities fall from grace has led to the rise of 'the machines' - robots of
varying shapes and sizes that that now run free across the lands. These robotic animals
are intelligent, coordinated and lethal: requiring quick thinking to survive and even greater
preparation and planning to successfully destroy.
I'm Tommy Thompson and this is AI and Games: a series on research and applications of artificial
intelligence in video games. For the next two videos, I'm exploring the gameplay and
decision making systems that bring these creatures to life and power the core gameplay experience.
I'll be looking at not only how they make decisions to scavenge, to hunt and to survive
but how they coordinate themselves in the wider world either individually or as part
of a herds. Plus I explore how the game creates an ecosystem of machines that is balanced
and managed across the many territories and regions of the new world.
The robotic wildlife of Horizon Zero Dawn dominate the landscape and hold control of
territory throughout this open-world experience. Each machine carries a unique aesthetic and
structure that reflects their function and purpose within the narrative of the game.
As players work their way through the campaign, we come to learn of the Faro Plague - an army
of rogue war-machines that rendered the world asunder - alongside the race of machines constructed
by the AI known as GAIA that are designed to restore the planet to its natural glory
and are largely inspired by long-extinct terran species.
As a result, the function of a given machine often holds the secrets to their weakness,
players can use their Focus to scan local machines to ascertain not only their type,
but vulnerabilities as well as weak points. In addition to the weak spots, a machine's
function within the narrative ultimately influences their in-game behaviour, this includes:
- Acquisition and Scavenger robots such as the Strider and Grazer. These worker class
machines are typically found in open grass lands working the lands and while they are
relatively docile and often flee confrontation they are nonetheless deadly when provoked.
- Recon machines such as Watchers and Longlegs that scout the region for threats and alert
nearby machines such that they can fight or flee.
- Transport machines such as the Behemoth and Shell-Walker that gather up resources
from the acquisition robots making them valuable targets. They are slower, more cumbersome
and typically defended by more nimble units. - And Lastly combat machines such as Sawtooths
and Stalkers that are primarily there to attack the player and other nearby threats.
There are 28 unique machines that can be found roaming the wild lands, each with their own
distinct passive and aggressive behaviours and players need to learn how best to approach
them. While the greatest threats on the path to Meridian - such as the Stormbird and Thunderjaw
- roam the wilds on their own, players can still run afoul of less lethal machines that
operate in herds. It's common to see the smaller robots such as Watchers, Striders, Scrappers
and Longheads co-existing: supporting one another as they achieve their base functions.
Over time these herds increase in size and diversity as players continue to explore the
world with the likes of Shell-Walker convoys, packs of Tramplers and Sawtooths protecting
nearby Grazers and Longheads. In order to pull this off, the game requires numerous
AI systems that dictate the behaviour of each robot, how they behave within a herd, the
collective intelligence of the herds themselves but also balance the herds across the open
world for a more varied experience.
The developers behind Horizon Zero Dawn, Guerilla Games, came to this project lacking many of
the AI tools required in order to bring this experience to life. Having previously worked
on their Killzone franchise, their toolchain was largely focussed to accommodate first
person shooter and none of the design challenges that faced them for a large open-world experience.
As such, during the course of development they built newAI and gameplay systems and
expanded existing ones within the studio to bring this experience to life and over the
next two videos I'm going to explore the following:
- An agent hierarchy system that dictates how each machine behaves both individually,
but also as part of a pack or herd. - The expansion of the existing planning and
execution system originally built for Killzone that allows each machine to make decisions.
- A set of sensor systems that enables the machines to quickly and effectively detect
threats and investigate when necessary. - A dynamic and intelligent navigation system
that provides interesting movement patterns for both ground robots that respects the world
geometry and varying machine size. - A completely separate navigation system
that catered for air-based machines such as Glinthawks and Stormbirds.
- An animation system that respects each machines composition and ensures they move and attack
in an intelligent way. - Last but not least, a system known as The
Colle ctive that manages every single machine that is active in the game world and how they
are distributed across herds.
So let's begin by exploring how each AI character is built to make decisions and how the game
manages herd behaviour.
Each machine is represented in the world as a separate non-player-character AI called
an 'agent', but perhaps more interestingly, so is each herd. Horizon Zero Dawn adopts
a hierarchy whereby a machine can exist entirely on its own as an agent or as a child of a
group agent - which is used to store information about herds.
A group agent doesn't physically exist in the game world, but it's responsible for coordinating
all agents within it. Each individual agent has its own set of sensors that enables it
to hear noises closeby or spot threats in proximity. But in addition, a group agent
has a blackboard system that stores in-game information a machine might want to use. This
not only allows the game to keep performance costs down by storing information each individual
might try to calculate on its own - like safe areas to stand in or active patrol routes
- but more importantly, agents in a group can share information between each other.
Hence when a recon machine spots the player, each machine can make a decision on how to
respond to that information. Hence you might see acquisition robots in the herd such as
grazers opt to run away while nearby combat machines move in for the kill.
Now the actual AI system used in Horizon Zero Dawn is called a hierarchical task network
planner. Unlike the planning systems used in games such as F.E.A.R. that planned one
action after the other, HTN planning generates plans comprised of action macros. Each macro
contains multiple actions in a set sequence. This is ideal for game development given designers
can put together macros of good behaviour they expect to see in-game. I've previously
explored how HTN planning was used in Transformers: Fall of Cybertron, but they're also used in
games such as Max Payne 3, Dying Light and - rumour has it - in Dark Souls. But it's
worth noting that Guerilla have been used HTN planning since Killzone 2 and as such
this wasn't built overnight, but has gradually matured over years of projects within the
studio.
The HTN planner is used in two capacities by machines: individual agents can ask for
a plan of action to resolve a specific task, such as finding new areas to visit, attacking
a human, or fleeing to safety. Meanwhile group agents can run plans that dictate what goals
individuals agents should be assigned, allows groups to actively pass information between
one another within a hierarchy and potentially form new subgroups or destroy existing ones.
In short, each individual machine is still largely responsible for their own behaviour
and addressing the goals assigned to them and uses the planner to make decisions. Meanwhile,
the groups keep individuals coordinated, re-group them based upon what's happening in the world
and - perhaps critically - assign roles to each machine such that they know what their
job is to support the herd.
Now with all of these machines being spawned into the world either individually or in groups,
how does the game know what machines to spawn in the open world and where? There are designated
sites across the game map where we can expect specific machines to appear either individually
for machines like Thunderjaws and Stormbirds or in flocks for the likes of Striders, Grazers
and Glinthawks. However, the rest of the game-world is full of herds comprised of different machines.
So how does the game know when spawning in a specific machine where to place it in the
world and what groups to assign it to based on the needs of the worlds design?
This is achieved courtesy of a system known as 'The Collective'. The collective is the
supergroup: all machine groups and individual machines exist in the collective. It manages
the spawning of all machines into the world, keeps track of whether they're in a group
and handles their movement between groups and manages the machine ecosystem, such that
performance isn't impacted overall.
Individual machines in the world can request to join an existing open-world group if they're
on their own. This is useful feature given many machines are left isolated after the
rest of the previous group either fled or have been destroyed. So the collective can
recycle isolated machines to join other groups. To do this, each machine has a passport! Their
passport stores facts about that machine, such as its level and machine type and the
collective uses this to determine whether it will fit the requirements of another group,
only moving across if it satisfies the groups needs.
Now for the remainder of this video I want to focus on how herds operate and how group
agents execute as players interact with them. Herds are typically placed into the world
in a relaxed state where they're not under threat and simply hanging out in a specific
region of the map. Any given herd is typically a mixture of recon, acquisition and combat
machines, with acquisition robots in the centre, whilst combat and recon patrol the outside
to defend them. Even in the earliest phases of the game we see the simplest form of this
behaviour, as watchers patrol around striders and grazers.
When a herd is placed into the world, a collection of groups within that hierarchy request a
role. This role assigns all machines it's controlling with behaviours that reflect their
purpose. This can include patrolling for threats, searching nearby disturbances, attacking enemies
or simply scavenging resources in the world. These roles can shift as time passes and events
force the herd to change its behaviour and structure. There is typically a limit on how
many machines can fill a particular role within a herd such that their make-up isn't unbalanced.
A machines role dictates the goals that the an individual agent will seek to complete.
Hence with a relaxed herd, the acquisition machines will use the HTN planner and data
stored within the group to find a local area to mine. Meanwhile recon and combat machines
will generate patrol paths to follow. Patrol paths are auto-generated based on the local
geometry, avoiding awkward terrain and - more importantly - avoids passing through long
grass and other vegetation the player might use to stealthily approach the herd. However,
in order to give players a fighting chance, many patrol paths will deiliberately pass
close to stealth vegetation such that you can place traps or attempt to hack them more
easily.
But of course it doesn't always stay this way; as players start to attack machines their
behaviour will shift accordingly. For machines that aren't part of herds, this comes down
to the specific machine class. Only a few will actually run away when provoke directly,
with the vast majority of machines attacking the player, with even acquisition machines
often attacking the player if they're on their own.
However, should the player attack a herd, the groups within hierarchy react in kind,
but only if the herd has been alerted. As mentioned earlier, the group agent enables
data to be shared between each machine, but it doesn't update on every tick. As a result,
the herds don't have a hive intelligence and it is possible for the player to kill a machine
without alerting the rest of that group. However, should this be spotted or if the machine under
attack survives, it will notify the group that they're under attack.
At this point, the hierarchy within the herd shifts, groups of machines within the herd
request a new role and priority for execution of the corresponding behaviour. The group
comprised of acquisition machines assume the flee role and run away together, while the
groups comprised of combat and recon machines re-balance themselves with their roles prioritising
each character that is attacking the herd. While in many cases this is just the player
on their own, this enables the system to scale for story missions where the player is part
of a group or for other dynamic events in the world where other human NPCs are attacking
herds.
While at this point things have gone horribly wrong, it still gives players plenty of chances
to turn the situation around, as both the grouping systems and AI combat systems work
to create an experience that is both fun and challenging. This is achieved courtesy of
an AI combat system that attempts to structure combat such that it isn't chaotic or broken.
The combat groups that formed have different roles and priorities that are balanced out,
hence the larger machines in a herd will be spread across the groups where possible, with
smaller machines lined up to support them. In addition, groups select machines to attack
the player based not just on whether the HTN planner thinks an it can execute a specific
behaviour, but also whether that attack is going to be interesting. If all machines simply
attacked the player at once, then you would die a horrible death pretty quickly. As such
the combat systems use an approach similar to that found in Halo 3 - a topic I covered
in 2018 - where it balances which enemy attacks you based on how interesting that attack will
be in the context of combat.
This requires an action selection utility function that is calculates how interesting
it would be for that machine to attack. This is based on the current machines state, whether
the player is aware of that machine in proximity, how close it is to the player and the amount
of damage it has received and dished out. This becomes a precondition of the HTN planning
system, and aims not only to provide some challenge, but also to inject some variety
into the combat.
Perhaps more critically, when the HTN planner is asked to generate actions for attack groups,
while an attack is being selected the other machines in the group are also being given
movement behaviours to circle the player and wait their turn. This is done deliberately
to create opportunities for the player to exploit, either attacking a passive machine
or countering the one that just attacked you. In time you'll slowly be able to wear them
down, largely because the combat system is deliberately leaving itself open to attack.
As we've seen throughout this video, the machines of Horizon Zero Dawn can prove lethal either
on their own or as part of a pack. Through Hierarchical Task Network planning and an
intelligent agent hierarchy system, machines can roam the wilds on their own or protect
and support one another where necessary. Creating a fun and exciting world to interact with
that keeps players on their toes at all times.
Thanks for watching part 1 of this case study on the AI of Horizon Zero Dawn here on AI
and Games. In part 2 I'm going to dive into the tools and systems used by individual AI
characters: the sensor systems used to recognise and attack nearby threats, how the AI systems
need to work in-tandem with animation to ensure these machines look both dynamic but realistic,
the navigation systems that allow machines of different sizes to move through the same
terrain plus how flying machines such as the Stormbird use an entirely unique navigation
system that allows them to explore the world and hunt you down as best as possible.