In the past few years, an important part of the gameplay experience has been slowly sacrificed
at the altar of immersion. Heads-up displays, or HUDS, were once a vibrant and iconic element.
Now they’re but a wispy ghost of their former selves, only appearing to announce a snowball
or bullet-based opportunity.
We’ve gotten to the point where immersion is synonymous with having no UI elements.
But I’m sure a professional, experienced designer of user interfaces would agree with
me that immersion is much harder to define.
Zane: How do I define immersion? Again, anywhere
where the UI is not present in the game is probably the first and foremost answer.
Outrageous! If Zane isn’t going to defend unnecessarily elaborate UI elements, then
it’s up to me.
But even I have to admit that UIs have earned their bad rep; if you’re a big triple-A
game trying to tell a dramatic, emotional story, the worst thing that can happen is
[OMINOUS TRUMPET SOUND]
– yeah, a button prompt that ruins the mood and suddenly
your serious game is a meme.
Obviously, intrusive UI can disrupt immersion. But actually, Press F proves that immersion
is more than just eliminating any reminders that you’re playing a game.
Immersion wasn’t always a buzzword in games because frankly, I mean… look at this. Being
sensorily transported to another world wasn’t really a risk until relatively recently. Early
UIs often existed outside of the frame of the gameworld because there simply wasn’t
enough graphical fidelity to put it on top.
As resolution and color palettes grew, it became less necessary to cordon off those
elements, allowing the game world to expand and the HUD to sit on top like fat friendly
beetles resting on on a leaf.
A lot of HUDS in the 90s and 2000s had a design flourish that I am obsessed with and that
we just don’t see in big games any more; it’s called skeuomorphism. A skeuomorph
is an element in a digital interface that looks or sounds like the real-world analogue
it’s based on. The fact that the function that removes data from your hard drive is
designed to look like a little trash can? That’s a skeuomorph and also darn cute.
A skeuomorph might be a collectable shaded to look like it’s made of gold (Donkey Kong
Country 1994), Or buttons that are beveled so they looked like real buttons (Ocarina
of Time 1998). These UIs are iconic and flavorful and they reflect something about the design
of the game.
With the rise of 3D graphics in the mid-90s, a lot of triple-A games aimed for more complex
storylines and less UI, in an attempt to provide a cinematic feel. Although these games weren’t
easy, they had some gameplay affordances that made it easier to strip away the UI, like
auto-aim and relatively linear level designs.
Plenty of big titles during this time still used elaborate UIs with incredible unnecessary
flourishes. But as the gameplay complexity increased, so did the amount of heavy-lifting
the UI had to do. Game devs had to get creative with how to integrate this information without
just adding a thicker and more elaborate picture frame.
So UI designers worked to integrate information diegetically. Diegetic refers to sounds or
visuals that exist within the world of the narrative, and can be seen and interacted
with by the game’s characters. When I interact with my Pipboy in Fallout, the implication
is this item exists in the game world, and I’m interacting with it as both the player
and the player character. And that’s my arm on the screen.
Diegetic UI elements are a god-send for game developers trying to avoid HUDs.
DINO: When we got to the UI, the goal was trying
to figure out like, what, what breaks the immersion of the game…
Dino Ignacio was the UI Director for a little sci-fi horror franchise called Dead Space,
which is still the gold standard of diegetic UIs, over a decade after its release.
DINO: ...Because what we found effectively is like
the HUD is like a piece of glass that separates you, the player from the character that you
were playing. And so… the way, if we can lift that off, then suddenly you've maintained
immersion
Ignacio and his team proved that you could make an entirely diegetic UI - or, really,
a UI that seems entirely diegetic - if the game has done the rest of the leg work to
keep you immersed.
Despite Dead Space’s success, not a lot of games that followed were as ambitious with
their design. Ignacio’s team used their particle engine to make the UI, which is a
uniquely difficult method. But in the time after the game’s release, a lot of big studios
relied on a third party software called Scaleform, which allowed designers to make UIs in Flash
and import them into the game. Which is handy but it did result in very similar-looking
UIs.
The 2010s also gave us the concept of flat design, which is precisely as exciting as
it sounds. It’s why your phone’s note app doesn’t look like a notepad any more.
You don’t really see skeuomorphic design anymore, outside of smaller indie games, or
big games where the interface is, like… the whol e game. The big titles have mostly
given up on skeuomorphism, which is… [sigh] fine. Shifting winds will always change the
shape of the dunes.
But franchises that once used a lot of texture now look… same-y.
Zane it's become more homogenous over the past
decade. And I don't know that that's a bad thing. But again, I do think at the same time
It's few and far between that we're seeing UI really being used in creative ways.
Unlike me, Zane’s not unduly emotionally invested in over-designed HUDs.
Zane: I think the joke is that if I could make any
game, the perfect UI is no UI.
Zane! You’re blowing up my spot!
But if the goal is immersion, and UI is the biggest obstacle to immersion, why not just…
get rid of it?
Zane: It is a replacement for the senses that we
would normally have in real life. So for example, when you're playing Halo or Call of Duty,
we have to have a threat indicator to show you for getting hit off screen or in the back.
Now in real life. You could probably tell if you're getting shot in the back, but you
are limited to what you can see on screen.
The same goes for holding your breath under water or climbing up a rock face or knowing
exactly how many milliliters of pee you have left in your bladder. At least for now, these
physical sensations are very difficult to communicate outside of UI elements. But you
better believe that piss-tech is coming.
But that’s not the only function UIs have. Part of the difficulty of UI design is giving
players the right kind of information at the right time.
Dino what of these elements do we need in the game?
And what of these things need to be persistent? And we found out very quickly that the only
thing that really needs to be persistent is your health. Because you want a second to
second view of how you are doing and how far you are from failing or succeeding.
In a combat-heavy game, you always need access to your in-game fitbit, but for say an exploration
game, you might not. Some older games provided constant reminders of button functions, which
I admit is overkill, but most modern games keep this information contextual, only popping
up when relevant.
The HUD is basically a replacement for your memory, according to Celia Hodent, the Director
of User Experience for Fortnite who incidentally has a PhD in Psychology. She argues that UI
is essential for cutting down on the player’s cognitive load; so instead of you holding
onto the information of how many rounds you’ve shot, the HUD keeps track for you. That frees
up your brain’s processing power to focus on important things, like choosing the most
appropriate dabbing animation.
What this all means is that too little information can be as disruptive as too much. Consider
Breath of the Wild; anytime I was cooking, I had to open a recipe list on my phone to
get the right combination of super vital in-game effects I needed. This was way more disruptive
than just having an in-game cookbook would have been.
The Last of Us 2 doesn’t have a minimap and I didn’t miss it, except in the moments
after finishing a combat encounter, when Ellie will announce, out loud, to nobody, everybody’s
dead here! Which was disruptive for me because, how did Ellie know that? I didn’t know that,
shouldn’t I know, aren’t I her?
I found this very frustrating. You might not have! Either way, the fact that a lack of
UI can be disruptive proves that immersion isn’t just about how few UI elements there
are. This understanding of immersion is so widespread that game designers Katie Salen
and Eric Zimmerman dub it the “immersive fallacy.”
In researching this video I read many different frameworks that expanded the concept of immersion,
but the one that seems most useful to explaining hiccups like Press F is the SCI-model.
SCI stands for sensory, challenge-based, and imaginative immersion. Sensory immersion is
the thing we typically think of as “immersion,” the comprehensiveness and realism of the audio-visual
experience. Challenge-based immersion is about striking the balance between the difficulty
of the game and ability of the player. Imaginative immersion is how interesting you find the
stories, characters, and overall world-building.
Sensory immersion in games has obviously improved pretty linearly over the years, but so has
imaginative immersion. Developers have become more invested in telling complex, dad-driven
stories. Gamers don’t just want to know how Mario jumps, but why Mario jumps, and
developers are here to tell them. It’s because his dad didn’t hug him enough.
It’s the challenge-based immersion that’s become trickier. Success in this category
is marked by a player achieving a psychological state called “flow,” known colloquially
as straight kickin it in the chill station. That’s when the challenge of the gameplay
and the ability of the player are working in sync. If the challenge is too high, the
gameplay is frustrating, but if it’s too low, the player gets bored. When it’s appropriately
balanced, you become focused, less aware of the outside world, and time drops away.
Now consider the skills you need to play a modern triple-A open world game. Platforming
and spatial problem solving, item gathering and crafting, at least one, but probably multiple
forms of combat. As the complexity of modern games has increased, it’s become harder
to strike that flow balance – which is where UI comes in.
A well-designed UI will give you that little hint you need at the right moment to keep
the cognitive load in check. Dynamic UIs provide context-sensitive HUD elements – weapon
stats when it’s beatdown time, stamina meters when you’re flexing on them haters. When
that HUD disappears entirely, you know you can kick back with a bottle of Moxie and relax.
Which is why quick time events and cutscenes are so
[long pause]
fucking frustrating
– the UI is telling you to kick back and chill but that’s a bait and Nintendo switch.
The SCI approach to immersion explains what’s happening in notable HUD flubs like Press
F. The sensory immersion is great - look at the lighting and the high gloss on that coffin.
The challenge-based immersion isn’t at stake either. If there was a predetermined “acknowledge
sadness” action and the player was taking too long to use it, it might make sense for
there to be a contextual pop-up. But the game hasn’t taught the player what button they
should use in moments like this, so it tries to provide that information even though it’s
unnecessary.
That’s disruptive to the imaginative immersion, which is the aspect that’s really getting
damaged here. You’re engaged in this serious moment, and the game boils down a complex
emotion like mourning a fallen comrade… to a button-push.
It’s not the HUD’s fault that feelings are complicated, but it takes the blame here.
Sadly Ghosts of Tsushimas that haven’t learned this lesson are doomed to repeat it.
It’s not about having or not having UI, it’s about how well that UI is integrated
into the game.
Or, as professional UI designer and UI Hater Zane would put it:
Zane: It's fine to have UI in a game as long as
they don't realize that the UI is there
Which is why people don’t talk about how absolutely wild it is that Isaac’s health
indicator is on the… back of his suit, where he, the character, cannot see it. That’s
not diegetic and it doesn’t make sense; Isaac is every boss you’ve fought that has
a glowing weak spot on its ass, the UI equivalent of a “kick me” sign.
But that’s where the leg work and finesse of the UI designer comes in; it feels right
because you, the player, are so engaged with using that information that you don’t process
the fact that it doesn’t make sense in-universe.
And in case you’re wondering, I did ask Dino about this:
Dino: Um a lot of these workers, a lot of these
people who have rigs were in the, in the industry of working with others in situations where
they could be compromised, because they're miners or they're in space. And so having
a readout that other people around you could see was important. I know that sounds hokey,
but that's what we came up with.
Dino: At the end of the day, the user and their
needs are more important than the narrative or the immersion that we're trying to build.
Which is why even if it’s stripped down to the most basic, boring-ass components, we can’t,
and shouldn’t, abandon UI entirely.