Even if it's hyperbolic, a complaint could still be
a fantastic feature in disguise if you are willing to dig into it
and maybe split it out into its most common theme and then see okay.
Is that a sweeping feedback online?
Am I seeing more of that?
If you focus on how negative or hyperbolic
the actual feedback is, you might miss something really good.
I think, especially in context of like multiplayer games.
There was a lot of design decisions that have huge fall back and then huge
kind of effect onto how you're going to build what you're building.
And no decision is made in isolation.
If I decide that, Hey, I want to have these,
like tiny little monsters attacking the players, like,
okay, actually they have to be controlled by the server and suddenly
the game is incredibly hard to scale and incredibly hard to run
and any multiplayer game actually is incredibly hard to test as well.
It's weird because like you want to be free with your ideas, but at the same time
when the paper is blank as like make a game, what kind of game?
What is the platform? Where are we making it for?
Do we have a brand?
It's actually much easier and and more fun actually
to be tasked with
with some kind of definition like, hey, you're making a Transformers game,
which is a built in battle title, for example.
That was the brief of my previous title.
So okay, that immediately narrows it down.
And we can focus on innovation and we can focus our strength,
like to play those strengths
and we can we know what we’re building so we don't have to find that.
We know there is an audience that we can address.
So it makes it much, much easier.
But yeah...
it will depend on the company and the time and the position
where you are, where you're at.
But sometimes you have total freedom, which is very daunting and scary,
and sometimes you have almost no freedom at all, like,
hey, let's make another Call of Duty, you know exactly what you're building,
and then you're focusing your innovation elsewhere and you're thinking about
the actual structure and the content.
It's more of an execution play as opposed to innovation play.
My job is to figure out what constraints we're operating within and effectively
the problems we're trying to solve.
So as we're figuring out what the game's going to be, what the product's
going to be, not just from a gameplay point of view,
but bringing in brand and community and all the components that go into a game.
What ingredients have we got to work with where we're trying to enter the market
and continuing to try and answer those questions to empower our designers,
our engineers, etc., to to make the best decisions of the right context.
That's really what my team are there to do.
My day to day tends to be quite fluid
from feeding into everything that you guys described from like feature
prioritisation to like brand positioning, communicating things like patch notes,
bugs, problems and issues, proactively sourcing
feedback, trends when it comes to like sentiment and stuff like that,
but also helping with like future roadmap prioritisation.
What's really good for a live game in terms of like your six months
in one year based on how your game has landed
and also balancing reception and sentiment with business needs and like
actual upcoming changes that sometimes they can be really tricky to communicate.
So that is pretty much my focus.
It's it's amazing how every single social media
has its own generalised tone of voice when it comes to giving game feedback.
Like it's a-- it is a joke online sometimes about like you know
oh Reddit sounds this way and Discord this way and Twitter this way.
But in that vein, all feedback is valid no matter the forums or channels,
which means it's delivered necessarily.
There’s comments that through common sense you can think that's completely
counter productive.
Why would I surface that to my devs if it's just-- there's nothing to learn from it?
But even if it's hyperbolic, a complaint could still be
a fantastic feature in disguise.
If you are willing to dig into it and maybe split it
out into its most common theme and then see okay.
Is that a sweeping feedback online?
Am I seeing more of that?
If you focus on how negative or hyperbolic
the actual feedback is, you might miss something really good.
So...
I think community is part-- is one tool you have.
Then you have the team itself and then you have the analytics.
So everything that players do
informs us and gives us pure data on whether they are completing
certain challenges or quests or they’re skipping or they're not using certain tools at all.
So what we do is we use community, but we also have to be careful
what they're wishing for because very often
you'll be speaking to people who are the most engaged players
and they're like the super fans and what they want is they want more like
ranking and like hardcore features, like on the leaderboards.
They want to compete against the top players, but actually
90% of people don't care about it.
They just leave the game because the tutorial is way too hard, for example.
So we have to marry the analytics, the kind of
quantitative
data with the qualitative from from community.
And then we look at that and we see where the issues are,
where things are not used
let's say frequently enough, where people are not completing certain things.
Then here's the things they want, they say they want.
Here's what they're actually doing and here's what we think
will make the game better.
And it's a constant kind of balancing act, and I'm pretty sure Alex
is kind of losing sleep over it is like, okay, which things do we prioritise?
Do we try to make these players happy here?
Do we actually try to help these players stay in the game?
Or do we do what the game team wants to do, which is this feature
that they believe will make players’ lives better.
I just think like when it comes to feedback, be it from community
or be it from players, or be it people from the team.
I found that very often you have to go through the veneer of what they're saying.
Like, I don't like this character.
This character feels weak,
and then you have to kind of
decode that message and you have to ask, why is it that it feels weak?
Is it, you know, the way that the tool works? Is it something else?
Is it the balancing? or is it just the aesthetics?
Could be the sound like we just... the weapon doesn't sound right.
And that's why it feels weak.
And so I think a lot of
what my job comes down to is trying to actually understand
what people mean behind all of these messages
and which dial to turn and how to change content, how to rebound stuff,
because it might be that they're complaining about one thing,
but actually it's a completely different source of the problem.
I think something we've spoken about consistently is how much of a team effort
it is to reduce this kind of feedback into possible features, roadmap adjustments.
And also it's important to remember in our context,
we are also a children's game.
So there's a lot of a lot of audience
we're never going to hear the voices of in an online context.
It's just not going to happen.
So sifting that kind of feedback for me is really easy.
It's not easy necessarily to understand the root of their problem,
but it's definitely easy to reduce it down into what the possible pin point is.
I'm a massive fan of something called Jobs to Be Done Theory, which is that
Henry Ford idea of people don't know what they want or what they don't want.
They know how something makes them feel.
So at least from a community perspective, I can say, for example,
players are not having as much fun on this particular thing at the moment
and then perhaps analytics could see what the adoption is of that particular thing.
Perhaps we can see how our marketing messages have landed from it,
from a sentiment perspective, we can check and see if there's any like,
you know, root are there bugs with it or is it simply how it functions?
Could we try testing sound changes?
And it's it's as a group effort,
you finally figure out what that experience might be.
Okay so, you know, we’re heading towards
main launch, we've got across several hurdles to get here.
It’s maybe that week beforehand.
How are you preparing for the storm ahead that you know is coming
when suddenly the lights turn on and everyone starts pouring. In that first week
before we get to the launch itself, what are you the most focused on?
Hopefully, if you're at your last week, you're doing nothing
because you're just like
everything is submitted and ready and then you’re just waiting to press the button.
But what you have done in before that is
you try to hack your own game.
We try that, yeah.
Especially in complex multiplayer games.
We try to hack ourselves.
So we make sure that’s stable, we try to DDOS ourselves.
So we storm our own servers with hundreds of thousands of bot accounts
and we basically use our own servers to attack our own servers
by just like playing the game.
So too many games in the past have fallen over, had a great launch, great success.
And what you're doing is looking at the queue or looking at the
excuse me, the server is offline.
And that's that's because they didn't anticipate the popularity
or they didn't anticipate the success that it would have.
They Oh you actually having 10 million active users right now
and 9 million of them
cannot be fit on your servers because you didn't scale them enough.
So so the word for that is load testing.
We do it months before
to prepare for that scale.
And so we turn all these servers on.
And in anticipation of the players,
you have your community guidelines, you know how to compensate players
if something bad happens, how to ban players who are really going
to break your rules in a massive way, how to tackle hacking.
So hopefully all of this is prepped and you're ready to accept.
It's launch day, right.
I’m gonna start with you, Adam.
You know, what's what's going on on launch day?
What is it you're doing? Yeah.
So so if I was my game lead would be at 4 a.m.
looking at stats.
Yeah, I myself
I'm looking at discord and Twitter and playing the game
and seeing how it is as a, as an actual player.
And then hopefully trying to put some distance
between what's happening right now because a lot of noise.
Yeah.
And a lot of the analytics are not reliable yet.
Community sentiment is all over the place.
We don't know yet what's going to happen.
So I'm trying to build the next great thing
so that when they ask for this new addition,
hopefully it will align with what
I've been making in my little workshop and trying to get ahead.
Because the moment you launch, even in soft launch or global launch,
you put yourself on a timer very often with a live product
and yeah Alex has had his head probably done in, like what
the battle pass three months from now? What is going to go in it?
Have we tested it? Is the art ready. Is the animations ready?
Do we have a build that's going to come and be included?
So there's a lot of prep work.
This is basically you have a clock over your head the moment that happens
and you have to continue to try to stay ahead of it.
Yeah, it feels like maybe on the on the path to launch,
launch feels like this glorious end of the tunnel.
And then when you get there, you find out there's just loads more tunnel, you know.
There's two tunnels now. There is the one that you're trying
to like make the game better
and the other ones you trying to keep whatever is there alive.
In that initial launch window,
if you are looking at millions
and millions, millions, there is an element of knowing
it's going to be a while until you see proper, quantifiable, actionable feedback.
So like usually at launch it goes like this.
I'll be sitting there and I'd be like, Wow, it's going really well.
And then in about 20 minutes I'll go, Oh no, it's going terribly.
And then another 20 minutes, I’ll go actually it's great.
And you'll go like that for 24 hours as that floodgate
kind of goes through the same motions of getting a new game and playing with it,
I will be lurking constantly on all those channels, and if I see things
that are clearly quantifiable bugs, I will prioritise it.
How many people are affected?
We have clear guidelines for that kind of thing at launch.
Usually when you're talking high severity, you're talking is it playable?
So then other stuff, you know, you follow a process and put that,
you know, to prioritise post-launch, either patches or a future release.
I'm usually surprised by balancing of things
because the things I find completely like broken
people think they're weak, for example, or the other way around,
or they cling to this one unit and like, why are you using this?
You have better things available.
Yeah, but they just grown in a sentiment in the community that this is super strong
and that's what they'll cling to.
So that's always the surprises me.
Also the ways in which community finds ways to break the game.
That's always surprising.
And like something like, hey, we've coordinated 18 people
to like try and score the most and like we've scored 50,000 points in a game.
Was you supposed to score a thousand. Like, how did how did they do that?
So, seeing how creative people are,
that's always a surprise because they think about things that even
the most seasoned quality assurance veterans have not thought of.
Yeah, we have one superpower.
However, in the mobile world is very often we will sneakily release a game under
a different name in a different country like everyone does it.
Um, and that will be our alpha and nobody will know it exists.
And hopefully, I mean, if it has a huge brand like Warhammer: Total Way, like
it’s no way we can like redress it and call it something different and then try it.
But on mobile we can if you're not like super obviously
branded, you can if you're doing a new IP, you can test it in a small territory.
You can see who is actually clicking,
who's actually playing, who's actually engaging.
And that will-- you can do it very early on like months into the development,
six, eight, nine months in and then you can learn a lot
and hopefully inform your actual release.
That's a superpower in the mobile space.
It's much difficult to do on console and PC.
I wouldn't expect a designer to sit there and look through lines and lines
and lines and lines of in-game data telling people about their exact
degree movements or every single purchase that's ever made.
You wouldn't expect them to do that.
That is tons of raw data that needs to be shaped and understood and transformed
into an insight that to Bee’s point is actionable and useful and contextualised.
The exact same thing applies for any sentiment,
reporting or any discursive channel. You are going to see lots
of uncontextualised, unfettered, raw data that takes forms in spicy tweets.
So being able
to contextualise that and not trying to do that in isolation
because you wouldn't be expected to do that with any other form of data.
And just because it's readily available in your personal life,
it doesn't mean that it’s then any more valid than this person purchase this one thing.
Therefore we should make millions of it. Yeah. Yeah.