Hello and welcome back to another exciting episode
of Game On!, our show for game developers.
I'm your host Todd Kerpelman.
I know it's been awhile since our last episode
and I've missed you.
Did you do something different with your hair?
It looks nice.
Anyway, this episode is all about not
shipping broken games.
Or I guess to put a more positive spin on it,
testing your game for fun and profit.
Mostly profit.
And it's all done through the magic of using Google Play's
alpha and beta services to get your game out
in front of a large audience before you publish.
So I'd like to start today by telling you
a little story about Don.
See Don is a game developer and he just
released the very first version of his hotly anticipated game
to the world.
He was so excited, he could barely sleep that night.
And so when he woke up the next morning, the first thing he did
was go out and check his apps review page, which
looked a little something like this.
Huh?
What happened?
Well, Don did some digging around
and he found out that due to a font rendering error,
the game crashed for any player who had an E in their name.
Well Don immediately got together
with his team members, Alicia Winkowski, Irwin Lin, and Chris
Fulbright and said, how did we miss this?
How could we have possibly overlooked something
as completely obvious as-- OK, wait I think I see it.
None of us have an E in our name.
That's kind of a problem.
That's when he thought, huh, maybe we
should have tested this with a bigger group of people
before we launched this thing.
But of course, that sets up its own share of problems, right?
How are you going to be sharing your game
with several dozen friends and family members?
Are you going to be mailing APKs around?
How are you going to make sure that only these people have
access?
How are you going to make sure that they're
looking at the latest version?
Do you have to mail them and bug them each time you update
and hope that they then download and resideload your APK?
Come to think of it, how are you going
to teach your nontechnical users how
to sideload an APK in the first place?
It can be a bit of a mess.
And that brings us to one of the best features of the Play Store
publishing process that some of you
are overlooking, which frankly is a real shame because it
was built for just this situation.
And that is our Alpha Beta Staged Rollout process.
Also sometimes known as Alpha Beta
Canary because that makes for a better looking acronym.
This lets you easily test out new apps and app updates
with large groups of people before you
publish to the world.
So let's see what this process looks like.
Now in the past you're probably used
to just uploading your APK into production,
where it gets published to the Play Store
and is available to the world.
And hurray, there's pixie dust.
Or I suppose, if things go badly, uh, oh my god.
Everything's on fire, run.
One of the two.
Not sure.
But you don't have to do that.
So hang on their buddy, stay.
You see, the Developer Console also
lets you specify a certain group of people
to be your alpha testers.
This could be a Google group or a Google+ community.
For example, for my first group of alpha testers,
I might want this just to be the employees in my organization.
So I could publish-- so I could define my alpha testers
to be this group and then publish my APK just to them.
Now assuming that all goes well and we haven't
found any major bugs, I could then
create a second group of beta testers, maybe people
outside my company.
And again, publish my APK just to them next.
Kind of similar process as before.
Now, as for how this looks to your testers,
you're just going to send them a link to a page where they
can click on a button to become a tester.
And once they do that, they can download the game
from the Play Store just as if it
were a completely normal app.
No weird sideloading involved.
Access is completely handled by your group settings.
You can leave it open or invite only, however you want.
And all those concerns Don had, they're basically solved.
So where do you get these testers?
Well, here's one fun example.
The developers of CastleStorm promoted their Android program
right on their site.
So when you went and visited their website,
you would see this page, click on this little green Play
Now button, took you to their Google group,
which was open for anybody to join.
And they got a ton of great beta testers this way.
But don't forget about your Google+ communities.
These are often highly engaged fans of your game.
And they'd be really good candidates for beta testers.
Not only for any new updates to existing games,
but also for any new product that you release.
And by the way, this is a fantastic way
to connect with your community.
Getting early feedback on new games or new updates
from your most dedicated fans will provide you
with some fantastic insight from people who quite frankly know
your game probably better than you.
And it also makes them part of the process and a collaborator
in your game.
And then they can become your biggest supporters
and your biggest advocates in the outside world.
Your players, they want your games to be awesome.
They're on your side.
Don't be afraid to engage with them.
However you do it, assuming everything goes well
with your beta test group, you're
going to push wrap up to production.
And then the world can finally play your game.
Now this is all great, but what happens
if you want to update your game?
Maybe you've got a round of bug fixes,
but you want to make sure this bug fix doesn't accidentally
create a whole bunch of bigger bugs in the process.
It's been known to happen to some game developers.
Quite possibly ones in this room giving presentations right now.
So as you might expect, alpha and beta
works similar for game updates as well.
I can upload a new version of my game
to the alpha group and all my alpha testers,
they'll be able to grab the latest version I have just
as if the app have been updated normally for them in the Play
Store.
In fact, in many situations, your alpha testers
will just get the update automatically.
And then from there, I promote my new version of my app
to beta, have my beta testers try it out and then push it
on to production, right?
Well, wait, stop, stay there one second.
See this time around, for updates to an existing app,
there's a shocking twist.
Yes, that's right.
Even our stock image photography model is surprised.
You see, instead of just updating my app fully
on production, I can do something
that's called a Staged Rollout.
What this lets me do is push out the new version of my game
to just a small percentage of the market.
This is a great way to make extra sure
your game is going to work well with a larger audience.
It also turns out is a nice way to see
what people who aren't your friends and family
might think of your game.
Turns out your mom might not actually
be your most objective customer.
Sorry, Mom.
So what this means is I can have Version 2 of my game
out to a small fraction of the world while having
Version 1 of my game available to everybody else.
And then as time goes on and I'm feeling
fairly confident about this update,
I can expand that rollout more and more and more
until the whole world is finally ready to see
the update to my game.
And the nice thing is that you can
see your reviews, your crashes, and your other stats broken out
by application version right there in the Developer Console.
So if your players hate your update,
you can see that right away in the reviews.
More importantly, you can also set up your own analytics
to be broken out by version number.
So you can see if there are any major problems with retention
or ARPDAU or ARPU or any other custom
metrics you happen to be tracking.
For instance, if we find out that nobody is completing
the tutorial in our v2 release, well that's
a sign that something is really wrong there.
And we can fix that before we push this update out
to the entire population.
So this all sounds great right?
Well, with that said, let me address three responses
I've heard from developers around Alpha Beta testing.
The first is, well I'm not sure I
want to use this because I don't want to miss the chance
to be a "Top Recent" game.
And let me explain to you what the concern is here.
So imagine that you spent a lot of time with your game in Alpha
and then Beta test stages.
Then you're game hits production here
and that's when it takes off.
Well, do we qualify as the "Top Recent" game?
We have spent a couple of months in Alpha and Beta.
And it's actually true.
We used to start the "Top Recent" time here,
when your game first pushed to Alpha, which I think we all
agree makes no sense.
So you'll be happy to know we now
start the "Top Recent" timer here,
once your game has been pushed to production.
So if you hear of anybody who's still afraid to Alpha test
because they're worried they're going to miss their chance
to be on the "Top Recent" list, let
them know we've addressed this issue.
They're working on old info and they
don't need to be concerned about this anymore.
And in fact, what I'm also excited to tell you-- and go
grab your business analyst person
and tell them to watch this part--
is that we are now calculating the "Top Recent" list
per country.
So if you want to launch your product in say, Canada
first and then launch two weeks later in the US,
your "Top Recent" window in the US
will be determined by the time you first
launch there and not in Canada.
And so you can launch in different countries
at different times and not worry about missing
the "Top Recent" window elsewhere.
See, your business analyst person, they're really excited
now aren't they?
They're glad you grabbed them.
So a second question I sometimes get
is how do you add achievements or leaderboards or game
services-- other game services-- to a version of the game that's
in beta?
And it turns out these days it's pretty easy.
When you create a new leaderboard or achievement
or quest or event or what have you in Play game services,
it remains in an unpublished state on our servers
that's only visible to your testers.
But if you go into the testers portion of the Play Services
Developer Console, right here, you
can see that you can easily add alpha or beta testers as part
of the group that has access to these unpublished features
on our servers.
So if you're new APK has say a new leaderboard and five
new achievements, you can simply publish that updated client
to your Alpha group.
And then, give those same alpha testers access
to this new achievement and the new leaderboard
and achievements on the server.
Your testers get to test these new services out
and then the general public will see this mystery leaderboard
or these mystery achievements that they
can't seem to access or unlock.
A third question I sometimes get is, OK,
so is this what I should be using for A/B testing?
And my answer is kind of, sort of, but not really, it depends.
Don't be confused by these A/B letters.
Alpha Beta testing is different than the split testing people
sometimes refer to as A/B testing.
The point of Alpha Beta Canary testing
is really to find out if there are
major problems with your game before you launch.
Now you could certainly use it to find out
if you've accidentally broken your in-game economy
or if some new feature has improved retention.
But if you're looking to A/B test small interface changes
or gameplay tweaks or the wording
that you've put in your purchase dialogue,
it's not really best for that.
You don't want your users constantly downloading new APKs
and using up their data plan every time
you want to test some minor gameplay tweak.
So instead, I highly recommend looking
at tools like Google Tag Manager, which basically allow
you to configure your game and run all the A/B testing you
want in the cloud.
Google Tag Manager is a really great tool.
I am a fan, but I probably don't have time
to talk about it here.
I'll have to cover that in a future episode.
So with all of that in mind, Don, our intrepid game
developer, got into his handy dandy time machine
and went back in time to the week before his game launched.
This time, instead of just testing his game in house,
he decided to launch it on the beta channel to several dozen
of his closest friends.
And that's where he found and fixed several other bugs
that got missed by his three person team.
With those bugs fixed, he launched his game
to great fanfare.
The reviews were way up.
And just a few months later, his company
was acquired for $12 billion.
Not because of the game though, but because of the time
machine.
Turns out those things, they're really valuable.
Anyway, if you are not in the time machine business,
but you are in the game business,
I'm hoping I've convinced you at this point
that Alpha Beta Canary testing is
something you should be doing.
It's generally pretty easy to get started with.
It's all part of the Play Developer Console.
But if you want to see a walkthrough
or see me record a screen capture or something,
let me know in the comments.
I'll do that.
So that's it for this Game On! episode on testing your app.
Thanks for tuning in.
Test your games with your family and friends and fans
and let's start seeing some great games out there.
Bye.