[MUSIC PLAYING]
SAMIR HAMMOUDI: Hi, everyone.
My name is Samir.
I'm a gaming solution architect at Google Cloud.
Today I will be introducing you to Open Match, an open source
matchmaking framework based on Kubernetes.
Then Ishikawa-san from Grenge will share about the experience
using Open Match in "Kick-Flight,"
a new 4v4 mobile game that just released last month.
So before diving in to Grenge's experience of Open Match,
let's first take a look at what exactly it is.
But to explain to you how it was founded,
I first need to explain the challenges that many developers
face with matchmaking services.
Matchmaking is one of the hardest unsolved problems
for online multiplayer games.
At a fundamental level, matchmaking takes players,
connect to your games, to match them up and connect them
to game servers.
Creating a good match is critical to your game's
success.
And what constitutes a good match changes from game
to game.
You have to consider attributes like skills, level, latency,
wait time.
And they are all critical to your game.
In general, making a good matchmaking service for a game
is hard, but it's core to your game design.
Running this as a global scale for your player population
presents a whole different set of challenges.
Architecting a matchmaking service
that can run at high skill, reliably, is very hard,
and may not be an interesting problem
to solve for a game developer.
This is exactly why Open Match was created.
So Open Match is an open source matchmaking framework
co-founded with Unity.
It leverages Kubernetes to provide flexibility,
scalability, and extensibility.
Just to give you an example of scale it can handle,
maybe you remember the Halloween Doodle from 2018,
that 4v4 multiplayer game you could play from the Google.com
website.
Yeah, that one was using Open Match to match players.
And it handled a concurrent peak of 500,000 players.
Because, with Open Match, you don't
need to worry about the scalability and reliability
of the underlying infrastructure,
you can focus on building the core logic of your matchmaker.
Because I think a picture is worth a thousand words,
let's take a look at the high-level architecture
of how online games will integrate with Open Match.
On the left side, you have a game front end
with your core game logic.
That calls into the matchmaking framework.
And at the same time, on the right side,
you have a pool of servers that players
can be placed on that's simultaneously calling
in the matchmaking logic.
Open Match sits in the middle, handling the players that
are available, splitting them by skill level, latency,
et cetera, and match them up with servers
that are available.
As a developer, you only care about developing the match
function, which contains the core logic of your matchmaker.
The left and right boxes are component from your game
that you need to integrate with Open Match.
Let's take a closer look at those components
in the left and right boxes.
The game front end is the endpoint where your game
clans will connect to.
To avoid cheating, you don't want them to directly
connect to the Open Match front end,
because only your game front end is capable of verifying
the data provided by the game clans
by accessing a database containing player's data.
So basically it's your game front end that
creates matchmaking requests--
also called tickets-- in Open Match.
On the right side, you find the game back end-- also
called director-- and the dedicated game servers.
Your game back end is a key component in the matchmaker
since it triggers the match function execution.
Once Open Match finds matches, it
will also handle the assignment of game service
to the match players.
This architecture is something you can then
reuse across your games as you build multiple games.
To have a better idea of how is the flow of a match,
let's take a look at the life of a match step by step.
First step, a player wants to play against other players.
The game clan makes a request to your game front end
to find a match.
It's the game front end that actually
makes the request to the Open Match front end
to find a match.
Every request are stored in the state store running on Redis.
As I mentioned in the previous slide,
it's the game back end that triggers the match function
using the Open Match back end.
The game back end requests for a match.
The Open Match back triggers the matchmaking function
that uses the query component to create Redis and find
candidates for a match.
Once a match is found, the Open Match back end
will return it to the game back in that will then
finalize the entire matchmaking process by assigning
a dedicated game server to the match players.
The last step of the life of a match is the assignment.
It's the last step where we assign a game
server to the matched players.
For this, the game back end, we allocate a dedicated game
server by using solutions like Agones or any other custom
server management system.
Once it gets the allocated game server IP and port,
it will reach out again to the Open Match back end
to update the tickets stored in the state store
with that information.
Finally, once tickets have been updated,
the game front end will push that game server details
to players, who will then connect to the server and just
start to play.
And, voila, that's how Open Match works in a nutshell.
Hope this was clear enough.
If you're building a small game, you
might not be interested in a complex, highly-scalable
matchmaking framework.
But you can still leverage Open Match
to write a simple matchmaking function that
will look like a single function in the language of your choice.
Plug that into the framework, and you'll
have a fully functional matchmaking service
without a lot of work.
But where Open Match really shines
is if you have a really large system with thousands
or millions of players that are all coming at the same time.
And the player population is segmented into different groups
and matched up very quickly.
So you might be running hundreds of concurrent matches
on the player population, looking for the best match
to make your game work.
So now that you know what Open Match is,
let's listen to the experience of the Japanese game studio
Grenge using Open Match in their latest mobile game,
"Kick-Flight."
YASUNORI ISHIKAWA: Hi, my name is Yasunori.
I'm an engineering manager at Grenge.
Here to talk about why and how we use Open Match
in "Kick-Flight."
Our experience is a game called "Kick-Flight."
"Kick-Flight" is a free-flying and midair battle action game
for smartphones.
Four-versus-four real-time multiplayer game.
You control a character called a kicker.
12 characters with three game modes.
Available in more than 130 countries.
Launched on February 13, 2020.
[ROCK MUSIC]
[GAME SOUND EFFECTS]
Here are "Kick-Flight"'s matchmaking requirements.
The video on the left shows the matchmaking using Open Match.
Using the kicker under Game Mode,
and by tapping the Battle button, matchmakings begin.
"Kick-Flight" will match two teams of four with
the following requirements--
match users with similar play skills.
Matching users with far different play skills
will result in a one-sided match.
Minimize character overlaps in battles.
Each of the characters has a role.
There are four types--
attack, support, tank, and speed.
Matchmaking is done so that the same character or role does not
suffer as much as possible.
You can cancel any time before matchmaking is found.
The progress is displayed in real time during matchmaking.
There is four reasons why we decided to use Open Match.
The first one is because we wanted
to make a general-purpose matchmaking framework,
because building a matchmaker for each game is pricey.
Building a general-purpose matchmaking
with a budget of a game is not possible.
As a result, each game has a different matchmaking
implementation, depending on the type of matchmaking.
Making a matchmaker game with one game app
can take anywhere from six months to a year.
Even in that case, there are many problems about scaling.
In "Kick-Flight", the development of matchmaking
logic was about one or two months.
And it took about one month for version updates and load tests.
Most importantly, this knowledge can
be used to build faster using Open Match in the future.
Secondly, because we wanted to focus on the matchmaking logic,
in general, scalability is a major issue in matchmaking.
This problem was minimized because Open Match
is, by design, scalable.
Third, because we could find working MM examples.
At that time, there wasn't much documentation,
but there were some working examples.
We did the source code, and got the help of Googlers.
The last one is because Open Match was used successfully
in the Halloween Doodle in 2018.
This is "Kick-Flight" architecture.
Except for Realtime Game Server, everything is built on GCP.
The central part is Open Match.
Creation and the addition of tickets
are performed from the API Server
through Cloud Load Balancer.
A creation of assignment is requested
to Open Match via Ingress and Endpoints.
So therefore, Open Match uses various parts on GKE.
We have changed that to Cloud Memorystore.
As a side note, we used Cloud Spanner as our main database.
In "Kick-Flight," there are three main components.
The API changes the kicker and enhances the skill
of a character, called a disc.
It is implemented using PHP7, and it uses Cloud Memorystore
for Redis for session cache and Cloud Spanner for player data.
Open Match's version is version 0.7,
with some customization added.
It handles players matchmaking, and matchmaking function
is also implemented in PHP7.
Dedicated game server handles the real-time aspect
of battles.
Let me show you what went well with Open Match.
We are able to focus on the development of the matchmaking
logic.
Open Match does the communication part
with the client and the scale.
So we are able to concentrate on the matchmaking logic, scaling
the system based on the number of players.
We can scale by increasing front end and the matchmaking
function.
Also easily modify the matchmaking criteria.
Since front end and match function are separated,
just modify the logic.
Then build and deploy the images.
We can deploy without any downtime.
The server management was simplified,
thanks to the managed services like GKE and the Cloud
Memorystore.
We used Cloud Endpoints to secure the access
to the Open Match front end.
It was very easy to use with GKE, because it
was made for Kubernetes.
By default, Redis in GKE is used.
But management is easier using crowd Memorystore.
This helped reduce operation cost.
Assignment communication is done via Cloud Endpoints.
Originally, it should be done via our game server front end.
But handling streaming connection in PHP is complex.
So we used Ingress and Cloud Endpoints to handle this part.
Finally, for monitoring, Open Match
telemetry worked seamlessly with Stackdriver.
Here are a few changes we faced with our game.
We launched our game using Open Match for the first time
in the world.
We used very early versions, from version 0.3,
of Open Match.
Back in version 0.3, there was not much documentation.
So we read source codes with help from Googlers.
Here is some tuning we have done on Open Match from
"Kick-Flight."
Parallel processing by splitting profile.
A match profile is Open Match's representation
of a match specification.
For example, processing is performed
in parallel for each game mode.
We adjusted the retry intervals.
If the retry interval is too short,
the load on the Redis server will increase.
Increasing this interval can reduce the load
on the Redis server.
We also adjusted the Redis TTLs.
Longer TTL will increase data volume.
The shorter the TTL, the lower the load on the Redis server.
We use C2 instances.
It is important to match in a short period of time,
as it will not be able to judge if the matchmaking
queue accumulates.
For that purpose, very fast processing
is realized using C2 instances.
For example, the difference between 1.0 seconds
and 0.8 seconds will make a big difference.
We reduced the filter condition of profile.
The load on increased according to the multiplication
of the learning profile and filter.
So we changed to minimum filter conditions.
We back-ported from version 0.8 because execution from back end
to function was not balanced.
In addition, we performed load test of communication path
using Locust.
Finally, let's look back at the game launch.
We are able to handle the load at the time of launch
flawlessly.
During our load test, we measured
Open Match could handle at least 250 entries per second.
We had no incident caused by Open Match so far.
We developed our own solution to increasing Redis data.
Open Match version 0.10 will include this fix.
In conclusion, we are very satisfied with Open Match.
We could accelerate the game development speed,
and we could provide a very fast yet scalable matchmaking
service to our players.
We plan to use Open Match much for our next games
and even expand it to other game studios in our group.
Lastly, if you are interested in Open Match,
I recommend looking at the website, openmatch.dev,
where you can find great documentation and many examples
to get started with.
I can't wait to see what you can come up with with Open
Match in your game.
[MUSIC PLAYING]