[MUSIC PLAYING]
TODD KERPELMAN: Hi, everybody.
Thanks for joining me virtually in this online extravaganza.
So I know the original title of our talk
was what's new in Firebase for back-end game developers.
But, hey, spoiler alert, we're going
to talk about Cloud Firestore-- what it is,
why you should care, and maybe why
that database engineer next you just got really excited.
So as you may have heard, Firebase
has a number of tools to make it easier for you to get
your game up and running.
The general idea here is, hey, why
don't you let us take care of the boring infrastructure work?
And you can focus on other tasks like making
your game fun, which, you know, probably
should be your top priority.
So we provide services like Firebase authentication,
which makes it super easy for your user
to sign into your game from a variety of different providers.
We have Firebase hosting.
The idea with this is we can make it easy
for you to get a website up and running quickly
on a global CDN that's SSL by default
and all that great stuff.
And this is really nice if you have, say, a one-off marketing
site for a game that you want to get hosted really quickly.
We have Cloud Storage, which is a powerful, yet simple, service
for storing large binary objects.
And this could be user generated content,
like screenshots or videos, or content you provide, say,
chunks of post-install game data.
There's Cloud Functions for Firebase,
which lets you essentially run back-end code without needing
to stand up any servers on your own.
And then we have our databases for storing data in the cloud.
Now, the Realtime Database is the one many developers
have heard about.
This is Firebase's flagship product
back when it was its own independent startup.
And it is still a great product.
The idea here is that, if the user changes the value on one
device, our SDKs in servers automatically
update that change on every other device that's
listening to that value.
And yes, I should note that realtime here
means something a little different than what you game
developers are used to.
It's a few hundred milliseconds of latency, so probably not
suitable for powering your first person shooter.
But you totally could use it to drive, say, a bingo game.
In fact, here's a fun fact.
The Realtime Database was used to power Loteria.
This is the bingo-like game that ran as a Google Doodle.
This ran on the Google front page in over a dozen countries.
And the Realtime Database took care
of the matchmaking, the calling out the squares,
showing your opponent's moves in realtime
and declaring a winner.
And what's really neat is this game
went from zero players to millions
literally overnight when we started displaying it
on the Google front page.
Turns out Google gets a lot of traffic, who knew?
And the Realtime Database just kind of handled it all.
There were no panicked calls to engineers or anything.
And in fact, we have a really nice blog post
going into detail around how the engineering team did all this.
And I encourage you to read it for more information.
And that's nice.
But for many native app and web developers,
we've introduced the next generation
of the Realtime Database, a product known
as Cloud Firestore.
And all in all, it's been a really well-received product
except for the fact that there are no client libraries for C+
or Unity until now, or I guess a few weeks ago when we announced
that it's available as an Alpha release.
And I am going to talk more about that Alpha tag
in a minute, because that's kind of important.
Now, maybe some of you are super excited about this already
and are running off to build your own composite indexes
or what have you.
But for the rest of you who aren't big database nerds,
you might be asking yourself, well, I
guess this is good news and all.
But what does this really mean?
And why should I care?
So let's back up and explain.
Cloud Firestore, like the Realtime Database,
is part of a family of databases known as NoSQL databases.
And if you want the overly simplified
version of what that means, well,
think about a traditional database.
You have a bunch of tables, each of which
stores types of highly structured data.
And you can join these bits of data together using
a query language called SQL.
And you can do a lot of fancy stuff here, right?
You can merge in different pieces of data
or calculated pieces of data from all sorts
of different tables.
And while that's pretty neat, performance
can be quite variable.
This query could take a long time or a short amount of time,
depending on how much data I have, how it's set up,
and what kind of query I'm asking for.
Now, NoSQL databases are different
in a few important ways.
One thing you'll notice is that they
tend to be a little more loosey goosey with their data.
Officially, this is known as being schemaless.
And it's often seen as a good thing.
It gives developers the ability to iterate
a lot faster on their underlying data
without having to worry about sort
of backfilling every change they make to all their older
records.
It also gives you the ability to store data that's similar,
but not exactly alike.
You can see here I'm storing a bunch of characters
together in the same collection.
And yeah, sure, they're similar.
But I can go ahead and add a spell book for my mage,
and a battle cry for my fighter, and I
set a lock picks for my thief.
And our database isn't going to complain
that our data structure isn't exactly alike
from one character to the next.
It can just kind handle this.
But I think the biggest difference with a NoSQL
database is the fact that there is no SQL.
Queries tend to be a lot simpler.
You don't get sort of all those fancy joins that
merge all that data or calculated data together.
And this means you'll often see things
that look kind of strange or unusual at first,
like having copies of the same data exists in multiple places.
Now, some of these quirks, particularly this concept here
which is known as d normalized data, kind of freaked
out longtime SQL Developers at first.
But in the long run, there are some really nice trade-offs
that make some of this weirdness worth it.
For starters, NoSQL databases can scale horizontally.
And this is really nice.
This basically means that, as your data set gets
bigger and bigger, you can just kind of throw
more and more machines at it.
And it will go ahead and simply serve its data across all
these different machines.
And that's really helpful in a cloud hosted
environment, like, I don't know, say just to pick one,
the Google Cloud Platform, where we have tons of servers
we can just flip on and off as your database needs it.
Your database simply grows and expands
onto these different servers kind of behind the scenes
and without your even knowing that we're
doing all this work for you.
By contrast, traditional SQL databases
tend to scale vertically.
This means that as your data set grows,
you need to find bigger and bigger computers to host them
on.
And that can be problematic.
For one thing, at some point, you're
probably going to run out of massive supercomputers
to run your data set on.
And it also means that every time you
need to switch out to a bigger and faster supercomputer
there's probably going to be some downtime.
And often with NoSQL databases, you
will find that performance is better.
In fact, one thing we are fond of saying in the Cloud
Firestore world is that queries scale
to the size of the result set, not
the size of the underlying data set.
And what does that mean?
That means that, if, say, you had a game where
you were storing user generated puzzles in Cloud Firestore
and you asked the database, hey, send me the top 10
active levels in the United States
sorted by rating, that query would take
the same amount of time whether you had to search through 300
records or 300 million, right?
No matter how large that underlying data set gets,
this query will basically take the same amount of time.
And that's pretty great, because it means you generally
don't need to worry about having to restructure
your database for performance reasons
as your game gets more and more popular.
So let's go over a few other features
that makes Cloud Firestore generally
enjoyable to work with.
For starters, the data structure model
tends to be pretty nice to work with, particularly
compared to something like the Realtime Database
or other NoSQL databases.
All of your data tends to be stored
in documents and collections.
Now, documents you can think of as, well,
documents that contain key value pairs.
And the values of these things can
be anything from arrays to strings
to floats to JSONy looking objects to even
small amounts of binary data.
Now, these documents can be up to 1 megabyte of data
for each document, which actually is pretty large given
that you're mostly dealing with text and numbers and JSONy
things here.
Now, these documents are stored in collections,
which are, as you might suspect, collections of documents.
And these documents can and often
do point to sub-collections that contain other documents, which
might even point to other sub-collections,
and so on and so forth.
Now, one important note here is that all queries are shallow,
meaning that you can grab a document without having
to worry about grabbing all the data underneath it.
And that means you can structure your data hierarchically
in a way that makes sense to you logically without having
to worry about sort of grabbing tons of unnecessary data
every time your user requests one of these documents
on the top here.
Cloud Firestore has really robust offline support
built into all the client libraries,
so the SDKs will cache your data automatically
as you grab it from the server.
And then if you're user where to hop onto new airplane,
turn on airplane mode, they would still have access
to all the data that's cached on the device, which means they
can keep playing your game.
And you can go ahead and query any data that's
stored in your offline cache.
In addition, any writes that your players make, they
get queued up on the device.
And then as soon as you user goes back online,
they're delivered to your database in the order
that they were recorded.
So none of their gameplay session data gets lost.
Cloud Firestore has support for simple one-time fetches
of your data, which is probably what
you're used to if you're coming from a more
traditional database.
And a lot of developers start using Cloud Firestore this way,
but you can also add realtime listeners,
so that your client gets notified right away when
a value in a document changes.
And that's pretty neat, right?
You simply tell the library, hey,
I want to listen to the results of this query.
And Cloud Firestore can send you updates
when these values change, which means
you don't have to deal with any of that
pull to refresh nonsense.
Your new data just appears as soon
as it's available, which is great for situations
like multiplayer games when you want your game
state to be updated after a user has taken their turn.
Cloud Firestore has multi-region support,
which means that we can store your data
in different locations a few miles apart from each other.
And this is really nice from a reliability standpoint.
Because if, say, some kind of radioactive lizard,
or what have you, were to take an entire city offline,
your data would still be accessible
from these other regions.
But at the same time, it's strongly consistent,
which means that the data you're retrieving
is the latest version of the data that's
been written to the database sort of no matter which region
your client originally pinged.
And I'll be honest.
Having this sort of multi-region support while
also being strongly consistent is
this really amazing and difficult problem
that we've solved behind the scenes for you.
And honestly, I don't think people really
appreciate how cool this is.
So trust me when I say, hey, this is a pretty neat problem
we solved.
Cloud Firestore also has support for atomic transactions
if you want to make sure some other client isn't
messing with your data the same time you are.
I don't have of a slide for that,
but I know it's a thing people care about.
So I'm going to mention it here.
And because we're all making games
and we know we can't always trust our clients,
we also have support for Cloud Functions and security rules.
Now, security rules are rules that you
define that will reject requests from the client
either where your client is asking for data they shouldn't
be, or they're trying to write data that doesn't
look the way you expect.
But if you need more sophisticated controls,
you can use Cloud Functions.
These are serverless functions that
run in the cloud in an environment
that you control and trust.
And that means you can make sure this functionality is
consistent across all your clients.
And more importantly, you trust the changes
that they make to your database.
So I know we've talked a lot about Cloud Firestore,
our realtime capable NoSQL database that
lives in the cloud.
And you might be asking yourself, well, wait a minute.
How does this compare to the Firebase Realtime
Database, the realtime database that lives in the cloud?
And to be clear, you don't actually
need to choose one or the other.
You can totally use them both within the same project
depending on your needs.
But if you needed to kind of compare and contrast the two,
I generally like to think of Cloud Firestore as a heavy duty
version of the Realtime Database.
In general, I think it can handle more complex data
better.
This is kind of my personal opinion,
but I think the data model is easier to work with.
You have more powerful querying capabilities.
Your transactions are a little more sophisticated.
It scales even better than the Realtime Database.
And you have multi-region support for better reliability.
There are a few things that the Realtime Database does better,
though.
It tends to be somewhat faster if your customers are
located in North America and has built-in support for presence.
This is something a lot of game developers care about, sort
of having players find out if their friends are online.
Now, there are workarounds for this and Cloud Firestore,
but it's sort of not natively built in.
And it's not as elegant as what's
in the Realtime Database.
Also, the pricing model is different.
In the Realtime Database, your costs
are primarily based on the amount of data
that you're sending back and forth to your clients.
Cloud Firestore's costs are primarily
based on the number of read or write operations you perform.
So in general, the Realtime Database
is going to be a little more cost effective when you
have lots of small operations.
Think of a game like bingo or poker,
where your clients are very frequently
sending across little pieces of information,
like call, raise, or B14.
Cloud Firestore tends to be more cost
effective when you have smaller amounts of maybe
large operations.
Think about an asynchronous drawing game,
where you're probably sending across a large chunk of JSON
data that represents the player's drawing, or even
a turn-based word or chess game, where your players tend
to think for long periods of time before they take a turn.
They're not performing that many operations.
But I would also say don't get too fixated on price.
There's lots of ways Cloud Firestore can
make your life easier.
And you know, your engineering time costs money, too.
If it takes 10 extra engineering hours
to save yourself 18 bucks in server costs,
that's probably not the right trade-off.
Use your engineering time wisely.
These people tend to eat fancy snacks.
So how would you use Cloud Firestore in real life?
Let's look at a few potential real world scenarios.
One of the most simple use cases--
saving your player's game to the cloud, right?
Your users want to play their game on multiple devices.
And Cloud Firestore makes it really easy
to save your game state to the cloud.
Just write your player state into a document,
write that document to the database.
And any other client listening to that value
can grab the latest version and make
sure your player's progress is reflected everywhere
that they're playing your game.
I should also point out that storage in Cloud Firestore
is relatively inexpensive particularly compared
to the Realtime Database.
So you could go ahead and save a few versions back.
So that when you're a player frantically
calls customer support and they're like,
something is wrong with my game, you
have the option of restoring them to an earlier save game.
Another good use case for Cloud Firestore is shared game state.
Think about a turn-based word game or drawing game.
It's really easy, for starters, for your clients
to go ahead and query for games that they're participating in.
And then every time your player wants to take a turn,
you can just record that new game state
in this shared document that all the appropriate players
have access to.
And then thanks to the magic of realtime listeners,
those other player's clients can get updated
as soon as the game state is changed,
so they know it's their turn.
And if you want extra security, rather
than having your clients update this document directly,
you can have them relay their desired move
to a Cloud Function.
Remember, these are our serverless functions
that live in the cloud.
This Cloud Function can then validate your player's move,
make sure everything looks legit.
And then it can perform the work of actually updating
your shared game state, which then gets
sent to your other players.
And also, thanks to our integration
with Firebase Cloud Messaging, you
could send a notification to your other players.
So that even if they don't have your app open at the moment,
they can still get alerted that it's their turn.
Daily challenges, these are probably another good example.
These things are sort of decent chunks of content
that you're going to need to send to all your clients
every time they sign in.
I know there's a lot of role-playing games, puzzle
games, crossword puzzles, and so on that have daily challenges.
And this is data that could easily be stored and served
through Cloud Firestore.
User generated content could be a good use case, too.
Again, for large binary objects, like videos or screencasts,
I'd recommend using Cloud Storage.
But think about any kind of puzzle game or platformer
game where users are creating their own levels.
This is probably data that you're storing as JSON data.
And these could work nicely in Cloud Firestore.
And then, again, you could run queries like,
find all puzzles created in North America sorted by rating,
or find most recent puzzles, or find all puzzles
by a particular author.
But I think the most interesting use case politically
for a lot of larger and more established companies
is to use Cloud Firestore as a realtime globally
scalable caching layer on top of your current infrastructure.
Let's think about an airline for a second.
These airlines, they've got their 30-year-old legacy
servers with a bunch of business logic
and regulatory compliance built in.
They probably just can't swap everything over
to a brand new NoSQL database, right?
But at the same time, maybe they want a nice modern application
infrastructure that can handle millions of queries
and have realtime updates and nice offline support
and all that fancy stuff, right?
Well, what these businesses can do
is copy over some of their data to a Cloud Firestore instance.
That makes it really easy for these clients
to do searches, like let's find ticket prices
or get flight status information or that sort of thing,
and get those updates in real time
while leaving your legacy server to handle important things
like, say, purchasing a ticket, stuff that probably still needs
to go through that original server.
Now, if you've been watching this presentation,
I'm guessing you don't have an airline to run.
But maybe you do have an MMO, a collectible card game,
or a role playing game where you have a ton of data,
whether that's sort of gameplay balance settings, or player
profiles, or what have you, that your clients are going to want
to read in on a regular basis.
And again, that's a lot of clients that are constantly
hitting your servers.
So consider putting Cloud Firestore in front of that,
right?
Your clients can go ahead and query the data
that you've pushed to Cloud Firestore
while leaving your original server
free to handle important things, like player transactions
or user purchases.
Now, with all that being said, maybe you're
super excited to start building with Cloud Firestore,
but I do want to talk about that Alpha label here.
While the Cloud Firestore back-end service
has been around for quite a while
and is used in real production apps,
these new client libraries are very new
and haven't really been battle tested yet.
So expect bugs.
There will probably be some.
We know that some advanced features
are missing from the Unity SDK.
Some of the more sophisticated queries aren't available yet.
And things like numeric transforms
aren't supported yet.
And we fully expect that the API surface will be changing
over the next few months.
So you are going to have to rename
your methods at some point when you update your client
libraries.
So I would encourage you to start giving
these new libraries a try.
See how they would fit within your game's infrastructure.
And of course, send us feedback.
But maybe hold off on moving your entire business to a Cloud
Firestore based one until we're a few more months along.
So with that scary disclaimer out of the way,
I hope we've gotten you at least a little
interested in using Cloud Firestore in your latest game.
If you want to find out more about Cloud Firestore or any
of the other services we offer, head on over
to firebase.google.com/games and check it out.
And as always, thanks for watching.
[MUSIC PLAYING]