SPEAKER 1: Money.
They say it can't buy happiness, but it can buy jelly beans,
and that's almost the same thing.
So what do you need to know about how Cloud Firestore works
to make sure you're not spending too much money?
How is it different than Realtime database?
And did I just make that jelly beans
remark so I could be eating jelly beans in the next scene?
Let's find out on this episode of Get to Know Cloud Firestore.
[MUSIC PLAYING]
Wow, these are good.
Do you want one?
SPEAKER 2: Unh unh.
SPEAKER 1: They're delicious.
So I think a lot of folks coming in from Realtime Database land
might be wondering where the automatic, hey, just
migrate my entire database to Cloud Firestore button is.
And the truth is, we don't have that button,
and I don't know if we ever will.
And there's a couple of reasons for this.
One, thanks to a lot of the things I talked about
in the last video-- shallow queries,
and the ability to run queries across multiple fields--
you can structure your data in Cloud Firestore in a way that
probably makes a lot more sense logically,
and some of the workarounds you had in the Realtime Database
world, like combo fields or unnecessarily shallow trees,
aren't really needed anymore.
But two, the way these two databases
determine pricing is very different,
and if you take your entire app that's
optimized for one pricing model and sync it
into a completely different pricing model,
you might end up spending more than you needed,
and that means less jelly beans for everybody,
and that would be sad.
So let's talk about pricing.
And before I do, let me hit you up with a quick disclaimer
to make our lawyers happy.
[CLEARING THROAT]
(SPEAKING VERY QUICKLY) Please note
that all pricing guidelines and examples in this video
are used only for demonstration purposes,
and may not reflect your full and accurate Cloud
Firestore pricing.
Your app's needs are unique, and might not
benefit from recommendations laid out in this video.
Be sure to review the full documentation on the Cloud
Firestore website for a more complete sense of Cloud
Firestore pricing to best determine which pricing
strategies are right for you.
Wow, that was exciting.
So if you're coming here from Realtime Database land,
you're probably used to having most your costs be
proportional to the amount of data that you download
and upload to the database.
Cloud Firestore is different.
When it comes to pricing, it cares less
about the amount of data you download,
and more about the number of operations you perform.
More specifically, Firestore primarily charges
based on the number of reads, writes, and deletes
you perform in the database.
But what is a read or a write?
It seems obvious, but let's go into this a little further.
So writes are charged whenever you create or update
a document, and this holds true no matter how much of that
document you are changing.
Whether you're changing a single field from true to false,
or swapping out 30 different fields all at once,
it still just counts as one write.
A read occurs any time a client gets data from a document.
So if I say, hey, Cloud Firestore,
give me the top 20 Japanese restaurants in my area,
and Firestore gives me back 20 documents,
that will count as 20 reads.
And the nice thing here is only the documents
that are retrieved are counted.
If you remember from our last video,
Firestore does all of its searching
through indexes, not the documents themselves.
So even if I have 30 million restaurants to search
through, asking for the top 20 Japanese restaurants in my city
still only results in 20 reads.
On the other hand, if I were to say,
hey, show me all the restaurants in my city, that
could result in a boatload of reads,
and honestly probably more than I need.
That's one big reason why pagination is important.
And we will talk about that in a future video.
Also keep in mind that these reeds apply
to real time updates as well.
Every new update my client receives also counts as a read,
but only for the document that's changed.
For example, let's say I've got a real time listener set up
for these top 20 Japanese restaurants.
Now if one of these restaurants suddenly changes their name,
that new document would be sent to my client,
and that will count as a read, but it only
counts as one document read.
And this is true even though in my listener method,
I'll still get all 20 documents.
Firestore is smart enough to merge
that one new document alongside the cached data that
hasn't changed.
(BABY TALK) Ooh, who's a smart little database?
Yes, you are.
So that's all generally good news,
but let's look at a situation where things
might get a little pricier.
Suppose instead of my restaurant app,
I have a stock ticker app where I'm
updating the price of every stock on the backend every 15
seconds.
And imagine somebody has my app open in the foreground,
and they're following one single stock symbol.
Well if they keep their app open for three minutes,
that app will receive about 12 updates,
which counts as 12 reads.
And if that person is following 10 different stock symbols,
well, we're now up to 120 reads for that user's three minute
session.
And my gosh, if I have, like, 1,000 different users all using
my app at that time, well that's 120,000
reads right there just for those three minutes.
I mean, if I were to keep up that pace,
that averages out to like 57 and 1/2 million reads a day,
and-- wait, wait, are you freaking out?
Stop.
Calm down.
It's OK.
Breathe into a paper bag or something.
You see, I get it.
We're engineers here.
We're all about calculating the big O of things,
and seeing exactly how big stuff can scale.
And if we see that the number of reads scales
proportionally to, like, stocks times users, times updates,
that does scare us, because we can
see exactly how big this could get in a worst case scenario.
But things aren't always as bad as they first appear.
Let's go back to our stock app.
If we've got 1,000 different users
all using our app for a three minute session,
well that works out to roughly half
a million daily active users.
And those 57 million daily reads,
that's going to cost us about $34 a day.
And honestly, if you're paying $34
a day for an auto-syncing multi-region database
with half a million DAU, that's not so bad.
And there are clearly ways I could limit this
if I needed to, right.
I could reduce how often I update these stock prices,
make it a pull-to-refresh situation, or extreme cases--
maybe not this one --set up a caching layer.
So I don't want you to automatically panic
at the worst case scenario, but I
do want you to be aware of the cases in which Cloud
Firestore could make you panic.
As a general rule, I like to say that the more real-timey
and the more crowded an app feels,
the more it will probably cost in Cloud Firestore land.
So let's go back to our restaurant review app.
This is really about delivering large chunks
of data fairly infrequently.
As a user, I might run a search, and that
could bring in 20 documents, and maybe
I drill down to see details on one of these restaurants,
and I might fetch in the 10 most recent reviews.
But these documents, they're not going to be updated very often,
and these reads are going to be pretty infrequent because, you
know, even skimming reviews takes time.
Writes are also going to be pretty infrequent
because it takes a pretty long time to compose a restaurant
review.
Even if my app auto saved a draft every 30 seconds,
that's going to be pretty small potatoes
in the grand scheme of things.
So I think my restaurant app is probably
set up pretty nicely to take advantage
of Firestore's pricing model.
The same would go for anything like a blog, a wiki, or maybe
a news app.
Cloud Firestore might also work well
with a turn-based multi-player game.
If you think about it, a typical turn in a board game
usually takes a couple of minutes.
So using Cloud Firestore to power
a multi-player chess game or a word game
would probably be pretty reasonable.
Cromulent?
SPEAKER 2: Mm-hmm.
SPEAKER 1: I don't know if that's a real word.
Now on the other hand, imagine you've got a group whiteboard
app where lots of people all get to scribble
on the same canvas at the same time.
If you're sending tiny updates at the rate of several times
a second for that real time drawing feel,
you're going to generate a lot of writes
and a lot of reads per second, and this
might be one case where the real time database would
be better suited to the situation,
at least from a pricing standpoint.
OK, now what about a chat app?
Well this gets interesting because this
is something that could vary wildly
because the number of reads is very much affected
by the number of people in the same chat room.
Let's think about a one-on-one chat.
If we assume that every person sends a chat message
every 10 seconds, two people chatting for five minutes
will generate about 60 writes and 120 reads.
So if I take 100 people and group them
into 50 different one-on-one chats,
I'm looking at 3,000 writes, 6,000 reads,
and that's pretty reasonable.
So if you wanted to use Cloud Firestore to power
a person-to-person chat in a dating app,
or maybe let users chat with customer service
representatives, that would probably
work out pretty nicely.
On the other hand, what if we took those same 100 people
all still chatting away at the same rate,
and put them all in the same room where everybody
could hear each other?
Well now all 100 people get an update whenever
anybody writes a message.
So I still have those same 3,000 writes,
but now instead of 6,000 reads, I
have 300,000 reads, which is a pretty big jump.
But even that estimate is probably misleading,
because it turns out that when you
get into a chat room that crowded,
people tend not to chat at the same--
would you be quiet?
I am working on a video for YouTube.
As I was saying, people tend to not chat at the same rate
as before.
It just gets too noisy.
And if the main lesson you're getting out of all of this
is that it's hard to provide heuristics or guidelines
that work in every situation, hey, then congratulations,
you're getting it.
You really are going to have to look at your app's behavior
and figure it out from there.
Now while we're here let's talk about a few things that
might affect reads or writes that you might not expect.
First up is Cloud Functions.
As you know, these things can also read or write
to the database, and when they do, that gets billed.
Suppose I've got a Cloud Function that
updates my average score on my restaurant app
whenever somebody creates or updates a restaurant review.
Seems reasonable, but what if I decided
I was going to calculate the average rating by gathering up
all the reviews-- the reviews sub-collection,
reading in every review score, and then writing
the average back to the original restaurant document.
I'm hoping this is generating some alarm
bells in the back of your head, right, because this
is going to generate a whole bunch of extra reads
that you might not want to perform every single time
a user changes a review.
I think it would be better to use some clever math.
Like if you store an average review and a number of reviews
value on that restaurant document,
then you could change the average for a new review
without having to go back through all of the other review
documents.
Or if you really like the idea of writing a function that
looks at every review for a restaurant,
at least make it a Cron job that runs only once every few days.
Another thing to look out for is security rules.
They could generate extra reads in your database, too.
I know I haven't really talked about security rules yet.
That will be covered in a separate video.
But when it comes to evaluating whether or not
a certain action is permitted, sometimes you'll
need to check an arbitrary document in the database.
These are checks you'll usually be making with get
or exists calls, and these two calls typically
count as an extra read.
That said, Cloud Firestore is pretty
smart about caching these values when it can,
and in practice, security rules are probably
going to be much less of a concern than Cloud Functions
when it comes to pricing.
Sure an operation might trigger an extra reader or two
with security rules, but it won't scale dramatically
the way a badly written Cloud Function might.
This is definitely an area where you're better off
optimizing for security instead of getting
overly clever in some quest to minimize database reads.
In fact, as a general rule, I'd be careful about overly
optimizing for pricing.
While you definitely want to avoid
any boneheaded maneuvers that will drastically
drive up costs,
I've also seen some pretty convoluted schemes out there
where developers will spend hours creating overly complex
code that ends up saving them $0.30 a day.
Remember, your development and debugging time
costs money, too.
Don't be penny wise and pound foolish.
Sorry.
Rant over.
So let's look at a few other places
where Cloud Firestore can cost you money.
I've generally ignored bandwidth costs in this video,
and that's because while there can be a cost associated
with outbound network usage or what
we like to call network egress, which it turns out,
is not a type of bird, it's generally a very minor factor
in the grand scheme of things.
Like, I think you get the first 30 gigabytes of data for free,
and for most larger customers, this ends up
being like 4% of their bill.
So I tend not to worry about network usage too much.
Similarly there's a cost for storing data in the database,
and this includes not just your data, but also
the indexes and metadata that goes along with it.
But again, this is pretty cheap, roughly 41 times cheaper
than what you'd be paying in the real time database.
So in general, it tends to be not as much
of a factor in your pricing.
And this cheaper storage in fact means
you can look at doing things that
might have been too expensive to be practical in real time
database land.
For instance, every time a user decides
to update the review for a restaurant,
I could totally save that revision
as a completely new document, and keep the old ones around.
Then my user could go back and revisit every single revision
they've ever made.
Sure maybe that's a little overkill for a restaurant
review site, but it would be really useful in, like,
a blogging site.
Also, if I wanted to store icon or small profile
picture for every user, I could totally
do that right there in the database
without having to look at a separate service.
Remember, Cloud Firestore supports
storage of binary objects as values,
so I could keep a small JPEG or PNG file in there.
Keep in mind you're limited to one meg per document,
but you know what, for a small icon,
I think you'd be fine here.
So now that you know what you're going to be charged for,
let's talk about measuring and estimating costs,
because I think all of us worry, whether you're
on Cloud Firestore or any other cloud service,
or just running your own backend,
there's going to be some error in our code,
or a badly thought out architecture
that's going to end up flooding our service without our
realizing it.
So while it's certainly important
whether a service is cheap or expensive,
sometimes what matters more is that it's
measurable and predictable.
And I will say the nice thing about looking at reads
and writes as your main pricing factor is that estimating those
is a heck of a lot easier than estimating download sizes,
especially because you can limit the number of items
that get returned from a query.
So I can easily set up my app to return only the top 20
results when I perform a search, and then I
know exactly how much that search is going to cost.
It would be way weirder to ask it
to limit my results to the top 2 meg worth of restaurant data.
I don't even know what that would look like.
But outside of estimating costs, let's
take a look at how you can get details
on your actual real world usage, and get warnings
if things seem out of whack.
And I'll be honest here, this is an area where
I find things are a little rough around the edges,
and I wouldn't be surprised if you saw some nice improvements
over the next several months.
But let's say your app has been out for a while,
and you're curious if your reads and writes match up
with what you're expecting.
How do you find this out?
Well to get this data, you're going
to need to get added to Firebase console
and jump into the Google Cloud console.
You see, every Cloud Firestore database that you create
is attached to an app in your project, which
means that if you want to see your database usage,
you'll want to go to the Google Cloud console at this URL
here, and select App Engine, and then head over
to the Quotas page.
You'll be presented with a bunch of different usage stats.
The three you are most interested in
are these three here.
Firestore Read Operations, Firestore Entity Writes,
and Firestore Entity Deletes.
And then for storage costs, you're
going to want to look at Firestore Stored Data.
And this can give you a pretty good idea
of what your usage looks like.
If you click on History here, you'll
see a list of your usage costs over the last 90 days,
and if you click on any of these numbers,
you'll get the exact breakdown of your usage.
And then maybe more importantly over here
in App Engine Settings, you can set up
a daily limit on spending.
By setting this daily limit, if Firestore costs ever
get over this threshold, then the API
will start refusing requests.
Now admittedly, this isn't a thing
you want to have happen on a regular basis,
and so my recommendation here is set this
to a point where you're like, yeah,
if I ever reach this kind of level something's
definitely wrong.
Put the brakes on until I can fix it.
Then make it a habit of coming back to this page
every once in awhile.
Look at higher costs are doing historically,
and then adjust your limit to something
that might be more accurate based on your previous levels.
I would keep in mind that if you're on the Firebase Flame
Plan, which is the fixed price per month plan,
then it's essentially doing this already for you
and setting these daily quotas isn't necessary.
Sure, it's nice to look at the summary
to see what your actual usage is like,
but you can kind of ignore these costs since they're
covered by your flat fee.
But if you are on the Blaze Plan, which
is the pay as you go plan, another step
I strongly recommend is to give yourself a monthly budget
alert.
If you go to the billing section of the Google Cloud console,
you have this option here to create a budget.
Basically this is your way of telling Google, hey,
this is kind of the most I expect
to spend per month on all your Cloud Services,
so I want you to tell me if I start
to get close to this limit.
Now unlike the daily quota limits in App Engine,
this won't turn off anything if you exceed your budget,
but it will send you a somewhat frantic-sounding email
so that you can go and check out your usage
to see what's going on.
So because of this, I'd recommend setting your budget
much closer to what you might reasonably
expect to see in a busy month.
There's a lot less harm in hitting a false positive here.
Just remember, the App Engine spending limit
is a daily thing.
The budget alerts are a monthly thing.
Don't get them mixed up.
Also, these budget alerts cover all Google Cloud Services, not
just Firestore.
So your cloud storage, your Cloud Function costs
are also included in there, too.
Finally, another tool I would keep an eye on is Stackdriver.
It's a very cool little monitoring service
that can track your app's usage of a number of different Cloud
Services, both on Google Cloud and AWS.
You can set it up to present you some really nice-looking
graphs, and have it send you alerts
if things start to look off in a number of different areas.
For instance, we recently added support
for the real time database.
So now Stackdriver can tell me if the amount of data
I'm sending in the real time database
goes over a certain limit, or if it's getting
too many or too few requests.
It can let me know if my Cloud Functions are taking
too long to run, and it can let me
know if there's an unusual increase in any of these stats.
That said, there's not a lot you can do with it yet with Cloud
Firestore.
But I wouldn't be surprised if that changed in the future.
I'm not promising anything.
I have no idea what their roadmap is.
I'm just saying it wouldn't surprise me, the same way, say,
a second jar of jellybeans wouldn't surprise me.
Anyone?
OK, fine.
So there you go, kids, just about everything
you want to know, and probably some
stuff you didn't, about pricing in Cloud Firestore.
Like I said, refer to the docs for all the latest pricing
information, and keep me on our lawyer's good side.
He looks cute, but he can actually be pretty vicious.
And I'll see you soon on another episode of Get
to Know Cloud Firestore.
Cheers.
Ooh.
Oh, I got buttered popcorn.
That's a weird one.
Wanna trade?
[MUSIC PLAYING]