BUKOLA AYOCELE: So far, we've covered three strategies
for scaling your serverless apps.
Lucky for Critter Junction, we have three more up our sleeves.
Stay tuned.
[MUSIC PLAYING]
To review, Critter Junction's new wallet app
lets users send and receive virtual currency.
They're using Cloud Functions to provide response to queries
around wallet amounts.
But they hit their database back end rate limit,
so they couldn't experience the scale that serverless
provides, and wondered what other tips they were missing.
So my team came in and helped them set up instant scaling
limits on Cloud Functions, Cloud Tasks, to limit
the rate of work done, and stateful storage
to defer results from long running operations.
Let's explore three more tips.
#4, use Redis to rate limit requests.
Sometimes you need to perform rate
limiting in the context of the http request.
For example, you might want to protect your system
from abusive users with per user rate limits,
like 100 wallet queries per minute,
to control the flow of requests your serverless workloads.
Or you might need a back pressure
to the caller of your serverless workload,
like if your wallet transactions called
the API that check for fraud, which has a limit of 100
requests per second.
You need a defense measure to ensure all your serverless
functions respect that limit and prevent
the API from excessive use.
And Redis is great for doing this.
It gives you a very low latency key value datastore,
plus basic data structures and operations
that make it one of the preferred backing
source for rate limiting.
Google Cloud offers a fully managed Redis service
called Cloud Memory Store.
If you want to learn how to use Redis to limit cost function
invocations, check out the link below.
Tip 5.
Use Cloud Pub/Sub up to process the work in batches.
When you're dealing with a large number of messages,
you might not want to process every message.
A common pattern is to wait until messages
have accumulated.
Cloud Functions integrate with Cloud Pub/Sub in messaging
and ingestion middleware.
Instead of invoking a Cloud Function for every message,
serverless workloads can use Pub/Sub as a place
to accumulate batches of work, since the service will store
messages for up to seven days.
Then you can use Cloud Scheduler in managed cron job service
to process these items on schedule.
You can even trigger the batch processes
based on the number and age of accumulated messages.
This will help you control the flow,
and your functions can better handle increases
in trigger requests.
And finally, our last tip.
Use Cloud Run for heavily I/O work.
One of the most expensive components of infrastructure
is compute cycle.
When your serverless workload is just
waiting for a remote API call or waiting for files to read,
these are billable moments.
Cloud Run, a managed platform for running
stateless containers, lets you specify
how many concurrent requests, it can handle.
This can mean big increases in efficiency
for I/O bound workloads.
For example, if the work being done
spends most of its time waiting for replies
from slow remote API calls, Cloud Run
supports up to 80 concurrent requests
on the same instance, which shares the same CPU allocation.
What you want to do, is one, tune your service performance
for the maximum stable concurrency.
Two, set your expected level of concurrency
in any code level configuration and deploy your service.
Three, set Cloud Run concurrency equal to or less
than any code level configuration.
Four, then use low testing tools that
support a configurable concurrency,
and make sure that your service remains stable.
Critter Junction now has six strategies
for their serverless scaling in their back pocket.
They can use rate limiting, set instance scaling limits,
and products like Cloud Run, Cloud Task, Cloud Scheduler
and Cloud Pub/Sub to help manage load
on their serverless instance.
Even highly scalable systems should have limits
on consumption to reduce the chance of cascading failures
and maximize throughput.
For more details on these tips and an awesome decision tree,
check out the link below.
That's a wrap for a "Season of Scale."
Stay tuned for more strategies for scaling on Google Cloud.
And remember, always be architecting.