SPEAKER: Since leaping into the world of serverless,
Critter Junction's been reaching new limits.
But I've got a few tricks to take it up to the next level.
Stay tuned.
[MUSIC PLAYING]
A core promise of serverless compute platforms,
like Cloud Run and Cloud Functions,
is that you don't need to worry about your infrastructure.
Your service scales automatically.
That works great when your whole stack autoscales.
But what if your service depends on API
or databases with rate or connection limits?
That's what happened with Critter Junction's
new companion app code named the Wallet app.
It allows users to send and receive virtual currency
over a marketplace.
Every time a user sends or receives cash,
the change to the user Wallet data in the database
kicks off a Cloud Function to return a confirmation.
Cloud Function is a lightweight, serverless compute solution
that lets you create single purpose and standalone
functions that respond to cloud events.
And because serverless compute products are stateless,
they use Cloud SQL, a fully managed relational database
service for stateful data, like user Wallet amounts.
After the Wallet app gained popularity because of YouTube
live streamers, Cloud SQL's maximum connection limit
was reached.
And that meant the upstream functions
weren't able to provide knew Wallet amount confirmations.
So my team came in to give them a few tips
to scale the Wallet app and any future serverless
workloads they plan to deploy.
Let's review how serverless functions
scale as you take a function from your local computer
to the cloud.
A function takes inputs and provides an output response.
That function can be repeated with many inputs
and provide many outputs.
A serverless platform manages the elastic horizontal scaling
of function instances.
Because Google Cloud can give you near infinite scale,
that can have consequences for other systems
that your function interacts with.
So what do you do?
Well, a lot of what I'm about to tell you
involves applying limits to operations
to protect underlying services and resources.
First off, you can configure instance scaling limits
to your functions.
Cloud Functions offer max instance settings.
This feature limits how many concurrent instances
of your functions are running and attempting to establish
database connections.
So if your database can only handle
100 concurrent connections, you might set your max instances
to a lower value, say 75.
Since each instance of a function
can only handle a single request at a time,
this means you can only handle 75 concurrent requests
at a given time, saving your database from maxing out
at its rate limit even during a spike.
Second, use Cloud Tasks to limit the rate of work done.
Sometimes, the limit you're worried about
isn't the number of concurrent connections,
but the rate at which work is performed.
For example, imagine you need a call to an external API that
has limited per minute quota.
Cloud Tasks's a fully managed service
that lets you manage the execution, dispatch,
and delivery of distributed tasks.
You can manage the way work gets done
and perform the work outside of the serverless handlers in work
queues.
Cloud Tasks supports rate and concurrency limits.
So regardless of its rate at which work arrives,
it's performed with rates applied.
For example, if large influx of API requests come in,
Cloud Tasks can buffer the rate at which your request can
be handled by the API.
Third, use stateful storage to defer the results
from long-running operations.
Sometimes, you want your functions
to be able to defer work that might take a while,
but still give a response to the caller
that the task was received.
You want to send the results of the task to the caller
eventually, also known as long-running operations.
For example, it might not make sense
to encode a large video file inside a serverless instance.
You could use Cloud Tasks if the caller of your workload
only needs to know that their request was submitted.
But if you want the caller to be able to retrieve
a status or eventual result, you need
an additional stateful system to track that job.
There are several ways you can do this using
combination of Cloud Functions, Cloud Pub/Sub, and Firestore.
While Critter Junction has just started to explore serverless
with cloud native apps, like the layout and wallet apps,
they're constantly revisiting how to improve performance
and scalability, like using max serverless instances,
Cloud Tasks to limit the rate of work,
and stateful storage for long-running operations.
For more details on these tips and an awesome decision tree,
check out the link below.
And stay tuned for our next serverless scaling tips.
And remember, always be architecting.
[MUSIC PLAYING]