CARTER MORGAN: Great news.
Critter Junction's game just got featured
as game of the century on every app store.
Let's look at how Critter Junction
is going to handle all of this new traffic.
Stay tuned.
[MUSIC PLAYING]
Critter Junction has done a lot of great work
to be scalable, resilient, and highly available.
But their traffic is set to increase exponentially,
and they're not sure their current infrastructure can
meet those needs.
The biggest problem is that Critter Junction's traffic
has consistent spikes as people all over the world log on when
they're in the mood to play with a virtual pet
with their friends.
To accommodate this traffic, Critter Junction
allocated more resources but was burning money
during non-busy hours.
Still, that was better than the alternative--
risking service outages by not allocating enough resources.
Luckily, when Critter Junction switched
to a microservice architecture, they
decided to host their application on Google
Kubernetes Engine, and so they got
to use two Kubernetes features that are designed
to solve exactly this--
horizontal pod and cluster autoscaling.
In Kubernetes, you can think of a pod as an application
instance.
It has one or more containers that
are coupled together and treated as one unit.
The horizontal pod autoscaler automatically adds or removes
pods based on the usage rate of your application.
So if a lot of people are currently using your app
and its computational needs go up, it'll add pods for you.
And if that usage rate drops, then autoscaler
will automatically remove pods.
This happens based on CPU and memory metrics by default,
but it's also possible to use custom metrics.
This is great for handling more requests
when you have enough space to run more pods on.
But if there's no space to run additional pods, then
even if the horizontal pod autoscaler wants to add more,
out of luck, unless you're using cluster autoscaler.
What this does is it adds more resources
to your clusters-- in this case, machines that pods can run on.
We added these features to Critter Junction's
architecture, which did help alleviate some of the problems.
But as we researched more, we realized that in many cases
the application would run out of memory under a heavy load.
And there wasn't a really good way
to estimate how much memory was needed at any given time,
so we took a different approach.
Cluster autoscaler and horizontal pod autoscaling
are both examples of horizontal scaling,
or scaling out, which means they focus
on adding more total resources for the application
to use as a whole.
But sometimes it's necessary to scale vertically, or scaling
up, by modifying the resources provisioned
to an individual service, like if a pod doesn't
have enough CPU or if it has too much RAM.
That's where GKE's vertical pod autoscaler comes in.
Vertical pod autoscaler exists to automatically set
up-to-date resource limits and requests for the containers
in their pods.
It can both downscale pods that are over-requesting resources
and also upscale pods that are under-requesting resources
based on their usage over time.
This seemed like a good fit for Critter Junction's problem,
so we helped them enable the vertical pod autoscaler
on their cluster.
Then, to get a good baseline for CPU and memory resources,
we used vertical pod autoscaler's
sizing recommendations by creating a vertical pod
autoscaler whose update mode was set to off.
Finally, we updated the deployment
to account for these recommendations,
and Critter Junction was able to run a lot more efficiently.
Guessing what resources your app needs
is tricky, but by leveraging horizontal and vertical
autoscaling, Critter Junction was
able to better manage their workloads,
become more resilient, and save money
on over-allocating resources.
All that was left to do was celebrate
their well-earned traffic boost.
So that's a wrap for this episode of "Season of Scale."
Stay tuned to learn how Critter Junction continues
to improve the design of their app architecture and dev
processes.
And, remember, always be architecting.
[MUSIC PLAYING]