SPEAKER: When it comes to cost optimization,
you want to make sure that you're running your workloads
as efficiently as possible.
Of course, that definitely means not paying for resources
that you're not using.
Today, we'll talk about Google Kubernetes Engine's autoscaling
and go over some tips.
Let's go beyond your GKE bill.
[MUSIC PLAYING]
One major advantage of Kubernetes
is how it was built with scaling in mind.
And GKE adds even more options to scale.
It can work pretty well without much configuration,
but there's a lot of cost optimization
potential by going deeper.
In the last video, we talked a lot
about how to choose the right machine type.
As always, it really depends on knowing your application's
resource needs.
And that's just as important for configuring autoscaling.
I'll start with an introduction to autoscaling, and then dive
deep into horizontal and vertical autoscaling
for your workloads.
Starting with the basics, autoscaling is really
all about two things--
getting additional resources when demand increases
and shutting down or removing resources
when demand decreases.
By setting autoscaling up efficiently
and specific to your workload, you'll
be able to minimize waste and cost.
In order to understand how you should configure autoscaling,
let's simplify everything into workloads and infrastructure.
Workloads are made up of pods, which are running
containerized applications.
Infrastructure is made up of nodes,
which are responsible for the actual resources,
like computing capacity and memory.
For each of these, there's also two ways to scale--
horizontally or vertically.
Horizontal scaling is done by adding more pods or nodes.
If there's a lot of demand coming in,
the autoscaler can try to increase the number of pods
or nodes to handle it.
Vertical scaling is done by making pods or nodes bigger.
When demand increases, a larger pod or node
may be able to reduce the burden by handling more.
With those basics, let's talk about how
GKE handles autoscaling.
Let's start with the workload and pod size
by looking at the aptly named horizontal pod autoscaler
and vertical pod autoscaler.
For horizontal scaling, you'll need
a metric that represents the demand on your workload,
such as CPU utilization or requests per second.
With that target, the horizontal pod autoscaler
can try to add pods to get your workload back to your desired
state.
Adding pods should mean that the work is spread out evenly,
and a metric like average CPU utilization
would go down across your pods.
Similarly, if there's not much demand,
the autoscaler can remove pods, which
is great for saving on costs.
When using a horizontal pod autoscaler,
remember that there might be a lag
time depending on how long it takes your container
to start up.
Setting the correct amount of available metric space,
or buffer, requires knowing how well you can handle spikes
in demand.
If the buffer is too small and the pods take a long time
to spin up, your app might get overloaded.
On the other hand, too large of a buffer
can lead to wasted resources and extra costs.
So a few things to keep in mind.
Understand what capacity might be needed for a spike,
and set your buffer accordingly.
Do what you can to have your pods spin up quickly and shut
down efficiently.
Make sure to set up the right readiness
and live probes so GKE can make the best decisions.
And remember to make sure that your metric server
is always up and running.
I know we haven't talked about what some of these
mean, but keep them in mind, and we'll
come back to these when we dive deeper
into optimizing your applications.
So that's horizontal pod autoscaling.
Now, let's look at vertical pod autoscaling.
Rather than adding additional pod replicas,
here, the pods actually get recreated
with more or less resources, depending
on CPU and memory usage.
Remember that too few resources can overload your apps
and too many resources will mean that you're paying for things
that you're not using.
Vertical pod autoscaling works to solve this.
It has three modes--
off, initial, and auto.
When it's off, no autoscaling actually happens,
but the autoscaler is looking at your pods
and creating recommendations based on usage
that you can inspect, which also happens in the other two modes.
If it's set to initial, those recommendations
will be used to create resized pods as they're needed.
After creating a pod at the recommended size,
based on those recommendations, it won't change the pod size
afterward.
If the autoscaler is set to auto,
those recommendations will be used to regularly resize pods
by deleting them and recreating them with the adjusted amounts.
In general, you should make sure that vertical pod autoscaling
is off for at least a day, or ideally a week,
in order to collect recommendation data.
Then, you can switch to initial or auto based on your needs,
or you can just keep it off and review the recommendations
to manually adjust your pods.
Here's some things to keep in mind.
If you're looking to handle sudden spikes,
use horizontal pod autoscaling instead.
Vertical pod autoscaling can take a bit longer
than horizontal since it can grow the number of pods
without taking any offline.
Make sure that your app can actually benefit
from additional resources.
If it's single-threaded or limited
by workers, rather than resources,
vertical pod autoscaling might not help at all.
Set minimums and maximums and avoid any large changes,
since abrupt changes can require new deployments.
Don't forget to keep your metric server healthy.
Note auto-provisioning, which we'll
talk more about in the next video,
can help by creating new nodes that can better
fit your resized pods.
And if you're using auto, make sure that your workload
can handle restarts.
Also, you'll want to choose a pod disruption
budget to control the number of changes happening at once.
You can use both horizontal and vertical pod autoscaling
at the same time, but there's a few things
that you need to be careful about.
First of all, you shouldn't use them on the same resource,
like CPU or memory.
Both autoscalers are trying to react to changes in demand
on that same metric, and that could
cause overlapping problems.
But you can always keep vertical pod autoscaler
in off mode to get recommendations
without any changes actually being made,
along with using horizontal pod autoscaling,
so your app can scale horizontally
based on CPU utilization while you can make decisions
based on those vertical pod autoscaling recommendations.
Or, you can use a custom metric for the horizontal autoscaler,
like requests per second.
In addition, you should ensure that your deployments
are receiving enough traffic when
both autoscalers are enabled.
That way, the vertical part autoscaler
can get accurate data for creating recommendations.
As always, it depends on your workload
to know whether you should use horizontal, vertical,
or both pod autoscalers.
But we're not stopping there.
On the next video, we'll cover the cluster autoscaler and node
auto-provisioning.
Remember to check out the link in the description
for the full guide on GKE cost optimization.
[MUSIC PLAYING]