SPEAKER: Autoscaling helps you size up
to handle increased demand and size down
to avoid paying for resources that you're not using.
That's not just limited to your pods.
It can also apply to the machine side.
Today, we'll talk about autoscaling infrastructure
with Google Kubernetes Engine and cover some best practices.
Let's go beyond your GKE bill.
In the last video, we covered how
autoscaling works and went over horizontal and vertical pod
autoscaling.
If you haven't seen it, I definitely
recommend checking it out, since we'll
look at some additional ways that you can configure
autoscaling and how that can help you
with cost optimization.
I'll start by going over the cluster autoscaler and then
something really cool, node autoprovisioning.
Then we'll wrap up with some tips
to use when setting up your autoscaling.
So if pod autoscalers can help you resize or add
additional pods, what can you do with the infrastructure side?
Well, just like the pod autoscalers,
you can think of the same horizontal and vertical options
for infrastructure.
This time, we're talking about changing the number of nodes
or automatically creating additional node
pools with optimized size.
This is really where the cloud and Kubernetes work well
together, since the cloud tends to give you
a lot more flexibility with your resources.
Let's start with horizontally adding more nodes, which is
done by the cluster autoscaler.
The cluster autoscaler is designed to add or remove nodes
based on demand.
But this time, demand is actually
based on pod scheduling and requests.
If additional pods aren't able to be put into the existing
nodes that are running, the cluster autoscaler
will create a new node, so that the pods can get placed.
You can and should use the cluster autoscaler along
with the horizontal or vertical pod autoscalers.
That way, as you create more or larger pods,
your infrastructure can grow alongside them.
It's also important to note that the cluster autoscaler is
perfect for cost optimization since it is actually
built to prefer lower costs.
That means that the autoscaler will
start by trying to create additional nodes
of the cheapest type and then move to more expensive node
types if it can't create the cheaper ones.
Isn't it great when cost optimization is just
built right in?
Just like the pod autoscaler, having an appropriate pod
disruption budget is important for having an effective cluster
autoscaler.
The budget defines the minimum number of running pods
for your workload.
So when the cluster autoscaler is removing nodes,
you can ensure that your workload will still
be running correctly.
Now, there are some pods that can't be restarted,
which will then block the cluster autoscaler
from removing a node.
System pods, like Kube-DNS and pods that use local storage,
are examples of pods that can't be restarted by default.
For system pods, you can set a pod disruption budget, which
will allow them to be rescheduled on another node,
freeing up their original node to be deleted.
But you may not want to set a budget for some system pods
that only have one replica, like metric server,
since restarting them can cause temporary disruptions.
For pods with local storage, you can enable a special annotation
called safe to evict to let the cluster autoscaler know
that they're safe to be restarted.
If you have any pods that are supposed to run for a long time
without being restarted, you might want to move those over
to a separate node pool, so they don't
block the cluster autoscaler.
There's also a few more things to think about when
using the cluster autoscaler.
Make sure to figure out how to read
the cluster autoscaler logs, so that you can understand
any unexpected actions.
If your workload can support it, see
if you can use preemptible VMs, since those
will be prioritized by the cluster autoscaler
and could save you a lot of costs.
But don't forget to have a backup node pool in case
there's any trouble with creating new preemptibles
when they're needed.
Set your minimum nodes by figuring out
the minimum capacity needed to keep your workloads running
without disruption, so you don't end up with idle nodes.
So the cluster autoscaler helps add and remove nodes
in response to pods needing to be rescheduled.
However, GKE specifically has another feature to scale
vertically called node autoprovisioning.
Node autoprovisioning actually adds new node pools
that are sized to meet demand.
Without node autoprovisioning, the cluster autoscaler
will only be able to create new nodes in the node pools
that you've specified.
This is perfect for helping optimize resource
usage for workloads and other apps
that don't need extreme scaling since creating
an optimized node pool might take some more time
than just adding additional nodes to an existing pool.
It's pretty cool to have node pools created based on what
best fits your application.
And as long as your apps support interruptions,
you can save even more costs by setting your pods
to be preemptible tolerant, allowing the auto provisioner
to create preemptible node pools.
A few other things to keep in mind.
Consider how each of your autoscalers
will affect your apps, and make sure
that you spent the time to configure each of them
properly.
New node pools can take a bit longer
to be created than simply adding more pods or nodes,
so you may want to adjust the metric value
of your horizontal pod autoscaler
to create additional pods sooner.
Set minimum and maximum resource sizes
based on what your apps need, so that the node auto provisioner
doesn't interfere with keeping your workload running.
With all four types of autoscalers
that we've talked about, there's a lot of configuration options
that you can use to customize your GKE clusters.
Effective use of some or all of these
can really maximize your cost optimization.
But it's also important to understand
how long it might take to scale resources,
especially if you're expecting large spikes.
Let's walk through what scaling up looks
like when demand increases.
For both horizontal and vertical pod autoscaling,
they'll respond by adding pods or resizing pods, depending
on the settings that we talked about in the last video.
On the infrastructure side, if there's
room on your existing nodes that have already deployed your app,
it might be able to skip pulling the image
and just start running the application.
If you're working with a node that
hasn't deployed your application before, a bit of time
might be added if it needs to download the container
images before running it.
But if you don't have enough room on your existing nodes
and you're using cluster autoscaler,
it could take even longer.
Now it needs to provision a new node,
set it up, and then download the image and start up pods.
If the node auto provisioner is going to create a new node
pool, there will be even more time
as you provision the new node pool first
and then go through all the same steps for the new node.
In order to handle these different latencies
for autoscaling, you'll probably want
to overprovision a little bit, so
that there's less pressure on your apps when scaling up.
This is really important for cost optimization
because you don't want to pay for more resources
than you need.
But you also don't want your apps to struggle.
Let's go through two strategies for figuring out how to
over provision using a pretty simple formula.
We'll start with a formula you can
use to find an optimized target for using horizontal pod
autoscaling.
The first thing we'll want is an appropriate buffer target
for the application to ensure that everything
is running smoothly.
Let's choose a buffer size of about 15%,
so that we can keep the application safely away
from 100% CPU utilization.
Then we can take a look at the percentage of traffic growth
that we might expect in a two- to three-minute period.
Here, let's say that we expect a growth of about 30%.
With these numbers in the formula,
we can calculate a target for the horizontal pod autoscaler
around 65%.
This generally means that we'll be about 35%
over provisioned in order to appropriately handle
the expected traffic growth while still minimizing
extra resources.
If you're using larger nodes and smaller workloads
with consistently less traffic than requested,
you could consider over provisioning
a bit less since there might be room
to go above 100% utilization.
The second strategy is to use pause pods.
These are low-priority deployments,
which mean that they could be removed and then replaced
by high-priority deployments.
Since these pause pods aren't actually doing anything,
they're basically just reserving buffer.
When the higher priority pod needs room,
the pause pod will be removed and rescheduled
to another node or a new node.
And the higher priority pod has the room
it needs to be scheduled quickly.
Using these can help because node spin up time
can happen without affecting the high-priority pods.
It's basically like scaling a little bit
ahead of when you need to since they'll
be spun up before they're needed, thanks
to those low-priority pots.
It's a good idea to try and only use one pause pod per node,
so it's quick to be rescheduled, the API
isn't overloaded with even more requests,
and the full node will be available quickly.
All right, that's a lot to think about for autoscaling.
And "pause pod" is really hard to say.
Once again, it's important to know what your apps can handle
and to optimize around that.
But you can also optimize your apps.
On the next video, we'll cover different ways
to do exactly that.
Remember to check out the link in the description
for the full guide on GKE cost optimization.