SPEAKER: The first key to monitoring
GKE cost optimization is ensuring
that cluster administrators have an understanding of how
much of their cluster resources are actually used by workloads
in the cluster.
With this data, they can take steps
to ensure that fewer cluster resources are idle or wasted.
Let's take a look at how we can utilize a monitoring
system around this.
[MUSIC PLAYING]
As we explored in the previous video,
the concept of bin packing is focused
on ensuring that most of our cluster resources
are allocated to cluster workloads.
Cluster administrators want to achieve
efficient bin packing to ensure that a cluster is not
overprovisioned.
Monitoring systems will be able to tell cluster admins if this
is the case by letting them know how much of a cluster's
allocatable resources are actually
consumed by workload requests.
There are three main areas we want
to monitor to ensure that we are not straining resources
across our cluster.
CPU, memory, and the amount of pods we run on each node.
CPU and memory are well-known concepts in Kubernetes,
so let's start with how we should
monitor how well they're actually
allocated in a cluster.
Up to this point, we have discussed monitoring
at the aggregate cluster level.
With per-cluster monitoring, we are
getting a high-level summary of how well we are
doing with cluster bin packing.
However, the larger the cluster and the more diverse
the machine types, the more important
it becomes to monitor at a more granular level
that will help cluster administrators take action.
Monitoring at the more granular per-node pool level
can help us take this action.
This monitoring can help identify target areas
that cluster administrators can improve cluster bin packing.
For example, if we are isolating a set of workloads
to their own node pool that is not correctly
sized to fit such workloads, underutilization
may not appear to be a problem at the aggregate cluster level,
but it could very well be a problem within that node pool.
For both cluster level and node pool-level monitoring,
we should collect these four metrics around CPU and memory.
Allocatable resources, which references
total CPU or memory available from the cluster or node pool.
This value will always be slightly lesser
than the total resources from the cluster
VM due to critical system components that
have to run on each node.
Requested resources, which references total CPU or memory
requested by pods running in the cluster or node pool.
Recommended resources, which reference recommended CPU
or memory requests from the platform, which
can come from components such as vertical pod autoscaler.
And finally, used resources, which references the actual CPU
or memory utilization from pods in a cluster or node pool.
In this example dashboard for monitoring CPU,
we can see how these four metrics fall together
over a specified time window.
This graph tells us, for instance,
that there is opportunity for cost optimization
due to used cores falling well below the requested core count
by workloads in the cluster.
However, when looking at this example dashboard
for monitoring memory, we get a much better picture
of what is going on.
Workloads are requesting and using most of the memory
we have available, but barely utilizing the CPU available.
With this information, we can consider
the types of machines we are running in our cluster
or in our node pool.
Can we provision VMs with the same amount of memory
but with less CPU?
With the appropriate data at hand,
we can now begin this exercise of answering this question
to practice cost optimization.
The third area we should monitor to ensure efficient bin packing
is pods per node.
In GKE, users can set the number of pods allowed
to run on a node to smaller than the default
value, which is 110 pods.
This is commonly used as a mechanism
for preserving IP addresses, since each pod in a cluster
or in a node pool gets its own IP.
This feature in GKE is called flexible pod CIDR.
If you set a small enough number of pods on a node,
there's a chance that you could schedule the maximum pods
per node without using a majority of the allocatable CPU
and memory on that node.
This is also possible even with the default value
of 110 pods per node, should you have very large nodes
with smaller workload requests.
Thus, it's important to monitor when you brush up
against the maximum pods on a given node,
especially if you're attempting to achieve IP address
preservation.
Monitoring across these key areas
will help cluster admins continually
optimize their cluster infrastructure
in a data-driven fashion.
This is especially important as workloads scale up and down
and when new tenants onboard into the cluster.
Be sure to check out the paper we published
on how to set up this monitoring system, linked
in the description below.
Setting this monitoring up will lay the groundwork
for the cluster admins to work with cluster tenants
to ensure that they are doing the work
to perform at right sizing.
We'll cover this in the next video, so stay tuned.
[MUSIC PLAYING]