>> Hello and welcome to
another video in the Azure Cloud native series.
In this session, we'll be looking at how you can right
size your AKS cluster using autoscale.
Before we dive into the details,
it's worth motivating this with an example.
Suppose you have a cluster with the following usage pattern.
For one peak hour during the day,
you require a 100 gigabytes of memory to meet demand.
For an additional eight hours,
you require 50 gigabytes,
and for the remainder of the day,
you only need 20 gigabytes.
The peak time can be met with a cluster of eight DS3,
v3 VMs while the load demand portion can be handled with just two.
Over the course of the month,
you can save over $800 by using cluster
auto-scaling to match your clusters
infrastructure with its demands.
Now you understand the motivation,
let's talk a little bit about how auto-scaling works.
There are two forms of horizontal auto-scaling in Kubernetes.
Pod-level auto-scaling sets the appropriate level
of pods to meet the demand of the application,
and is handled by the Horizontal Pod Autoscaler
while cluster-level auto scaling sets
the appropriate number of nodes to meet the demands of pods
scheduling and is managed by the Cluster Autoscaler.
It's worth noting that neither of
these auto-scaling techniques is
designed to look at raw utilization.
While the HPA offers CPU-based scaling out of the box,
it is typically most useful when combined
with application specific metrics,
such as requests per second or queue depth.
For its part, the cluster auto-scaler only looks at
cluster capacity relative to
the desired pods as seen by the Kubernetes scheduler.
In short, if the scheduler is unable to
place a pod due to lack of available resources,
cluster autoscaler will kick into action.
With that as background,
let's look an example of auto-scaling in AKS.
For this demo, we're going to instrument a.NET
core application to report how many requests it
receives that can figure in a case cluster to
automatically scale in response to
a spike in traffic to run that application.
We'll be using Prometheus,
an open-source monitoring tool that is
popular for use in Cloud-native applications.
To report request metrics from our applications Prometheus will
be using the Prometheus NuGet package for ASP.NET core.
This package makes it simple to capture metrics and then
expose them from an endpoint the Prometheus can scrape.
In the HomeController for application,
we've created a counter,
one of the classes that is provided by the Prometheus package.
We increment it every time the page is loaded.
Then we create a web server to expose those metrics on port 9,000.
Now let's quickly run this application
locally so that we can see what the metrics look like.
We'll refresh the app a few times
to generate some requests in the counter.
We'll hit the metric server,
which is running on port 9,000.
In addition to some default metrics,
we can see that our request count is currently at poor.
Within this cluster, we deployed a couple of Helm packages.
Prometheus, which is responsible
for pooling metrics from applications
and systems services in the cluster and Prometheus adapter,
which takes those metrics and exposes
them as an extension of the Kurbernet API,
so that they can be queried by the horizontal pod autoscaler.
For our specific application,
we've deployed a few different resources.
First, the applications deployment calls out
a resource request of
roughly half a gig of memory and a 10th of a CPU.
The scheduler will look to ensure that
those requests can be met up front.
If they can't, the pod will remain in a pending state.
The service monitor is responsible for
querying the metrics in the selected end points.
We've also provided some configuration for the Prometheus adapter,
which will convert the request count
captured from every instance of
the application into an average number of requests per second,
which can then be used to make scaling decisions.
Finally, we've deployed a horizontal pod autoscaler resource
that will control the scaling behavior for the application.
In this case, we've configured it to
scale between one and 100 over
replicas as required to get
the value of requests per second under 25.
So now let's see the auto-scaling behavior in
action by firing some load at the application.
As that happens, we'll watch the behavior of
the HPA and the associated pods for our app.
Within just a few seconds of the load starting,
you can see that the HPA is
detecting that load and it's starting to scale.
As additional pods come online,
they will start to take some of the load,
but they won't be able to get the average RPS under the threshold.
So the HPA will keep trying to add more pods until
either the target is made or the HPA reaches its max pod setting.
Eventually, you will see some of
those pods end up in a pending state,
which means that the Kubernetes scheduler
was unable to find a node that could fit them.
Keep in mind that the Kubernetes scheduler operates
exclusively off of the resource request down in the pod spec,
not what the pod is actually using.
So it's important to fine tune
those values to meet the needs of your application.
The presence of these pending pods is what triggers
the cluster autoscaler to begin adding more nodes.
We can monitor the status of the cluster autoscaler using
the cluster autoscaler status
ConfigMap into Kube system namespace.
Note that the cluster autoscaler can be enabled on
a per node pool basis with individual minimum and maximum counts.
When impending pods are detected,
the autoscaler runs a simplified version
of the Kubernetes scheduling
algorithm to determine which of
its candidate node pools would be most effective to scale.
Look in a few minutes,
these new nodes come online,
at which point the pending pods will be scheduled.
That can be again, taking traffic,
eventually stabilizing the average RPS below the threshold.
As the load stops, the process repeats in reverse.
After a few minutes, the HPA will notice
that the target metric, in this case,
requests per second, is well under the threshold,
meaning that it no longer needs as many pods as it has.
Begin slowly ratcheting down the target number of pods while
continuing to monitor that the metric is under the threshold.
Note that by default, the HPA is significantly more
conservative in scaling in than it is in scaling out.
This is intended to ensure that the reduction in traffic is
sustained and that the evaluated metric is stable.
As those pods shutdown,
denotes that the cluster autoscaler spun
up will become less and less utilized.
Cluster autoscaler will then detect
those underutilized nodes and set them as candidates for removal.
A node becomes a candidate for removal if less than 50 percent of
its capacity is allocated
and if all of its pods can be moved to other nodes.
That's with the HPA.
The cluster autoscaler is designed to
scale in much more slowly than it scales out.
By default, a node must be unneeded
for 10 minutes before it will be removed,
and nodes are fully draining before they are deleted.
Only one node is drained at a time,
and pod disruption budgets are fully
honored to ensure application availability.
So fully empty nodes are removed in bulk.
Before too long, the HPA has scaled all the way back down to
one pod and the node pool has scaled back to one node,
significantly reducing the cost of running the cluster.
Before wrapping up, it's worth calling out
a few additional interesting tools around autoscaling.
The vertical pod autoscaler monitors
the actual resource consumption of your pods and adjust
their resource requests as appropriate to ensure that
your applications are getting what they need to run
properly without wasting resources.
The Kubernetes Event-Driven Autoscaling project, or KEDA,
provides an easy way to drive horizontal pod autoscaling,
often common external events such as items being added to a queue.
To learn more about autoscaling in
AKS and how you can use it to save
costs, check out aka.ms/aks/autoscale.