SPEAKER: Google Kubernetes Engine
helps you focus on your application
while still being able to configure details
of how your app can run.
When it comes to cost optimization,
you can fine-tune both your application and your GKE
configuration, getting the most value for what you're spending
and reducing wasted resources.
Today we'll talk about some tips and best practices
for the application side of that.
Let's go beyond your GKE bill.
[MUSIC PLAYING]
In the past few videos, we've spent
a lot of time looking at optimizing GKE
through monitoring, machine types, autoscaling, and more.
If you haven't watched the previous videos,
I'd suggest checking those out first,
since a lot of specific optimization tips in this video
and the next builds on those previous concepts.
We're talking all about apps today.
So I'll start with the resource side,
then move on to some best practices for how
to build your containers.
After that, we'll talk about some ways
to make sure that your app and Kubernetes are
working well together.
The best way to start is to repeat this.
It is critical to know the details of your application
and how it responds to demand.
Monitoring can help understand what resources
are used as demand increases.
We covered monitoring earlier, but here's another tip.
Try running your application as a single pod replica
and then test it under load without any autoscaling.
Once you have that information, use
it to configure resource requests and limits, which
we'll talk about in a minute.
Then test your app again with cluster autoscaling
and horizontal or vertical autoscaling.
Now test large, rapid spikes in demand
to see how your app responds.
No matter which tool you use to stimulate demand,
try to use the same region or zone
as where your app is running.
That way you can minimize latency
and focus on understanding all the details of how
your app scales.
And there's more you can do with that knowledge.
Earlier in the monitoring video, we
talked about resource quotas and limit ranges.
You can also set resource requests
and use limit ranges to control resources.
With a resource request, you're defining
how many resources your application
actually needs to run.
With a resource limit, you're defining
the maximum amount of resources that your app
is allowed to use.
These will help Kubernetes figure out
how to schedule your pods and ensure
that they don't use up all the resources that are on the node.
One best practice is to set your memory
limit equal to your memory request
and to set your CPU limit higher than your CPU request.
Or you can even leave it unbounded.
Kubernetes can always throttle the CPU
that your pods are using down to the request amount
if the CPU resources are in high demand.
But the same doesn't work for memory,
since Kubernetes needs to shut down pods when scaling
and if there's too much demand on the memory.
Of course, any form of autoscaling
might not be very useful if your application can't scale.
Vertical autoscaling, it's only effective
if your application can actually use the additional resources
on the nodes.
In the same way, horizontal autoscaling
is only effective if additional pod replicas actually let
your app handle more demand.
If the actual performance bottleneck
is on something like a required database,
more pods might actually make the problem worse.
Either way, you can use the different autoscaling
techniques that we went over in the previous videos
to make a configuration that works best for your app.
But if your app can't scale, you might
need to consider refactoring to a structure that works well
with Kubernetes.
For example, single-threaded apps or apps
that can only run a limited number of concurrent processes
may not get much value out of the extra resources.
If your app can scale, the next step
is to look at spinning up and spinning down.
Here's some best practices for minimizing startup times.
Whenever the cluster autoscaler creates a new node,
that node has to download the base image for each app
that it'll run.
The smaller your image is, the faster a node can download it.
In the same way, use the smallest base image
that your app actually needs to run.
Also try to minimize the amount of startup time needed
to start handling requests.
Startup tasks like loading classes and caches
can significantly increase the amount of time between creation
and then being ready, which could cause requests to fail
while your app is spinning up.
Check out the best practices for building
containers link in the description
below for more information.
On the other side, you want to make sure
that your app can shut down gracefully.
If your workload has trouble shutting down,
it can lead to delays and disruptions in service.
When a pod is going to be shut down,
it should receive a SIGTERM.
Instead of just assuming that everything is done,
this is actually a signal to do some cleanup.
First of all, you want to update your readiness probe to fail.
More on that coming soon.
So that request will stop coming into the pod
if you're using container native load balancing, which we'll
talk about in the next video.
But your pod must still finish all the active requests
and any new ones that come in, since it can take a bit of time
for Kubernetes to update where those requests are
going to be delivered to.
In addition to that, your pod should run any cleanup steps
like persisting any in-memory state.
If you're not handling SIGTERM or you
can't because your app is using a system that you don't
actually have control over, like a web
server or a reverse proxy, you can use the preStop hook.
Here your pod can run the same cleanup,
and you can even put a sleep command in for a few seconds
to postpone the SIGTERM.
There's also a configuration to give your pods a bit more time
to shut down.
It's called terminationGracePeriodSeconds.
You should only change this if you really need it.
By default, it's set to 30 seconds,
but it can go all the way up to 10 minutes before the cluster
autoscaler forcibly terminates it.
Quick startups and proper shutdowns
help your autoscalers work in a more reliable, more efficient,
and cost optimized way.
The more specific you configure GKE to your application,
the better it can run it.
In the first video, we talked about pod disruption budgets,
when you statically set the minimum number of pad replicas
to keep your app running.
Kubernetes will try to respect that number
as it handles rollouts, upgrades, and autoscaling.
It's not a guarantee when unexpected things happen,
but it's still a best practice to define pod disruption
budgets for every application.
That way your app will almost always
have enough replicas to keep running effectively.
In addition to those budgets, you'll
also want to make sure that you're correctly
setting readiness and liveness probes for your app.
Kubernetes will use the readiness probe
to know if each pod is ready to handle traffic.
So you don't need to worry about race conditions between traffic
and setup time.
Liveness probes are used to tell if the pod is
still alive and working.
So if your application gets stuck or deadlocked,
Kubernetes knows to restart it.
These two probes are key to making sure that Kubernetes can
optimize working with your app.
And here are some best practices for working with them.
Always define readiness probes for every container.
Know when your application is actually ready.
For example, if your app depends on a cache being loaded,
the readiness probe should only say
the pod is ready after that cache has been loaded.
If your app doesn't need any startup time,
define a simple readiness probe, such as just returning
HTTP 200.
Also make sure the code for your probes
is as simple and isolated as possible
so there's no blockers when Kubernetes
is requesting the status.
Your probe logic should try to avoid remote services.
And if you have to use something remote,
try to test against the simplest and hopefully fastest one.
One more best practice for any and all of your services
is to consider how to handle retries.
Especially in a full microservices environment,
a service call could fail for any number of reasons,
like a network failure or rate limits.
When there's a failure, it usually
makes sense to retry the request in case
it was something temporary.
But if it's more permanent, you may end up
sending lots of requests and potentially even
overwhelming the destination.
For both the way you call services
and the way you create your own services,
use exponential backoff to protect against overwhelming
the service.
In short, you just add a bit of time
after the first failure and then more if there's another failure
and continue exponentially.
When calling a service, you can implement exponential backoff
logic yourself, use a library, or even use Istio or Anthos
Service Mesh that support a proxy level retry which
handles it all for you.
If your service is the one being called,
you can rate limit requests from the same caller
to force exponential backoff.
You'll also want to prepare yourself
for handling retry requests so you don't end up duplicating
functionality in a bad way.
Bank transactions, for example.
Of course, the specific details of how to implement that
will depend on your application.
These are some things to keep in mind when
working with your application, and each one will give you
different results.
The key here is to really have your application and GKE
work together as efficiently as possible.
On the next video, we'll look at tips for optimizing GKE
and do a wrap-up.
Remember to check out the link in the description
for the full guide on GKE cost optimization.
[MUSIC PLAYING]