YURI GRINSHTEYN: The ancient Greeks
told of how Prometheus stole fire from the gods
and brought it to humankind.
Today, Prometheus is bringing better observability
to Kubernetes operators as a leading open source monitoring
framework.
In this episode, we'll look at how to set up Prometheus on GKE
and use it to instrument your application with custom
metrics.
This is "The Stack Doctor."
[MUSIC PLAYING]
Last time, we looked at how to use OpenCensus to add custom
metrics to our application.
Today, we'll do the same with Prometheus.
Before we start, a bit of background.
Prometheus is an open source systems monitoring and alerting
tool kit originally built at SoundCloud.
It has a very active developer and user community
and joined the Cloud Native Computing Foundation in 2016.
So how do you set it up?
The fastest and easiest way to actually set up
Prometheus in your GKE cluster is
to use Helm, which is a package manager for Kubernetes.
Once you have Helm installed and configured,
you can install Prometheus in your cluster
simply by running the helm install
stable/prometheus command.
The install will create multiple components--
the Prometheus server itself, an exporter running on every node
to capture node statistics, alert manager
to send email notifications, and the push gateway
to accept metrics to be pushed to Prometheus.
We'll primarily interact with the server.
Let's validate the installation by port forwarding the server
pod to our local host.
This allows us to access the Prometheus UI and graph
metrics to confirm that data collection and persistence are
working.
Now that Prometheus is installed,
we are ready to start adding metric instrumentation
to our application.
But first, let's quickly review the kinds of metrics
that Prometheus supports.
A counter is a cumulative metric that
represents a single increasing value that can only increase
or be reset to 0 and restart.
For example, you can use a counter
to represent the number of requests served, tasks
completed, or errors.
A gauge is a metric that represents
a single numerical value that can arbitrarily go up and down.
Gauges are typically used for measured values,
like temperatures or current memory usage,
but also counts, like the number of concurrent requests.
A histogram samples observations and counts them
in configurable buckets.
It also provides a sum of all observed values.
Histograms are most often used to measure
the distributions of things like request durations or response
sizes.
Finally, a summary samples observations and provides
a total count and a sum of all observed values.
It also calculates configurable quantiles
over a sliding time window.
Summaries are usually used to values where the total value is
important, like payload sizes.
Now that we know what kind of metrics
we can expose in our application,
it's important to understand how Prometheus itself works.
It's a basic pull model.
Our application needs to expose a dedicated endpoint where
the metrics will be available, and we
need to tell Prometheus where that endpoint is
and how frequently to scrape it for metrics,
so let's do both of those now.
First, let's instrument our application.
We'll create a simple Go application
and expose a gauge metric in it.
Here's the code.
You can find a link to the repo in the video description.
The first step is to define the metric itself as a gauge
and provide its name and a description.
Next, we create a function that will
set the value of our metric.
In our example, we're just generating a random value
so that we know it'll change every time.
Now we can run our application.
When we access the metrics endpoint,
we can see that our custom metric is generated and shown
here.
Now our application is instrumented
with a custom metric, which is being exported to Prometheus.
We need to tell Prometheus where to find this information.
We'll do that by adding some annotations in the deployment
spec in our application before we deploy it to Kubernetes.
We'll enable scraping and define the port
and endpoint where the metrics are available.
Our last step is to set up Stackdriver integration,
to have this metric be ingested into Stackdriver.
We'll install the Stackdriver collector
on our cluster, which will patch the Prometheus server
deployment.
Now when we go to Metrics Explorer,
we can see our random value metric there.
I hope that you've seen how Prometheus is a fantastic way
to add custom metric instrumentation
to your application and have those metrics be ingested
in Stackdriver.
Thank you for joining me today, and come back
next time when we explore options for managing monitoring
and logging costs.
That's it for this episode.
This is "The Stack Doctor."
Stay healthy out there.
[MUSIC PLAYING]