In this video, you'll get an introduction to AWS Observability Accelerator for
Amazon Elastic Kubernetes Service (Amazon EKS).
With this solution, you can easily deploy and configure an application-specific
observability environment, use Terraform to create Amazon Managed Service for
Prometheus and Amazon Managed Grafana resources, and conduct
end-to-end monitoring of applications.
We’ll begin by cloning a GitHub repository inside our AWS Cloud9 environment.
The terraform-aws-observability- accelerator repository contains a
collection of Terraform modules that make it easier and faster to
configure Amazon EKS clusters with AWS observability services.
Now let’s open the directory we just created and navigate to the example
called existing-cluster-with-base-and-infra.
This is going to be the parent directory from which we'll be deploying the Terraform solution.
Before we deploy the terraform modules, let’s export some variables
so that the we will be able to deploy the components in the right EKS cluster,
store the metrics in the Amazon Managed Prometheus workspace
and visualize the metrics using Amazon Grafana.
Let’s first export the region.
Next, we will export the EKS Cluster ID.
This will be the cluster where the AWS Distro for OpenTelemetry will
be deployed and will start collecting the metrics.
Next, we will export the Amazon Managed Prometheus workspace id and the
Amazon Managed Grafana workspace.
Finally, we will export the Amazon Grafana API key.
We will use the CLI to generate the ADPI key and provide it at runtime which is a best practice.
Before we continue, let’s take a quick look at the workspace we’ll be using in
Amazon Managed Service for Prometheus.
Let’s look at the Rules management tab for this workspace.
We have no rules yet.
Let’s look at the Alert manager tab.
We have no alert manager definitions either.
Next, let’s look at Amazon Managed Grafana.
We have no saved dashboards.
Let’s go back to the terminal and deploy the solution by running Terraform.
Terraform has been successfully initialized.
Next, let’s run terraform plan to create a preview of the changes that Terraform
will make in our infrastructure.
We can see the different modules that will be created.
Notice the model with the namespace called opentelemetry-operator-system.
An AWS Distro for OpenTelemetry (ADOT) collector will be created to
scrape metrics from the application running on the EKS cluster and ingest
them into Amazon Managed Service for Prometheus.
In all, we’ll be creating 38 resources using the Terraform solution.
Let’s go ahead and apply the solution.
It might take a few minutes for all the deployed resources to appear.
Let’s go back to Amazon Managed Service for Prometheus and check the alert manager again.
The alert manager is being created.
Let’s check the Grafana dashboards.
Let’s refresh the view.
We now have a folder called “Observability Accelerator Dashboards.”
Let’s return to Amazon Managed Service for Prometheus.
The alert manager has been successfully created.
The Rules management tab is still empty.
We can open a separate terminal and see the parts of the solution as they get deployed.
Let’s go back to Amazon Managed Service for Prometheus and take a
look at rules management again.
Now rules are being created.
We have namespaces for recording rules and for alerting rules.
Let’s check on the Observability Accelerator Dashboards folder in Grafana.
The dashboards have been populated and appropriately tagged.
We’ll view these dashboards when the ADOT Collector fully deploys
and metrics have been stored within Amazon Managed Service for Prometheus.
Next, let’s look at our data sources.
We can see that the AWS Observability Accelerator workspace has been
configured as a data source for Amazon Managed Grafana.
Let’s drill down.
Let’s save and test this data source.
We’ve confirmed that the data source is working.
Let’s go back to the terminal.
The Terraform apply is complete and 38 resources have been added.
Let’s check the deployment status of the ADOT collector.
The ADOT collector has deployed successfully.
There is also an EKS ADOT managed add-on operator running on the Amazon EKS cluster.
Now let’s go to Amazon Managed Grafana and explore some dashboards.
Let’s view the “Kubernetes / Compute Resources / Cluster” dashboard.
Metrics are already appearing on this dashboard.
We can see information about workloads running on our Amazon
EKS cluster and the corresponding CPU usage.
There is also information about quotas, including the CPU requests and CPU limits.
We can also view memory request information and many other metrics.
Let’s look at another dashboard.
We’ll view the kubelet dashboard.
Three kubelets are running,
and we can see a great deal of information pertaining to their operation.
This level of detail provides valuable insight into the
workloads running on our Amazon EKS cluster.
Let’s look at one more dashboard.
We’ll examine networking at the cluster level.
Here we can see information about bytes received and transmitted by the cluster,
as well as metrics for specific namespaces.
We can also view other information,
such as the rates of received packets and transmitted packets.
You’ve just seen an introduction to AWS Observability Accelerator for Amazon EKS.
You can learn more about this topic in the description and links for this video.
Thanks for watching. Now it’s your turn to try.