YURI GRINSHTEYN: Google Cloud offers
a wealth of managed services and serverless
options for your workloads.
But many mission critical services
use compute infrastructure directly
and run on Google Compute Engine instances.
How can we improve observability for those workloads?
Today, I'm going to show you how to use the Cloud Operations
agent to do just that.
Welcome to Engineering for Reliability with Google Cloud.
First, what are agents, and why do we need them?
Here is an example of an infrastructure summary
dashboard for a project with a few GCE VMs in it.
Notice that we're not getting any data about how
our instances are using memory.
That's because the VM hypervisor only
knows how much memory is allocated to each VM,
not how much of the allocated memory
the VM is actually using.
This is just one example of data that can only
be gathered by a process running in the guest VM,
such as an agent.
There are other benefits of running
the Ops Agent inside the VM.
It monitors your VM instances without the need
for any additional configuration after the install.
It exposes many additional metrics beyond memory,
and gives you better visibility to CPU, disk, and network
performance.
And the Ops Agent unifies gathering of metrics
and logs into a single agent.
If you've been using Google Cloud for a while,
you're probably familiar with the Legacy monitoring
and logging agents.
However, at this point, the Ops Agent
should be used in all cases.
Now that we've covered the benefits of agents,
let's have a look at how to install the Ops Agent.
I'm going to show you how to install
the agent in three ways.
First, we'll install the agent on a single VM
using local commands.
Then I'll show you how to install it on a fleet of VMs
by using Agent policies.
And finally, we'll automate Agent installation
using Terraform.
Let's get into it.
From the project dashboard, let's go
to monitoring dashboards.
Select the VM Instances dashboard.
Here, we can see that there's a VM without any agents on it.
Let's SSH.machine.
Once the SSH connection is established,
run the commands to download and install the Ops Agent.
We'll link to the documentation that has the up to date
command in the episode notes.
After some time, you'll see an installation succeeded message
from the Cloud Ops Agent.
At this point, we can go back to our dashboard
and confirm that the VM is reporting as having the latest
version of the Agent installed.
We can then go to the VM detail screen
and see that we're getting agent metrics for that VM,
including memory utilization and details of CPU usage.
So that's an easy way to install the Cloud Ops
Agent on a single virtual machine
by using an SSH connection and installing the Agent
through the command line.
That was easy enough.
Now that the Agent is installed, let's
have a look at all the additional data
it's gathering from our VM.
Let's go to our VM instances and select our VM.
The observability tab is a single place
to see monitoring and logging data collected for this VM.
In the Metrics section, we can see lots of data,
like memory and disk utilization.
This is just some of the information
that's being collected by our Agent running on the VM.
The Processes section shows what's
running on the VM and resource utilization of all
the various processes.
Again, this information is only available by having
an Agent on the VM.
The Memory section shows us how memory is being used by the VM.
And finally, the Log section will show us all the logs
being collected from the VM.
We can open these in Logs Explorer
to better understand where each of these is coming from.
So that's how all the additional data being gathered
by the Agent is collected in a single place
and made easy for you to use.
The Ops Agent collects a lot of valuable data.
But how can we install it at scale across our VM fleet?
Let's have a look at automating the installation
and maintenance of the Ops Agent with agent policies.
Let's start on the VM instances dashboard.
Here, we see that we have multiple VMs with no Agent
installed on them.
We can install an Agent on all of them
at once by using an Agent policy.
Let's have a look at an example of how
to do that using G Cloud.
We'll post a link to the details of the command in the episode
notes.
But here's an example that will require installing the Ops
Agent and all the instances in the specified
zone that match the operating system and version filter.
Note that you'll have to enable the relevant APIs
in your project in order for this to work.
Again, refer to the documentation
for those details.
After some time, the operation will report a success.
We can then go back to our dashboard
and see that the VMs that match our operating system filter
and reside in the zone we specified for the policy
have the latest version of the Agent installed on them.
We can go to the details for one of the VMs
and confirm that we're getting Agent information there.
So that's how you can use Agent policies to ensure
that the Cloud Ops Agent is installed and automatically
upgraded across a VM fleet.
Using Agent policies to automate Agent installation
and maintenance is much more scalable and error proof
than doing it manually.
But even entering commands like this
is still prone to input and human error.
Let's have a look at how to turn Agent policy configuration
into code with Terraform.
Let's start by defining our Agent policy using Terraform.
The Agent rule section will define
which Agent we want to install, and whether we want to enable
auto upgrading for it.
The group labels, OS types and zones filters,
will define which VMs are subject to the policy.
To create the policy, first issue the Terraform validate
command to make sure your code is correct.
Once it's validated, use the Terraform
apply command to create your Agent policy
and confirm that the operation is successful.
Once the Apply command is finished,
run the G Cloud describe command to validate that the policy has
been successfully created.
Finally, verify that the policy has been applied,
and that the Agent has been installed on the instances that
are subject to the policy.
Thanks for joining me today.
We covered the Ops Agent, how it differs from the Legacy
monitoring and logging agents, manual and automated Agent
installation, and the data that the Agent provides
to help you improve the observability if your service
is running on GCE.
I hope this helps you find issues in your VMs
faster and keep your users happy.
Thanks so much for watching.
Don't forget to like and subscribe to never miss out
on more Engineering for Reliability with Google Cloud.
See you soon.