- [Narrator] This is Kayode with the Azure monitor team.
In this video tutorial, I'll walk you through the process
of routing platform logs using diagnostic settings.
Azure platform logs are built in logs
that provide diagnostic and auditing information
for Azure resources and the Azure platform
on which they depend.
They are part of most Azure services,
but each service defines its own categories of logs.
Platform logs are automatically generated,
but to store them, you must enable a diagnostic setting.
This selects the logs of interest
and specifies the destination where the logs will be sent.
Platform logs can be sent to one
or more of three destinations.
The first is log analytics,
also known as Azure monitor logs.
Log analytics workspaces store logs
from multiple sources in a central location,
and allow you to query and analyze logs
using a powerful query language known as KQL.
You can also build alerts that notify you
whenever a log matches user defined criteria.
Log analytics is an Azure native monitoring tool
with resource specific schemas and integrations.
The second destination is Azure Storage.
Storage is an affordable way to archive logs
for an extended period of time,
for reasons such as security, policy compliance,
or bulk offline analysis.
Third, there's Azure Event Hubs.
Event Hubs is a real time data ingestion service
that allows for scalable integrations
with partners and custom applications.
Many monitoring tools offer Event Hub integrations
with built in support for platform logs.
There's several ways to configure diagnostic settings.
But I'll show you how to enable
it through the Azure portal for an Azure resource.
Note here that you can also use PowerShell,
CLI, Arm API's, or Azure policy
to automate enabling diagnostic settings for your resources
I'm gonna use Cosmos DB as the example here.
But this will look similar for most other resource types.
On the left hand Table of Contents Scroll down
to the monitoring section and I'll find diagnostic settings.
This takes me to the list of diagnostic settings
where up to five diagnostic settings are supported.
You may want to use more than one
so that you can send different categories
of logs to different locations.
I already have one setting defined,
but I'll step through the process
for adding another one now.
On this screen, we select the logs we want to collect
and where we'd like to send them.
First I'll give the diagnostic setting a name.
In my case, that's Audit.
And on the left, we choose what logs and metrics to collect.
Every resource type defines what log categories it emits.
So this list will vary depending on the resource type.
Reading the docs for that service is a good way
to learn more about the available categories of logs.
For Cosmos DB: This data plane request categories a log
of every single back end request to all API's.
These can be very verbose logs and storing them
for a large database can become expensive.
Other categories could provide areas
or performance summaries.
Metric categories allow you to export Azure metrics
to retain them as logs for more than the 90 days
that they're normally stored.
However, not all metric types are exportable
via diagnostic settings.
On the right, we choose where to send the logs.
For this demo, I already have a workspace created.
So let's start there.
I simply pick the subscription and then the workspace
where I want to send the logs.
Now move on to storage.
For storage, you can only send a storage account
in the same region as the resource sending the logs.
Note that retention boxes now appear on the left.
These only apply to storage and allow you to define
when the logs will be deleted.
And if you set your retention to zero,
they'll be retained until you delete them yourself.
For Event Hub, you have the same
regional restriction as storage.
Here, you can pick a subscription,
then an Event Hub namespace,
which can contain one or more Event Hubs.
And you can pick an Event Hub inside
your namespace or diagnostic settings
will create one for you automatically.
If you have any policies, you can select them here.
Now make sure to click the Save button when you're done.
Be aware It can take a few minutes
for logging to start happening.
Let's go ahead and look at the logs.
I'll return to the monitoring section
of my Cosmos DB blade, and I'll click on the logs button.
This redirects me to log analytics,
and I'm presented with several example queries.
The queries already scoped based on the resource type.
And they make it very easy to get started
extracting useful information from the logs.
After a quick browse of the example queries,
I'll start by running the audio of a time query.
It takes a few seconds to complete.
And it reveals an anomalous spike
in usage in the time series chart.
But it doesn't really tell me any more details,
such as the type of operations responsible,
or the API's used, or so on.
From here, I can pivot and choose a different query
from my example query set.
I'll use the one that shows top operations by RU consume.
This initially presents me with a table.
But I can add a couple lines at the end of the query
to render the data as a pie chart instead.
So to recap what we've covered here:
Diagnostics settings can be used to route platform logs
to log analytics, storage accounts, and Event Hubs.
If the logs are stored in log analytics,
you can navigate to them, run queries, and perform analysis
directly in the Azure portal.