Welcome back to the final part of our Azure Logging and Monitoring Series. For the final part of the series, we
will look at how we can visualize our metrics in a way that’s meaningful, analyze what we have for better future performance
as well as set up alerts that can be used to bring important things happening in our solution to light.
Visualizations such as charts and tables are effective tools for summarizing monitoring data and presenting it
it to different audiences. Azure Monitor has its own features for visualizing monitoring data and leveraging other
Azure services for publishing it to different audiences. So as mentioned you can send platform logs
to one or more of the following destinations- One of them is Log Analytics workspace (which we will talk about later on in
depth)- In Log Analytics workspace you can analyze the logs of all your Azure resources together and take
advantage of all the features available to Azure Monitor Logs including log queries and log alerts.
The second location you can send to is Event hub. You can send platform log data outside of Azure,
for example to a third-party SIEM or custom telemetry platform.
The third location you can send to is Azure storage – where you can archive the logs for audit or backup
Now that we have a way to store all of this data let's begin seeing what are the ways that we can view it.
Azure dashboards allow you to combine different kinds of data, including both metrics and logs, into a single pane
in the Azure portal. If you’d like you can share the dashboard with other Azure users.
Elements throughout Azure Monitor can be added to an Azure dashboard in addition to the output of any log query or metrics
chart. For example, you could create a dashboard that combines tiles that show a graph of metrics, a table of
activity logs, a usage chart from Application Insights, and the output of a log query.
Power BI is a business analytics service that provides interactive visualizations across a variety of data sources
and is an effective means of making data available to others within and outside your organization.
You can configure Power BI to automatically import log data from Azure Monitor to take advantage of
these additional visualizations. You can combine data from many sources, including
Analytics queries from Azure Application Insights.
Workbooks provide a flexible canvas for data analysis and the creation of rich visual reports within the Azure
portal. They allow you to tap into multiple data sources from across Azure, and combine them into
unified interactive experience. Workbooks can query data from multiple sources within
Azure. Authors of workbooks can transform this data to provide insights into the availability, performance, usage
and overall health of the underlying components. For instance, analyzing performance logs from virtual machines
to identify high CPU or low memory instances and displaying the results as a grid in an interactive report.
Workbooks can be especially beneficial to ISVs. There are many ways ISV deploy their apps.
In the managed app scenario. ISV resources are in end-customer subscriptions via Azure Managed
Application deployment model. One of the resources is Log Analytics workspace that collects data
from VMs, AKS, and other resources that make up the managed application.
ISV have access to these resources via the "Managed Resource Group" with either Owner or Contributor access .
But, as an ISV you’d want to see a single view of all your customer’s tenants.
If you want to see that single view across all tenants, Workbooks are the most elegant way
to create the single pane of glass right now in the portal.
Alternatively, you can also use Grafana for visualization outside of the portal
Azure Monitor metrics explorer is a component of the Microsoft Azure portal that
allows plotting charts, visually correlating trends, and investigating spikes and dips in metrics' values
Use the metrics explorer to investigate the health and utilization of your resources.
So we’ve talked about visualizing our metrics now we move on to how we can analyzing our metrics to extract meaningful insight.
Now I’ve briefly mentioned Log analytics throughout the presentation so let’s take a moment to talk about what it really is.
Log Analytics is a tool in the Azure portal to edit and run log queries from data collected by Azure Monitor Logs
and interactively analyze their results. You can use Log Analytics queries to retrieve records matching
particular criteria, identify trends, analyze patterns, and provide a variety of insights into your data.
In order to collect data, you'll need to create Log Analytics workspace. These Log Analytics
workspaces are unique environments for data collected by Azure Monitor. Each workspace has its own data repository
and configuration, and data sources and solutions are configured to store their data in a particular workspace.
If you're familiar with the Azure Data Explorer Web UI, then Log Analytics should look familiar.
That's because it's built on top of Azure Data Explorer and uses the same Kusto Query Language (KQL).
Log Analytics adds features specific to Azure Monitor such as filtering by time range and the ability to create
an alert rule from a query. Both tools included an explorer that lets you scan through the structure of available
tables, but the Azure Data Explorer Web UI primarily works with tables in Azure Data Explorer databases
while Log Analytics works with tables in a Log Analytics workspace.
To utilize log analytics work spaces, you mainly use the kusto query language. So a Kusto query is a read
only request to process data and return results. The request is stated in plain text, using a data-flow model
designed to make the syntax easy to read, author, and automate. The query uses schema entities that are
organized in a hierarchy similar to SQL's: databases, tables, and columns.
More help can be found in the documentation linked in the references.
As previously mentioned, Virtual machines and other compute resources require an agent to collect
monitoring data required to measure the performance and availability of their guest operating system and workloads.
The Log Analytics agent collects monitoring data from the guest operating system and workloads of virtual machines in Azure,
other cloud providers, and on-premises. It sends data to a Log Analytics workspace.
The Log Analytics agent is the same agent used by System Center Operations Manager, and you can multihome
agent computers to communicate with your management group and Azure Monitor simultaneously.
Use the log analytics agent if you need to - Collect logs and performance data from Azure virtual machines or hybrid
machines hosted outside of Azure. Send data to a Log Analytics workspace to take advantage of features supported by Azure
Monitor Logs, such as log queries. Or if you Use different solutions to monitor a particular
service or application. The log analytics agent comes with limitations however. These limitations include
the fact you: Cannot send data directly to Azure Monitor Metrics, Azure Storage, or Event Hubs.
it is also difficult to configure unique monitoring definitions for individual agents.
In addition it is difficult to manage at scale since each virtual machine has a unique configuration.
Instead you would want to Use Azure Monitor for VMs, as previously mentioned, which allows you to monitor your machines
at scale and monitors their processes and dependencies on other resources and external processes..
Now we move on to the final part of our presentation.
For the final part of our presentation we will go over how to set up Alerts for monitoring our resources.
We begin with action groups. An action group is a collection of notification preferences defined
by the owner of an Azure subscription. Azure Monitor and Service Health alerts use action groups to notify users that
an alert has been triggered. Various alerts may use the same action group or different action groups depending on the user's
requirements. You may configure up to 2,000 action groups in a subscription.
You can configure Service Health alerts to notify you by text message or email when an Azure service incident affects you.
Then we have Metric alerts. These provide a way to get notified when one of your metrics crosses
a threshold. Metric alerts work on a range of multi-dimensional platform metrics, custom metrics,
Application Insights and standard custom metrics. Finally we have Log alerts.
Log Alerts allow users to use a Log Analytics query to evaluate resources logs every set frequency,
and fire an alert based on the results. Rules can trigger one or more actions using Action Groups.
Log alerts run queries on Log Analytics data. First you should start collecting log data and query the log data for issues.
You can use the alert query examples topic in Log Analytics to understand what you can discover or start
writing your own query. These are some of the common scenarios where a log alert is useful.
Now we’ve gone over a lot this last hour but to summarize, Azure Monitor is a complete platform to give you
insights on your solutions built in the cloud. It goes beyond just data collection and allows you to analyze and visualize what you’ve
collected. Traditionally, you had specialized roles and specialized monitoring tools.
App developers used APM to look at app runtime issues, ITOps used infrastructure
monitoring tools to take care of infrastructure issues and Network admins used packet capture
and other network monitoring tools. With the introduction of DevOps and SRE culture the responsibilities
are changing. Also, with new paradigms such as microservices and containers, the traditional division of infra,
app and network is breaking down. With Azure Monitor, our goal is to help you accelerate this transformation to a
DevOps/SRE culture by giving you a full stack monitoring solution that gives you visibility across app, infra and network
and works equally well with new technologies and legacy applications
I will leave you today with some final words of advice when setting up your monitoring strategy. Always have a plan
and think carefully about what it is you want to monitor and who should be monitoring it.
Make use automation and don’t merely copy your on prem practices
We also have some best practices that are key to keep in mind as you build out your solution.
Thank you all for joining me for this Azure Logging and Monitoring Series. This is a high level over view of what we have to offer
in azure for a logging and monitoring platform end to end. I hope you were exposed to some novel services and concepts
that you can dive into to get started. Don’t forget to check out the learning resources available for the Azure Monitor platform
I hope this can facilitate the discussion on how you incorporate Azure Monitor into your existing solutions.