Application Insights provides the APM capabilities in Azure
Monitor by tracking server response times, application
performance metrics and failures, performing
availability testing, dependency tracing and much more.
Application telemetry is forwarded from your application
code to an Application Insights resource and from here you can
monitor, troubleshoot, investigate and perform forensic
analysis using curated App
Insights views in the Azure Portal, or you can access
the raw metrics and logs in Metric Explorer and Log
Analytics, respectively. I'll begin by showing a
dashboard as a starting point to drill into
Application Insights. This is the default application
dashboard and has a summary of many APM KPIs grouped
into domains. Usage, reliability, responsiveness
and browser performance are the main areas, but it also
includes shortcuts which allow me to navigate
directly to my App Insights resource or specific App
Insights blades. I'll kick things off by drilling into
the application map.
App insights uses distributed tracing to track dependencies
across multiple app components and back end services in
microservices environments. One of the ways distributed tracing
data is used is to automatically build an application map to show
who's talking to whom and which components of the app were
dependent on others during a given time period. No manual
effort was required to build this view. simply enable App
Insights monitoring on your app components and App Insights.
takes care of stitching together the traces to generate the map.
Each circle on the map
represents an independent application component that's a
part of a larger distributed microservice application. These
components may run on any number of instances, hence the instance
count of each app component may vary. Moving from left to right,
You can see the client side telemetry and availability tests,
then application instances are in the middle and finally back
end dependencies are over to the right. In this map, those
dependencies are Azure storage resources and databases. The
arrows represent the number of calls between the components.
And they are labeled with the response times for those calls.
Red coloring indicates failures and also shows the percentage of
failed versus successful requests on the calls.
This application map is also starting point for
investigation. I can highlight a single app component and I'm
shown options to look at performance and failures in more
detail or I can switch tabs and move over the infrastructure to
see more info and where the app is running. I'll click on
investigate performance to drill in. This takes me over to the
performance blade. This view lists server side operations and
dependencies by their duration and has a toggle to show browser
page views and dependencies as well. You can filter by a single
operation name to change the scope of the data displayed.
I'll select get home index because it's the slowest one,
and all of the tables and charts are updated to reflect the
selection. Next, I can use the histogram in the top right to
filter down only to slow cases of the operation, where I get to
define slow and this limits the operations shown now to a much
smaller subset than where we started. And then I'm shown the option
to look at groups of transactions with commonalities
or drill into individual
transactions. I'll click on the suggested example to get down to
the end to end transaction view. This is a Gantt chart showing where
my transaction spends its time per dependency and includes
exception details and the call stack to investigate more in
the event of failures.
Based on this view, I can tell none of my dependencies is
responsible for the poor response time, so maybe it's
something related to the application code itself or the
infrastructure resources where the app is running. If it's a
code related issue, maybe I'd open a work item for the
developer, but here let's continue the investigation for now.
I'll drill into the profiler traces which gives me finer
granularity into the classes and methods used by the transaction.
The profiler is collecting sample traces on a predetermined
interval, and you can also configure it to run based on CPU
and memory triggers. I'll select the sample transaction from the
slowest ones on the left hand side and we can see that the
bulk of the time was spent not processing, but just waiting,
waiting for resources to become available to continue its
processing. And now, this point us to some resource constraint
on the infrastructure side so I can return to my app map to see
where the app component is running. The app map allows me
to see the resource details, and in this case it's a VM.
Specifically, it's retailVM01. Now I can go to this VM
and look at its basic resource utilization, items such as CPU,
memory, disk and network IO in the form of time series metrics
to find any underlying issues. Or if your VM is forwarding
performance logs to Azure monitor logs just as your app
code is forwarding logs to
Application Insights, you can use Log analytics to do a more
deep dive analysis and look for correlations. This is a key
workflow that's available when all of your resources have been
configured to emit logs to centralized workspaces where ad
hoc, statistical analysis and machine learning can be used to
reveal unexpected relationships and patterns.
To start this workflow and navigate to logs. the UI will
automatically be scoped to just for the VM of interest, and
I'll be presented with a set of related example queries. I'll go
ahead and look at CPU over time just to see if there's anything
obvious that jumps out at me.
As it turns out, we do see a spike to 100% CPU usage
periodically. Now that you've seen just how easy it is to get
started with example queries, I can use a couple of queries that
my team is written to do the type of visual correlation
analysis that I mentioned earlier. The first thing I need
to do is switch the scope of the logs from the VM to the Log
Analytics workspace which contains logs coming from a
wider range of resources, and this is also where my team's
queries are stored. The query that we'll start with looks at
CPU usage by process.
And here's our culprit process causing the repeating CPU spikes
that we saw earlier, and the query below looks at app
response time and CPU usage of the problematic process on the
same pair of axes. But now it's going to be scaled appropriately
using 10s of milliseconds for app response time and CPU from
zero to 100%, and this definitely shows is a clear cut
increase in app response time every time the previously
identified process pegs the CPU at 100%. That's the root cause
of the application performance
problem in this example.
The performance blade looks at operations and dependencies from
the context of duration and response time. That is, how long
different transactions take to complete. But now we're going to
move on to failures. These are the transactions that fail due
to exceptions and timeouts, or for some other reason that
transactions end prematurely. Again, I'll start with the
application map and I'll look at a component that's showing
failures. I'll select it and now I'm presented with the option to
investigate the related failures, or I can take a step
back and simply click on the failures blade to get to all the
failures that occurred without filtering based on the app
component. On this screen, among other criteria, I can filter by
operation and exception type. These filters limit my data set
to the subset of operations shown below the drill into
button. In this example, it's about 290 odd exceptions and
about 14 debug snapshots that match my selections. Now I can
select a single sample operation to see it's end to end
transaction view. This is the same view that we used to
analyze performance before, but now I'll focus on the exception
causing the failure. I can click on the exception to view its
details and the call stack leading up to it. And depending
on what I find here, I can choose to open a work item and
assign it to the appropriate developer as part of my DevOps
process or if I'm familiar with the code myself, I can open the
snapshot right here, right now. I'm presented with the call
stack and I can highlight the individual methods to see the
function arguments and local variables and this level of
troubleshooting detail is only available if the APM tool being
used to do the monitoring has the option of collecting snapshots
in the event of failures.
So as I start clicking through the methods here, you'll see
that we're missing the value of a local variable, and if I go
far enough back in the stack, I'll find the root cause in this
address validator function where an alphanumeric is being used for
the zip code instead of an integer as required. There are
also options to view the decompiled code or download the
entire snapshot into Visual Studio for further analysis.
So now you've seen how to do performance analysis and failure
analysis based on historical data for the last 30 minutes,
one hour or for even longer time periods. Let's pivot to start
performing real time analysis on the last minute's worth of live
data, down to 1 second granularity. Live metrics enables
you to track incoming and outgoing requests and overall
health. This blade also provides a list of sample telemetry that
can be used to view traces, dependencies and failures with
some additional detail if you decide to click in.
You can also filter the live metrics by a server of
concern for a smaller scope of real time data. The only
limitation here,is this view only holds 60 seconds worth of
data, but you can pause it at anytime for snapshot if
something interesting occurs.
In the previous workflows, I performed troubleshooting and
investigation on demand. I started by looking at
application performance as a whole and when I saw something
anomalous then I drilled down and used filtering to identify the
specifics of the problem.
This is a great capability, but I'd like to point out that it's
very reactive and an operator or analyst has to navigate the UI
to observe and isolate the issue. Smart detections on
the hand are automated warnings about the performance problems
and failures occurring in your
app. They identify failure anomalies by using machine
learning to determine the expected value of the number of
failed requests, and it uses historical baselines to detect
performance anomalies such as response time degradations and
memory leaks. Also, depending on the type of detection displayed,
There will also be account of the affected users that helps you
gauge the extent or the impact on your application's users. You
can click into any of these detections for more details,
including the ability to see the underlying log query used,
profiler traces, response time reports, and the
request properties.
Smart detection is a great tool and provides very meaningful
automatic trend analysis and anomaly detection. But at the
end of the day you also need the ability to create your own
alerts based on signals and thresholds of interest for you
organizations use case. Alerts meet this need so I'll move over
to the alerts blade. Now you can get an across the board view of
all alerts in a subscription, filter to those alerts in an entire
resource group or to those alerts from a specific resource,
all within a user defined time range. These alerts are
displayed by severity. And you can drill into the details for
all alerts or just for one severity level I'll click into
my Sev 1 alerts. Each row shows the alert details
starting with the name and severity and going on to
describe the monitor condition. And this is whether the alert
is in a fired or resolved condition. This is different
from the alert state which is managed by the operator and can
be manually switched to acknowledged or closed as part of your
escalation or resolution process.
Now I'll walk you through the process of creating a new alert. First,
I'll go over to new alert rules to get started. Notice that the
scope of the rule is already prepopulated with the App
Insights resource where I started, so the context has
already been passed in. Next, I'll set the conditions under
which my alert will fire. Here's a list of all of the available
signals from App Insights, and they can also be filtered based
on metrics, logs, or activity logs. I'll use exceptions for
this example, this shows a historical view of exceptions
over time. So I can see the metric values that have recently
been collected to better inform my choice of the threshold and
if the metric you're using supports dimensions you can
filter you alert rule to use one or more of the dimension values.
In my case I can decide what I want to look at, all exceptions
or just those exceptions from the app running on the VM or
exceptions from the app when it's running on the app service
platform. Now I can move on to setting the threshold. This
going to be a static one and done fixed type of threshold. Or I
can use dynamic thresholds,
to determine the threshold
automatically based on past data.
Dynamic thresholds also have flexible sensitivity where you
can choose high, medium or low to adjust how tight your
threshold adheres to the previously seen values. Note
that there's a preview above where you can see the value
being sat by the dynamic
threshold. You can also reduce the amount of noise generated by
the alerts in a few different ways, by configuring the
aggregation period, the frequency of evaluation, or the
number of times the condition must be met before the alert is
fired. Lastly, you need to set the action or set of actions
that will be taken when they alert fires. This is done by
setting the Action Group. These can be configured in advance or
on the fly while you're creating the alert. The choices here
include the typical email and SMS notifications, but also
extend to executing run books or
opening service tickets, or to more custom actions, such
as calling a logic app, webhook or secure web hook.
Up to this point, I've focused mainly on the back end
application performance, what's happening in the application
code, what's happening on the infrastructure hosting the app.
But App Insights also provides great visibility into end user
behavior. Usage Analytics enables you to answer common
usage questions such as which web pages and features are the
most popular ones by the number of users, sessions, or events,
where the users typically drop out of the website or business
funnel. How often and at what time intervals do users return?
With this type of information
available to you? You can better determine when and where to
assign development resources and align your development strategy
with the desired business outcomes. I'll start with the
most basic usage view called Users. Here I have the
flexibility to choose a time
range of interest. Let's say 3 days and the data is first
displayed as the number of users over time, but I can then split
or segment that data using other properties like country or
region for example. Additionally, we can view more
information associated with these users, geographic
location, OS, platform and browser version are being shown
here, but you have lots of other options. You can also have the
option to look at your user journeys by selecting random
users or from your most active
users. Select a single user to view their timeline. The
number of sessions, pages viewed, and any clicks that are
being tracked as events.
In contrast to the single user timeline, which is very narrowly
focused, User Flows offer a more generic picture of where users
go next. Instead of looking at what a single user is doing,
user flows construct a visualization of the session
flows for all users. It shows the subsequent series of pages
and events used based on an initial page or event that you
get to select. In this example, I've started with the home page
and now the homepage becomes the first point of entry for this
user flow. From here you get a quick high level view.
At the user paths from many sessions at the same time. In
this view, you can discern where users go next from specific
pages on your site. Where do users typically leave when the
session ended? Or are there any places where users perform the
same action repeatedly or loop through the same series of
pages? Application Insights data is also available in out of the
box workbook templates.
Workbooks allow you to query data from multiple sources in
Azure, create custom visualizations for reporting and
analysis, and combine multiple UI elements into a single
interactive experience. You can access the workbooks Gallery
directly from your Application Insights blade. This is a
collection of predefined workbook templates focused on
application availability, performance, and usage that you
can take advantage of as is or you can edit them and
customize them to better meet your requirements. Some of the
more common examples are the failure analysis or the
application index workbook.
But let's take a look at the performance analysis workbook to
get started. It shows a count of operations and their response
time by average values and percentiles for multiple
Application Insights resources all in the same report. And this
is only the starting point because each of these templates
can be edited for content and visualizations. The editing
capabilities include access to the underlying Log Analytics
platform where queries can be updated or rewritten from
scratch depending on the objective of the workbook.
While workbooks provide great at scale interactive reporting.
Dashboards are near real time views that allow you to monitor
resources and quickly launch task for day-to-day operations.
They can also be customized to meet the needs of various user
roles and shared across teams in your organization. The benefits
of dashboards include automatic refresh intervals, standard
Azure role based access control and more. However, the key point
about this particular application dashboard being
shown now is the fact that it's
available out of the box and doesn't require any user
input. All of these tiles are automatically based on the
telemetry being sent to your Application Insights resource.
In addition to the workbooks and dashboard shown just a minute
ago, you also have the option to monitor the status of alerts and
resource health of the entire resource group with the
Application Insights resources
stored. This view displays all of the resources group by type
and their related alerts divided by severity. Next I'll show the
option to include the resource health status on the same table
that has all of the alerts and all the links to drill into
insights associated with
specific resources. Finally, this view also has the option to
navigate an application map. The main difference here between
this application map and the one that we showed earlier is that
this map is much wider in scope and includes all of the
dependencies related to all of the Application Insights
resources in the entire resource group. Whereas the earlier
version was much narrow in scope and only focused on dependencies
associated with a single Application Insights resource.
I'll end the demonstration by highlighting that workbooks
aren't limited to App Insights data, but can also be launched
directly from Azure Monitor. Azure Monitor receives metrics
and logs not only from the app layer, but from the guest
operating system, Azure resources and Azure platform.
You can see that the workbooks Gallery now lists templates for
other infrastructure resources, such as virtual machines,
storage accounts, databases and more. Of course, App Insights
related workbooks are still present, but they're now among
other templates available in the Gallery. Remember that
workbooks provide at scale views with the option to drill
down into the details of one resource at a time. Here in the
application failure analysis workbook, I can choose several
App Insights resources, a time range, and evaluate failed
versus successful requests, trends and affected users
that wraps up our overview of Azure Monitor Application
Insights. Hopefully you can leverage what you've seen today
to take advantage of these capabilities.