(music)
>> Hi, this is Steve Michelotti from
the Azure Government engineering team.
I'm joined today by Sachin,
my colleague also from the
Azure Government engineering team.
And we're going to talk about monitoring
on Azure Government. Welcome, Sachin.
>> Thank you, Steve.
>> All right. So, I guess the most logical thing
for us to start out with is,
what is monitor on
Azure Government and what can you do with it?
>> Azure monitor is
a very important platform service that
enables a common pipeline
for retrieving and accessing metrics,
diagnostic logs and other type of monitoring related data
from all sort of Azure resources in a common place.
The data then can be visualized on the portal,
can be also accessed via CLIs,
PowerShell and REST APIs and used to further
create Action items on top of
that data like alerting and notifying people.
>> Okay. So sometimes people have
these concerns that if I deploy to the Cloud,
I might not have visibility
as to what's going on my resources.
This Cloud is a scary thing.
I don't know, what's up there and with Azure monitor,
we can get visibility
into those things, that is what I hear you say.
>> Absolutely. And it's one common thing.
So, you have to subscribe to one thing and then,
as new resources pop up and they all go to monitor,
you do not have to change your code to access the data.
It's already there.
>> Okay. So, I heard you talking about different sources.
What are some of the sources we have here?
>> So, most important sources are the computer sources.
That is where websites are running.
That is where VMs are running and
doing a lot of activities,
running the production and systems and things like that.
And those are the very important one to monitor first.
So, some of the sources for
monitoring data are application and metrics,
from your applications as well as the VMs,
if you are using VMs,
Guest OS logs, Activity Logs.
Activity Logs are Azure
infrastructure monitor activities against the resource.
If you delete or create
a resource or perform any activity against it,
these are also a part of the whole pipeline.
>> So, not just runtime information from logs,
but even administrative activities,
like creating and deleting resources.
>> Absolutely. So, some day you come in
and see a new resource popped up,
you can go back to the monitor and see like who
created it and it was created, things like that.
>> Great. Okay. Well, you
definitely got me interested here,
but I think I need you to prove this to me with a demo.
>> Absolutely.
Before we get there,
I also wanted to kind of go back to our slide here.
There are one arrow
that points to multiple ways you can access the data,
so you can read out to other services,
like notification service and things like that,
you can dump the data Archive and the storage account,
use it as APIs for real time,
visualize all sorts of things.
>> Great.
>> So, let us go to the demo.
Okay, so there we are on
the Microsoft Azure Government portal.
I have precreated some
resources that we are going to monitor.
>> Okay, great.
>> So, very first thing we do is
go to the monitoring blade
and you'll see a couple of things.
So, lets us start looking at them.
The very first thing by
default that shows up is activities log.
So, basically it's going to list on
few things that I have done since this morning.
I can do some searching on top of it.
Apply, show me the data for last six weeks,
two weeks, three weeks, things like that.
Also, I can filter the data based off of resource groups,
resources, things like that.
>> So, activity logs are things that happen
with Azure resources like creating resources,
deleting resources or listing keys, that sort of thing.
>> Absolutely.
>> Okay.
>> Next big thing is the metrics.
So, metrics include performance and
how it's related data
from compute services and non compute services,
like if you are running Vet service,
you want to be very careful about CP utilizing on it.
Then I get the CP utilized and goes up.
I want to know then it just hasn't.
So, those are the metrics we're going to look at here.
So, I'm going to quickly select
that is so it's good that I have precreated.
Go into my resource, that this story is a con.
And we want to quickly look at a couple of
most important one that I at
least monitor my stock of every day.
Is and to end latencies
and sort of a latency for my stories.
>> So, already as I've seen you do
this I'm seeing these phrases like activity log and
metrics and alerts and I heard you talk about
these terms back on
the slides and now I'm
seeing these concrete examples now on the portal.
>> Absolutely. So, there is
some activities on my metrics of my stories.
And I can definitely change the time range,
change the type of chart and things like that.
And the cool thing here up is
I can't pin this to my dashboard.
So, I'm going to pin it and I can go back to
the dashboard quickly and
come and look at it every morning.
I do want have to go back to
the Monitor blade and repopulate the chart.
It's right there. I can
visualize it any time I will come to the portal.
>> Okay. And I see that this is
your default dashboard that
you got when you're logged into
the portal and it kind of just got added to
the what you have existing on this dashboard.
>> Yes. So, my dashboard did not have much stops,
so I can add a chart to an existing one,
but if you have a customized workload,
if you are running a website as part of your team,
then you want to create
your own dashboard and I can quickly go ahead and add
the new dashboard here and then you
can add little charts on this dashboard
for various monitoring and charting activity.
>> Okay. So, I can have my own dashboard,
where maybe I have a dashboard for
a certain application or
solution and another dashboard for
a different application and another dashboard
for Network Health or whatever my imagination is.
>> Absolutely. The simplest scenario is you
have a team managing the backend systems and
the other team managing fronton systems and you can have
separate dashboard visited services on.
>> Okay. So, I can I can
pin these to any dashboard I want.
>> Absolutely.
>> Okay.
>> The next thing we will
quickly touch base with is alerts.
So, we talked about activities logs.
We talked about metrics.
What about if I want to get
notified when something changes,
like some new resource gets created or
my metrics start sending me CPU is
reaching the threshold or I'm getting too much latency
even requesting my front end is
requesting data from my back end,
things of that nature.
We can use alert to set
notifications on this and I'm going to
try to get one here.
So, we looked at the metrics for a story the cons.
And this is where I can
simply go and say add metrics alert,
give it a name, couple of other properties,
I can specify how I would like to get alerted,
using emails, or configure,
this is all can be configured.
>> I can email, I can page someone and
wake someone up in the middle of the night if I want.
>> Absolutely.
It can integrate with your other monitoring
and service oriented operations systems.
>> Okay.
>> I'm going to cancel it
and talk about a couple of other things.
Auto scale is another way
to proactively make your services
are scaling up based on the demand.
So, we talked about the metrics and how
the metrics indicated that CP utilized,
the queue lands that disguises and things like.
I can use that information at
a runtime to automatically scale my application.
So, if I'm running a VM skillset or if I'm running
a web service or web app,
I can use those parameters to simply
say double down the number of Norges,
or skill up my VM things like that.
>> So if I start giving you a huge
spike in traffic and I'm
not in front of my computer
to move the slider to scale my instances,
I can rest peacefully
knowing I've already got this proactively set up.
It will scale it up for me.
>> Absolutely.
>> Gotcha.
>> And again, you can put
the notifications it will
tell you that this is what happened.
>> I can email letting me know. Okay, I got you.
>> Actions groups are quick things to combine,
the set of items and
people that you want to get notified
when certain things happen.
So, you can create Action Groups and it
specifies like four numbers for sms,
email IDs and things like that.
We probably go ahead and create one from here.
>> So, this is if I'm notifying
a whole group of people, not just one person.
>> Absolutely. So, if you have
a team you want to notify or
if we have email,
we talked about, if you have sms
that needs to be sent, all that stuff.
The last thing on
this particular blade that we
want to discuss about, is Diagnostic Settings.
And this is how you control for specific resources.
What happens to that data?
We talked about the data coming
from various Azure sources.
We have a common place to have that data.
What we want to do about it,
we can visualize two things in real time.
We can also set alerts,
notifications, things like that.
But what about my historical data?
If I have logs for three weeks,
six weeks, what I'm going to do?
How effectively I can query that?
And this is where you're going to go
to Diagnostic Settings.
And I have precreated a setting,
so we're going to quickly review that.
>> So, let me make sure I
understand where we're going here.
You're saying, I may be monitoring
different sources, web server logs,
event logs, but then,
I can then put those somewhere else?
>> Yes.
>> Historically archive them?
>> Right. So, in the very beginning we talked about
what can be done with this monitoring data.
You can read out to a different service.
You can take action, like alerts notifications.
You can also do auto-scaling.
But at the same time you can archive
this information for future uses,
or if you wanted to go in the past and
diagnose something or audit something.
So, for that, you can choose a storage account and
simply archive your data in a storage account.
>> Okay.
>> It allows you to set the retention policies on
both metrics as well as on diagnostic datas.
Diagnostic data includes the replication logs, IS logs,
cast downs, things of
that nature and metrics
our performance and have the metrics.
>> So we can archive all of this stuff?
>> We can archive all of that.
>> Gotcha. >> The second option,
this allows you to pipe this data to event hub.
So, if you happen to get some points where you want to
have other services retrieve that data, event hub.
And the cool part of this is that I can
go back and retrieve and review
that data any time I want.
So, I have already set up these things,
so I'm going to quickly take a peek at
the storage account and see how
this data actually looks like there.
>> Okay.
>> So, for both diagnostic logs as well as metrics,
we have separate containers created,
and if you can drill into the containers,
there is going to be a long path for each of
the resources from the monitoring data is coming from.
So, you can easily identify and there is going to
be multiple files indexed by days,
hours, and things like that for effective retrieval,
and they're going to be there for
the regions and period that you set.
>> So, you just flipped over here
to Azure Storage Explorer,
which allows us to visually
see our storage account and what's contained in there?
>> Absolutely. But, again, as you can see,
if I'm running a protection service
that has a lot of services running,
a lot of activities are happening.
It's nearly impossible for anyone to go into
the Storage Explorer and try to make sense of the data.
So, how do we solve that problem?
And we want to quickly discuss
our solution on Azure Government,
and that is Log Analytics,
another cool solution on Microsoft as a Government.
Log Analytics is slightly different than monitor.
It does collect a lot of information,
but it has a very good tooling for
quickly analyzing and searching against that data.
And, also, Log Analytics enables you
to collect data from your own prime resources as well.
So, it is complementing
monitors as well as serving other purposes.
But, in this particular conversation,
we want to focus on the integers and bit monitor.
As you can see on the slide,
there are multiple sources the data
can come through, and then, again,
there are several different ways you
can use that data to perform actions,
visualize, and send it to other services,
export, things like that.
>> So, is Log Analytics querying
that storage account you were just showing us in
Storage Explorer or is it querying everything?
>> So, storage account
is just going to archive this data.
When we configure the diagnostic settings to
also send the data to Log Analytics,
there's a separate process in
real time moveing that data over
to a different storage that
is optimized for real time querying.
>> So, a Log Analytics has
its own repository of that day.
>> Absolutely.
>> Okay.
>> You get the very fast search engine on top of it.
>> Gotcha.
>> So, this data, we will now go
to our Log Analytics workspace and try to query our data.
Going back to, and this
is the setting I was talking about,
so "Send to Log Analytics".
And from here, you can create
a new Log Analytics workspace or use
an existing one to send this data to.
I have already done that,
so I'm going to skip this piece,
and we will quickly go back to Log Analytics workspace.
>> Okay. So, we're still in the portal,
but we're now drilling into your Log Analytics workspace.
>> Absolutely.
>> Okay.
>> Here we are on the Log Analytics page.
Now, you can see a lot of stuff going on in here,
but we probably don't have time
to dive into all of these.
So, we're going to focus on just querying the data
that's coming from the monitoring
that we configured earlier.
So, I'm going to go ahead and click on "Log Search".
And this place is going to
allow us to run real time queries against our data.
It has a very efficient query language to be used
for analyzing the data and
also creating charts and other type of visual ideas.
But, there is a cooler thing than this.
I would go to "Advanced Analytics",
because I like to be close to the data.
So, we're going to click that,
but I already have some sample queries.
We clear this, so we're going to
discuss could be touchbase.
>> So, this is a separate portal
from the Azure Government portal.
This is the Log Analytics portal.
But we can link to it directly
from the Azure Government portal.
>> Absolutely. So, this is
just a separate workspace for you to very closely,
intimately kind of engage with the data.
But this is part of the system,
so it's not going to prompt you
for a re-authentication or any of that.
>> Okay.
>> Now, we are going to quickly
review the type of data this has and there
are multiple sources we discussed can send the data.
We're not going to review all of that.
So, I'm going to quickly go back to our topic,
Azure Diagnostic and Activities Log and monitors.
So, this query right here is simply given
me aggregate count on
all persons performed against the resource.
>> And you just ran that query.
>> I just ran the query.
>> That's how quickly it responded. Okay.
>> It is very fast.
And the way this table
is drawing the data isn't what we really need.
So, we're going to go and have some charting here.
And I'm just going to go and do a pie chart.
And basically what it says is
the number of operations performed
against a resource that I had
created and the re-distributions.
So, get keys, recently I'm using keyboard here,
so get keys, create,
and stop persons like that.
The second example is,
since you were looking at the metrics data from
storage and the latencies that we looked in Azure portal,
now, the same data I'm using from Log Analytics
to draw a chart that indicates how
the latencies are varying over a period of time.
>> Okay.
>> Now, I'm just querying this for last 24 hours,
but if there is a data for several weeks,
several months, this is still
going to be very fast retrieval.
Portal doesn't have that long timespan.
It's more for quickly reviewing past day,
24 hours, and hours, things like that.
But, for longer than
that you might want to come to Log Analytics.
>> Okay, so for Log Analytics, you
can really drill down at
a granular level to
exactly the data that you're interested in,
beyond just the high level things that portal gives you.
>> Absolutely, and again this place will
have data from all other sources, from your VMs,
from your own prime machines,
the network, things like that,
so you can slice and dice data
several ways in diagnosing things more.
>> This is just the log query
functionality of Log Analytics,
not even all that other stuff you mentioned.
So, very powerful tool.
Your next video we'll bring you on to do
a deep dive into Log Analytics.
Okay. All right, so
this has been a hugely informative demo.
I guess the last question I have is,
how would someone get
started with monitoring in Azure Government?
It look pretty easy, everything you were doing.
How would one get started?
>> Okay, so here are some resources for anyone to refer,
read about how the monitors,
Log Analytics work, and how to get started quickly.
>> Okay, great. All right, well,
this has been an extremely informative session.
Thanks for coming on and sharing
your knowledge about monitoring in Azure Government.
This has been Steve Michelotti
along with my colleague, Sachin,
on the Azure Government Engineering Team,
talking about all the things you can do with
monitoring in Azure Government. Thanks for watching.