>> Hi everybody. My name is John Kemnetz.
I'm a program manager on the Azure Monitor team,
and today we're going to talk about
how you can rapidly Onboard to
Azure monitor to get set up with all of that good data,
application data and infrastructure data,
heading into your Azure environment,
and getting consumed by some of
the most popular monitoring tools
that you might be using.
So, we'll talk about two primary things today.
The first one I want to talk to you about is getting
data onboard it into Log Analytics.
Now, if you're familiar with Log Analytics,
Log Analytics is a really powerful log tool
that allows you to bring all of your log data together,
do advanced analytics, query
over that data, build charts,
and even use these out of the box solutions
that give you visibility into both your infrastructure,
and the applications that are
running on your infrastructure.
So, what we wanted to do is show you how can we
really easily get all that data
from all these areas of
your Azure environment setup to go into
Log Analytics in just a few clicks.
Now, if you've used
Log Analytics or Azure Monitor before,
you'll know that we have
a very source-centric view of the world,
which means most of
that data is going to be routed from a VM,
or from a sequel database,
or even from your Azure activity log,
and if you've got an environment that
involves tens, hundreds of subscriptions,
and potentially thousands of resources,
it's really not easy for you to
manage getting all that set up,
and maintaining the state of that over time.
So, we'll show you how to do that
first with Log Analytics,
but then we'll also show you how to do it without
your monitor routing that data into an event hub.
Now, Azure Event hubs are a very powerful,
high scale, telemetry of venting bus,
and they're a great way to collect all of
your Azure telemetry from
various sources into a single location,
and then connect up to either one of our partner tools.
So tools like Splunk, Q-radar,
Sumo logic, or to your own custom tool.
The Azure Event hub is kind of
a developer tool that would allow you
to collect all this data into your own custom instance,
or even just your own
machine learning analytics tools that you may have
built out as part of
your security or application
development monitoring teams.
So, we'll show you how to do
this first for Log Analytics,
and then show you as well how you can do this
with an Event hub as the output as well.
Azure is an open ecosystem.
We're happy to have both a premium offering
like Log Analytics that makes it super
simple to get set up and analyze
your logs with Azure end-to-end,
but we also want to make sure that you get
access to this data for your own tooling,
and your own custom environments as well.
So, what we'll show you first is a little bit
about how to connect this data into Azure Log Analytics.
So, Log Analytics really
revolves around this concept of a workspace.
This is kind of the container or
the database that all of your logs are going to go into.
So before you even get started here,
make sure that you've gone into the Azure portal
and created a Log Analytics workspace.
Once you've got that workspace,
that resource ID to your workspace,
is what we're going to use to setup
a variety of sources to send data in,
and we'll talk specifically about three different types
of sources of data,
that you can send into Log Analytics Azure specific data.
The first of these is data from Virtual Machines.
Now, if you're using Virtual Machines
to run your applications,
you're well aware that the majority of
the interesting data coming from Azure Virtual Machines,
is admitted to something like
the Windows of analog or to Linux's log.
You may be emitting this data via
your own custom log files that
are on the Operating System,
and using an agent like the Log Analytics agent,
we make it very easy for you to get set up,
to collect any sort of log data,
whether it's coming from a standard channel like ETW,
Windows of end viewer SS log,
or data that may be coming via your own custom mechanism,
a custom log, or
a custom API called a custom pipeline like that.
So, the Log Analytics agent really is
a great way to aggregate this data from a single VM,
and then upload it into your Log Analytics workspace.
Another type of data that you have
available is Azure resource data.
So, once you move out of
the virtual machine space and really what
we would call the compute space,
there is interesting log and
metric data available from your Azure resources,
where you couldn't install an agent.
So, I'll give you an example of
a network security group or an Azure Sequel database.
Chances are you care about those Sequel audit logs,
your network security group events,
all of that data is helpful in
understanding how your application is working,
and you want an easy way to route all of
that data into Log Analytics,
and we do that using what we
call the Azure Monitor Diagnostic Settings.
Azure Monitor Diagnostic Settings are essentially
a capability that allows you to say
what data you want a route from an Azure resource,
and where that data should go.
One of those destinations is Log Analytics.
So, you'll see it's very easy to set this up,
route either your Azure
data coming from within a Virtual Machine,
or from any other Azure resource
to send that data into Log Analytics.
Finally, a core piece of data that you want
to have access to in Log Analytics is
the information about kind of all lap,
what's happening in your Azure subscription.
We would call this your Azure Activity Log data.
The activity log tracks things like
what new resources have been deployed,
what resources have been deleted,
who is authenticated and given access to a subscription,
all of that interesting data
appears in your activity log,
and you can also connect that app per
subscription into Log Analytics.
Now, if you think about this
across all of your Virtual Machines,
all of your Azure resources,
and even all of your Azure subscriptions,
there's a lot of touch points, and so,
what we've done is set up
a few simple steps that help you
get setup end-to-end without
investing a significant amount of time.
So, the first step here is going to be to
enable an Azure Resource Manager built-in policy.
Now, the Azure Resource Manager policy engine
enables you to
automatically deploy the right setup
as soon as a resource gets created.
We call this Greenfield,
or when a new resource gets created,
and I'll show you right here in the Azure portal,
there is a very simple way to
setup one of these policies.
So, all I have to do is click a sign policy,
I've already navigated through
the Azure policies section of the Azure portal,
and then I'm going to basically choose
which policy definition I need.
Once I click this button here,
I'll be able to browse a certain type.
Now, we've pre-built a set of built-in policies for
Azure Monitor and routing
resource level data to Log Analytics.
So, you can just apply
those without having to think about that.
I'll show you one in particular,
which is we are both Windows Virtual Machine,
and Linux Virtual Machine support
to automatically deploy the Log Analytics agent,
and get it set up on
Virtual Machine as soon as
that Virtual Machine is created.
That means that without having to think
about this every time you create a VM,
this just happens without any intervention.
So, I can select one of these policies,
and then if needed,
I can choose a scope for this if I want it to only be on
a certain resource group or a certain set of
subscriptions to find any exclusions,
and then I'll give my Log Analytics
workspace, and click "assign".
Once I've done that, any new Virtual Machine
that I create will
automatically have this policy applied.
So, we showed you how to do that on a Virtual Machine,
the same is also true for
these other Azure resource types.
So, policies are available for both of those scenarios.
Then the step two,
we provide a set of script in power shell,
that with a few simple inputs can be
automatically setup this for
all of your existing resources.
So, these scripts enable you
to in one power shell command,
go out and immediately set up
the agent on all the machines that already exist.
We also have an equivalent for
the Azure activity logs for
all the subscriptions you have that already exist,
and of course for all of your resource level data,
these scripts will go out and create
the right diagnostic settings
routing all your logs into Log Analytics.
All you really need to do is provide the Workspace ID,
you'll let us know which resource type.
So, you can scope by
which resource type you want to enable this for,
as well as a subscription ID,
and then one simple power shell command lead,
you'll have all of your resources set up here.
So, that shows you that in just a few quick steps,
you can get all this data set up to
be routed into Log Analytics,
where you can analyze it,
where you can use solutions,
and you can really understand and make
meaning of the data much
more simple than navigating one by
one to each one of your Virtual Machines,
and configuring those VMs one by one.
I also wanted to share a little bit about how you can do
this if you're not using Log Analytics.
In fact, if you're using Log Analytics,
you can also use this path.
Many customers tell us that they love Log Analytics.
They like being able to route their data
there and use the built-in solutions,
but they also use another tool such as
Q-radar for the security analytics,
and so, they also want to have
an additional way to route the data to that tool as well,
and that's where Event Hubs come in.
Event Hubs as I mentioned,
can handle an extremely high volume of event data,
extremely cheaply, and get that data in near
real-time to whatever logging tool you're using.
So, what we wanted to show you was how easy it is to
equivalently get all of this data
set up to be routed into an Event Hub.
So, all I need to do in the Azure portal,
is basically get my Azure Event Hub,
create an Azure Event Hub,
and then get the authorization rule for that Event Hub.
So, if I were to navigate to the Azure portal,
and navigate to an Event Hub that I've created,
what you'll see is on the Event Hub blade,
I will have a set of shared access policies
or authorization rules that
are associated with this Event Hub namespace.
So, in this case, my policy name
is RuppManageSharedAccess key.
Now, this is that key that you're going to give to
Azure Monitor to setup various types of data to route in.
From that point onwards,
things are going to look a lot like they
did when we set up Log Analytics.
So, first of all,
we have built-in policies for this.
That means that it's very easy for
you to go in and use one of our built-in policies,
that will automatically set up routing
of any Azure data to an Event Hub,
just as soon as that resource gets created.
Then we also have scripts that will set up using
the Event Hub authorization rule to set this up on
any existing resources that
you may already have in your subscription.
So, getting that set up is very simple,
and it can be done for both Log Analytics
as well as for Event Hub.
Now, how do you get started?
Visit us at aka.ms/AzMonOnboarding
for the full guide and access to
download these scripts that will help you to get set up.
You can always learn more about
log analytics or about any of our partners,
that are able to integrate using Event Hubs
using the links that you can
see up on the screen right now,
and there's more to come.
We've been working on making
this easy for you to get set up with
a lot of data being routed to any of these destinations,
and we'd love to hear
your feedback via the Azure user voice.
So, thank you very much for your time,
and I hope you'll take the chance to try this out.