[upbeat music]
♪ THWACKcamp ♪
[gentle upbeat music]
>>I'm SolarWinds Head Geek Chrystal Taylor.
And today I'm really excited to have this presentation
about what you can do with custom properties.
Now, previously, in this THWACKcamp,
we did a primer on custom properties.
So if any of this is new to you, please go check that out.
And there's lots of documentation on custom properties.
And the THWACK community is rife with suggestions
for things you can do with your custom properties.
And today we have Marc Netterfield on who's a THWACK MVP
and you've seen him around the forums as mesverrum,
he's very, very capable and he's full of ideas for the SDK
and the API, and just everywhere in the Orion Platform,
anything to make his job easier.
So welcome Marc.
>>Thanks Chrystal.
You guys had asked me to come on here and talk about
some of the more advanced use cases that I've run into
relating to custom properties.
So I'm gonna run through two examples that came to mind.
The first one,
is a way of using properties to opt out of alerts.
It's easy to implement, but it can have powerful results
for managing your alerts.
And the second one is a little bit more elaborate.
It's gonna go through a method of automatically
testing templates against new nodes when they get added.
And then based on the results of those tests,
it'll automatically set custom properties and ensure that
the correct SAM templates are always applied in the future
without you having to remember and go assign them every time
when you get new nodes added.
>>Awesome. That sounds really interesting and very exciting.
>>So let's get the easy one out of the way first.
This is an example of how to make it so you can have servers
opt out of an alerts without having to worry too much about
it on your admin side.
As you go through the learning process
of getting your monitoring set up,
you'll find that initially most people start out
where they have simple standard alerts
and you apply those across the board.
Soon afterward, you start to see edge cases
and you start modifying the rules
to exempt particular host names or systems
with different kinds of machine types
or whatever kind of logic ends up getting made.
For example, let's say you have a subset of your servers
that are part of an auto-scaling cluster
and they get powered on and off automatically.
So their admins tell you that they don't wanna get
node down alerts for those nodes anymore.
You go in there and you edit your alert definition to exempt
this set of servers based on their naming convention.
And then later on, you have to come in
and do a different set of exemptions.
And then later on another one comes in and after time,
these lists can get really out of control
in a large environment.
A few years from now no one will even remember
if those hosts are being monitored.
Potentially a simpler way to handle all of your exceptions,
instead of editing the alert logic itself
is to create a custom property.
We'll call this one Alert_Management.
Essentially, it's just a text property that you factor
into all of your alert logic.
So if you have a node that needs to be exempted
from node-down alerts, you just add down
to the alert management property and the node-down alerts
decide to ignore that particular node.
It's like muting, but for specific individual alerts.
You can see here where we just add the condition
to the alert where the node property called Alert_Management
is empty, or it does not contain the word down.
Here's another example to exempt nodes from a CPU alert.
And here you can see an example of how to do an interface
bandwidth exception.
You can also string together as many exceptions
as you needed into a single property.
So I could potentially do something like node, memory,
and bandwidth all in one big string and then exempt
from all three different alerts.
I've seen where people will create different, true, false
custom properties for each alert,
but that can get really messy-looking when you have
a large environment with lots of different alert conditions.
As an extreme example,
in the place where I'm just recently started working,
we had more than 90 custom properties
that regulated alert logic. And I'm trying to simplify that.
>>Awesome. I really, really love this.
Simplifies automation of a simple task,
it keeps your alerts down to a minimum.
So you're not worried about managing hundreds and hundreds
of alerts. I mean,
there's hundreds of out-of-the-box alerts alone.
So if that's all you've got going for you,
you definitely need some way of managing that
and tuning out some of that noise.
So this is a really great way to do that.
>>Exactly.
I think it makes it just a lot easier for people
to understand what they are and aren't getting alerted on.
Cause they can just look at the custom property
and see everything except for node-down alerts
is on this server.
>>Yes. Yeah. The more complicated it is,
the less likely someone later on is to come
and make changes that's gonna affect that thing.
>>So the second example always stuck out in my memory
as being an interesting scenario that I ran into
while I was consulting.
This one is a little bit more involved though.
I had a client who had an agent on their servers,
with some log files that they wanted to monitor,
but it needed to be across the entire environment.
The problem they were running into is the software
had been bought out by another company at some point.
So depending on when you installed it,
the process name would be different and the file paths
would be different.
And so setting up the correct SAM template
was a real headache.
There wasn't a good way to know ahead of time,
which processes were gonna be installed on which server.
So I couldn't go off of the server naming conventions
or anything like that.
We just had to try to guess and check as different servers
got added in, which was very tedious.
The solution that we ended up building was a scheme
using custom properties, alert actions, and dynamic groups
to set up a system that automatically checks every node
to figure out which template
should be assigned to that node, and then it continues
to take care of it for us automatically
with no future admin effort required.
>>I love that.
Anything that requires less work in the future,
it might be a little more complicated up front,
but it kind of keeps house on its own
that keeps you from having to constantly maintain something.
And then it makes future work minimal.
>>Exactly. That's what I'm interested in.
If I can do it upfront and then never have to think
about it again, then it's just gone from my memory.
I'm gonna walk you through an example that's similar
to that scenario that I described,
but I don't have that particular program in my environment.
So I thought I would go with an example that we should all
be relatively familiar with,
based on our Orion servers themselves.
In my system, I have polling engines, additional web servers
and a SQL database
that I need to set different templates on.
And so we'll set up the scheme to map the correct template
to the servers automatically.
As with almost everything in the Orion Platform,
there's probably other ways that we could accomplish this,
but I like this method
because it didn't require any custom SQL.
There's no scripting, no interactions with the API,
and yet it still automated away a chore.
>>Yeah. and I'm sure everyone
would love to know that [laughs].
Any ways that you can do it without scripting is happy.
>>So we're gonna start the second example
with creating a new custom property again.
This one is gonna be a node property
and we're gonna call it sorting.
After we create the custom property,
we're going to need to create a set of groups.
So if we go to manage groups and in my case,
I'm gonna start out with one group called Sorting Groups.
This one is just kind of a bucket to put everything else in
so that we can keep them all together.
In the first group, we had a dynamic query.
It looks for any nodes where the sorting custom property
is equal to TBD.
After that, I'm gonna add in three child groups
inside of this. One for each server role of the collector,
the database, or an IIS server.
Should have the child groups, the query is looking for nodes
where the sorting custom property
is set to match the group name.
So in IIS group, we're looking for sorting to equal IIS.
And in the collector group, sorting will equal collector
and database sorting will equal database.
Pretty straightforward.
Next we use the SAM assigned a group feature
to automatically have the templates assigned
to where we need them.
You can see here in the screenshot,
I have a template for each server role.
So we've got our database template,
the IIS template,
and the one that I used for my collector servers.
To make the system work, all three templates
are gonna need to be assigned to both groups.
The sorting group, and then the correct child group.
>>Why is that?
>>So initially when a new server gets added in
we're gonna get it put into that sorting group,
and it's gonna get all three templates assigned,
and then based on whichever template goes up
and has the correct processes running,
then a different alert is gonna trigger off
that sets a custom property that says,
this one has SQL services running.
So it must be a database server.
And that's how it's gonna move them
out of the sorting group
and into the appropriate child group.
In this screenshot,
you can see I assign the database template
to both the sorting group and the database group
like I just said.
I repeat that process two more times for the IIS
and the collector servers to be assigned to their groups
and still the generic sorting group.
To recap, any node with TBD in the sorting property
is gonna go into the parent group
and have all three templates assigned to them.
If they have IIS, database, or a collector
inside of their custom property,
then they're gonna get moved into the child group,
and they'll only have the specific template
that matches the server get assigned to them again.
So the last piece to make this all come together
is that we wanna set up some alert actions
that will automatically move new nodes
through the various groups,
test out all the templates on them
and tag the property as needed.
Dynamic queries don't currently allow for us
to select objects with null value.
So to kick everything off, we have to set up an action
where any node that doesn't have anything
for the sorting group is gonna get TBD set into it.
So our first alert will look for the nodes
and check their main app and make sure
that says SolarWinds Orion.
And that it has the sorting property set to empty.
Empty or null would be the default when you add a new node.
We can leave the reset condition as default,
reset when it's no longer true.
And for our action, we're gonna change a custom property.
Select the sorting custom property and set it to equals TBD.
Because of the automatic SAM template assignment
that we set up in the previous section,
we're gonna know that this will assign all three templates
to the node after the action fires.
Our next step is another alert that figures out if a node
can be moved from the TBD sorting group
and into the correct group for in this case,
we'll use database.
To save myself some clicks,
I'm just gonna copy the alert I created for my first one,
and then use that as my starting point.
I just changed the name a bit,
so instead of saying sorting equals TBD,
I'm gonna go Sorting to Database,
pay attention that the target of this particular alert
is an application because we're checking to see
if the application
is equal to this process example database, and that it's up.
So that will let us know that the SQL Server process
is running on this particular server.
We'll remove the action that got copied over
from our original alert and create a new action
that changes the custom property again.
This time, since the target of the alert is an application,
you get an option where you can pick if you want to change
a property on the app itself or on the parent node.
For this, I'm gonna choose the sorting property
from the parent and set it to database.
After I get that database alert set up,
I'll just copy that, edit it two more times for the IIS
and the collector templates.
By the time I'm done setting all of this up,
the alerts had already fired and gone through the database
and IIS templates and set up their whole sequence of events.
So I couldn't grab screenshots of that as it happened,
but you can see the end results here in this screenshot.
My first rule got all my nodes where sorting was empty
and set it to TBD,
which puts them in the main sorting group,
then that got all three templates assigned to the node.
As soon as one of those templates shows as being up,
then it triggers the second alert,
which switches the custom property for sorting
to match whichever template came up.
You can see in this snippet for my event history,
that the templates were assigned and removed based on how
the different tests went.
And it was entirely automatic,
it was done before I was done editing
the third set of alerts.
So you know that it goes quickly
and you never have to think about it again
when you add a new node.
>>That's awesome.
And for anyone that forgot what the original example was,
since the example we used was Orion servers,
I think the original example is super useful
because it definitely happens for a lot of software
that's been taken over by somebody else
and then the files change.
I mean, even just in SolarWinds software itself,
you've seen some file changes
that were maybe in an acquisition like SEM originally LEM
was an acquisition from TriGeo.
So being that it was an acquisition,
there's still files out there that are titled as TriGeo.
So, depending on what version you're on,
you may not have the right file names and things like that.
So it definitely makes sense to have something like this
where it can do that for you automatically.
And you don't have to go individually
and look for those servers, especially,
it sounds like it was quite a few servers
for the original example.
So it definitely makes a lot of sense
to kind of set something up that's quick
and you never have to worry about it again.
Thank you so much for showing us these innovative ideas
for using your custom properties to make
being an Orion admin easier, really,
cause I feel like that's what they were about.
So I definitely appreciate that you showcase many different,
built-in parts of the product.
You showcase groups and dynamic queries
and the things you can do with your alerts and applications.
And it's very much a lot of things that were involved there,
but it definitely showcased a lot of things
you can do naturally in the product
that don't require a scripting and don't require the SDK
and don't require the API or any of that stuff.
And I think that's phenomenal to showcase kind of
what you can do,
how powerful the product is for an Orion admin.
If you just use a little bit of imagination.
So thank you again so much for doing that.
And if anyone has any questions about custom properties
and how they work,
I did do a primer earlier on this THWACKcamp
called Custom Properties Revisited.
So if you did wanna check that out, you can do that.
There's also lots of documentation
and there's lots of things out on the THWACK forums
that people have shared their innovative ideas
for custom properties.
And please share your innovative ideas for custom properties
because making this job easier is what everyone looks
forward to.
So thanks again Marc for joining me
and showcasing all of this for our viewers.
>>No thank you Chrystal, this was fun.