Splunk has 100s of apps that extends its core capabilities.
The Machine Learning Toolkit App delivers new ML specific search commands, visualizations,
and other tools for performing machine learning to your Splunk data.
In addition, the app has ML Assistants that guide you through the process of building
custom models and operationalizing them on the Splunk Platform.
Let’s walk through the “Detect Numeric Outliers” Assistant.
This assistant identifies values that are statistically dissimilar from previous values,
which could indicate that something interesting, unusual, or possibly bad has happened.
Note that most of the input and output panels in this assistant have tooltips, giving us
more information about the particular panel.
Hover over a panel title or field label to reveal the tooltip.
Here we are trying to prevent the system from unauthorized accesses and security threats.
Alerts can be generated whenever unusual number of logins is detected.
Using the search bar, we first load the logins dataset, packaged with the app.
After clicking the search button, we get the “Raw Data Preview” panel below, which
shows the fields in the dataset.
This assistant helps us find outliers in a single field.
We will detect the outliers in number of logins by finding residual between predicted and
actual number of logins, and applying detect outlier algorithm over the residuals.
This will identify any unexpected change in the number of logins.
After predicting the number of logins and finding the residuals for Field to Analyze,
we need to choose a Threshold Method based on the distribution of the data and how sensitive
you want to be to outliers.
Select Standard Deviation if your data is normally distributed
and you don't mind outliers having a significant impact on the outlier threshold. Otherwise,
if you want to incorporate more robustness to outliers, try the other methods. We will
use Median Absolute Deviation method.
Next, we specify a threshold multiplier. Larger multiplier, increases the bandwidth of outlier
envelope.
We have an option to set a value for the sliding window, so you can specify the number of values
which will be used to compute each slice of the outlier envelope. Otherwise, if you don't
specify a sliding window, the envelope is computed using the entire dataset at once,
therefore creating a static outlier envelope. For example, a response time for a server
is often high in evenings rather than mornings which is not considered as an outlier, so
to make our envelope trained upon these gradual ups and downs we need to give appropriate
sliding window value. But in case your data is always expected to have same value such
as controlled room temperature of server room then you don’t have to set the sliding window
value. After clicking the Detect Outliers button,
Outliers in the residual field are identified based on the selected threshold method. These
outliers are then presented using time series visualizations and other tabular formats which
are generated automatically.
The Outlier(s) panel shows us the number of outliers detected across our events, which
is 61 in this case. Note that we can increase the number of outliers by changing Threshold
Multiplier. We also have the ‘Total Events’ panel which shows the total number of events
examined.
Next, we have the time series visualization of the response time field. Anything outside
the upper and lower bound envelope are considered outliers as shown in the visualization here.
We can see that there are few outliers detected around 26th Nov because of Thanksgiving. This
change in login due to the festive season was not known to our model because our model
was never trained on any thanksgiving dataset. The ‘Outlier Count Over Time’ visualizations
shows the time series bar plot of number of outliers detected. This visualization would
be very useful when we have high number Outliers spread across the time.
At the bottom we have a table showing us time, residual, lower bound and upper bound of envelope
and lastly the binary field ‘is Outlier’ where 1 means it is Outlier and 0 means it
is not. Using this table, we can get the exact amount by which the outlier was above or below
the envelope’s threshold.
We can ship these results to the Security Admin for further investigation into the root
cause of change in number of logins.
On the “Load Existing Settings” tab we can compare different setups used for detecting
outliers using statistics such as Number of Outliers, Threshold Methods, Threshold Multiplier
etc. and reload the one that worked best. We can also apply our configuration to new
data. These assistants not only walk you through the steps required to setup an outlier algorithm,
but all of these tools can be reused and provide a variety of options for operationalizing.
You can view the SPL commands used at every step, create alerts based on the output generated
or use one of the validation panels on your own dashboard.
To learn more about the Machine Learning Toolkit, including the other Assistants, browse through
our ML videos.