Splunk has 100s of apps that extends its core capabilities.
The “Machine Learning Toolkit” app delivers new ML Specific search commands, visualizations,
and other tools for performing machine learning to your Splunk data.
Splunk has 100s of apps that extends its core capabilities. The “Machine Learning Toolkit”
app delivers new ML Specific search commands, visualizations, and other tools for performing
machine learning to your Splunk data. In addition to all this, we have ML Assistants that guide
you through the process of performing particular analytics, building custom models and operationalizing
them on the Splunk Platform.
In this video, we’ll explore the “Predict Numeric Fields” assistant, which helps predict
the value of a numeric field using other fields in the event. This task is commonly referred
to as regression. There are countless uses for regression, from inferring unknown values
to anomaly detection. We will cover those later in next videos.
Note that most of the input and output panels in this assistant have tooltips, giving us
more information about the particular panel. Hover over a panel title or field label to
reveal the tooltip.
Using the search field, we first load the Server Power Consumption dataset that is packaged
with the app. We want to detect any unwanted consumption of power in the servers and to
do so we will build a model to predict ‘ac_power’ using co-related fields. Next, we will look
at the deviations between the predicted ‘ac_power’ and the actual ‘ac_power’ values and create
alerts on large differences to find outliers. After clicking the search button, we get the
“Raw Data Preview” panel below, which shows the fields in the dataset. In this dataset,
Metrics such as ac power, cpu utilization, and disk utilization, are standard metrics
for any server.
Our next step is to create model for which we need to select the Algorithm to be used
for regression. Splunk’s ML App includes several regression algorithms, each of which
are better suited for different kinds of data. You should experiment with the options to
see which one works best for your data. For this example, we will use Linear Regression.
In this example, we will predict the ‘ac_power’ based on total-cpu-utilization, total-disk-accesses,
and total-disk-utilization. Selecting fields, in general, is a hard problem that depends
on the use case, characteristics of the data, and properties of the learning algorithm,
itself, that are beyond the scope of this video.
Once the “Field to predict” and “Fields to use for predicting” are selected, we
split the data into training and testing set. The reason why we should split our data is
that we want to use the training data to create our model which will be used to predict ‘ac_power’
and to test the trained model we would need a dataset which our model has never seen.
Let’s pick a 70:30 split to withhold 30% of the data for testing.
We also have an option to fit intercept in the models equation or remove it. Before removing
intercept, we have to be sure that when all the predictors are having value zero, then
to be predicted value should also be zero. For example, in case of mobile logs when the
battery percent and no. of apps open are zero we can say that screen brightness would also
be zero. Therefore, intercept fit can be unselected.
Note: Do not unselect the fit intercept option when you are not sure.
Next, we have the option to save this model as per a naming convention that we can find
on the “Load Existing Settings” tab; we’ll revisit this tab later in this video.
After clicking the fit button, a model is built using the training data. To evaluate
how well the model predicts the test data there are several visualizations and statistics
generated automatically.
The first visualization, “Actual vs. Predicted Line Chart”, compares the ac_power predicted
by the model to the true values in the order they appear in the dataset; better models
produce more similar lines. Next is the “Actual vs. Predicted Scatter Chart”, which can
reveal patterns of misprediction. In this example, it’s clear that our model overestimates
power consumption on the high end. This particular deviation was not evident from the first visualization.
Residuals, or prediction errors, are plotted in the “Residuals Line Chart”. This corresponds
to the difference between the two lines plotted above in the “Actual vs. Predicted Line
Chart”. Better models produce smaller residuals. The last visualization is the “Residuals
Histogram,” from which it is evident that there are some values below -30, corresponding
to the high-end mispredictions in the Scatter Chart. Different visualizations can reveal
different properties of our model.
The assistant also computes validations statistics. The R-Squared value represents the strength
of the correlation between predicted and actual values. It varies between zero and one; closer
to one is better. In this example, the R-square is 0.93 which means we have a good correlation
between ac-power and our predictors. On the other hand, Root Mean Squared Error describes
the prediction error; closer to zero is better. Note that these scores are not always a good
rating, for instance, a doctor predicting if a patient has cancer or not, getting an
R-square as 0.93 suggests that out of 100 successful predictions, 7 predictions could
be wrong. That is, some patients suffering from cancer are told that they do not have
cancer. So in this case 0.93 is not a fair score.
The “Fit Model Parameters Summary” provides a summary of the learned model. In this examples,
we’ve used linear regression, so the model has coefficients and intercept. (Note that
a large coefficient does not imply that a field is important.)
Based on the large deviations between actual and predicted values alerts will be generated.
Using these alerts, admin could further investigate for the reason behind the deviation. This
deviation could be due to issues in system power supplies, unwanted device plugged in
with the main system(breach) or something else.
On the “Load Existing Settings” tab we can compare models trained with different
parameters using statistics such as R-squared and Root Mean Squared Error, etc. and reload
the one that worked best. Once we are happy with the validation statistics, we can set
the training set to 100% and deploy our model. We can also apply our configuration to new
data. These assistants not only walk you through the steps required to train, validate, and
deploy a machine learning model, but all of these tools can be reused and provide a variety
of options for operationalizing our model. You can view the SPL commands used at every
step, schedule the configured setup of the model, create alerts based on the output generated
or use one of the validation panels on your own dashboard.
To learn more about the Machine Learning Toolkit, including the other Assistants, browse through
our ML videos.