Splunk has hundreds of apps that extend its core capabilities.
The “Machine Learning Toolkit” app delivers new ML Specific search commands, visualizations,
and other tools for performing machine learning to your Splunk data.
Splunk has hundreds of apps that extend its core capabilities. The “Machine Learning
Toolkit” app delivers new ML Specific search commands, visualizations, and other tools
for performing machine learning to your Splunk data. In addition, the application has ML
Assistants that guide you through the process of performing particular analytics building
custom models and operationalizing them on the Splunk Platform.
In this video, we’ll explore the “Predict Numeric Fields” assistant, which helps predict
the value of a numeric field using other fields in the event. This task is commonly referred
to as regression. There are countless uses for regression, from inferring unknown values
to anomaly detection. We will cover those later in next videos.
Note that most of the input and output panels in this assistant have tooltips, giving us
more information about the particular panel. Hover over a panel title or field label to
reveal the tooltip
For this example, we want to highlight any unwanted change in energy output of a power
plant and to do so we will build a model to predict ‘Energy_Output’ using co-related
fields. Next, we will look at the deviations between the predicted ‘Energy_Output’
and the actual ‘Energy_Output’ values, and alert on large differences to find anomalies.
Using the search field, we first load the Power Plant dataset that is packaged with
the app. After clicking the search button, we get the “Raw Data Preview” panel below,
which shows the fields in the dataset.
Next step is to create model for which we need to select the Algorithm to be used for
regression. Splunk’s ML App includes several regression algorithms, each of which are better
suited for different kinds of data. You should experiment with the options to see which one
works best for your data. For this example, we will use Linear Regression.
In this example, we will predict the ‘Energy_Output’ based on Humidity, Pressure, Temperature and
Vacuum. Selecting fields, in general, is a hard problem that depends on the use case,
characteristics of the data, and properties of the learning algorithm, itself, that are
beyond the scope of this video.
Once the “Field to predict” and “Fields to use for predicting” are selected, we
have to split the data into training and testing set. The reason why we should split our data
is that we want to use the training data to create our model which will be used to predict
‘Energy_Output’ and to test the trained model we would need a dataset which our model
has never seen. Let’s pick a 70:30 split to withhold 30% of the data for testing.
We also have the option to fit intercept in the model's equation or remove it. Before
removing intercept, we have to be sure that when all the predictors are having value zero,
then to be predicted value should also be zero. For example, in the case of mobile logs
when the battery percent and the number of open apps are zero we can say that screen
brightness would also be zero. In this case, intercept fit can be unselected.
Note: Do not unselect the fit intercept option when you are not sure.
Next, we have the option to save this model as per a naming convention that we can find
on the “Load Existing Settings” tab; we’ll revisit this tab later in this video.
After clicking the fit button, a model is built using the training data. To evaluate
how well the model predicts the test data there are several visualizations and statistics
generated automatically.
The first visualization, “Actual vs. Predicted Line Chart”, compares the energy output
predicted by the model to the true values in the order they appear in the dataset; better
models produce more similar lines. Next is the “Actual vs. Predicted Scatter Chart”,
which can reveal patterns of misprediction. In this example, it’s clear that our model
underestimates the energy generated on the high end. This particular deviation was not
evident from the first visualization.
Residuals, or prediction errors, are plotted in the “Residuals Line Chart”. This corresponds
to the difference between the two lines plotted above in the “Actual vs. Predicted Line
Chart”. Better models produce smaller residuals. It is evident from this visualization that’s
there are some points having residuals less than -30. The last visualization is the “Residuals
Histogram,” which shows the normal distribution of the residual for this dataset. Different
visualizations can reveal different properties of our model.
The assistant also computes validations statistics. The R-Squared value represents the strength
of the correlation between predicted and actual values. It varies between zero and one; closer
to one is better. In this example, the R-square is 0.92 which means we have a good correlation
between ac-power and our predictors. On the other hand, Root Mean Squared Error describes
the prediction error; closer to zero is better. RMSE value suggests that for every prediction
of ac-power the value could lie between plus or minus the RMSE value. Note that these scores
are not always a good rating, for instance, a doctor predicting if a patient has cancer
or not, getting an R-square as 0.92 suggests that out of 100 successful predictions, 8
predictions could be wrong. That is, some patients suffering from cancer are told that
they do not have cancer. So in this case 0.92 is not a fair score.
Based on the large deviations between actual and predicted values, alerts will be generated.
Using these alerts, an analyst could further investigate for the reason behind the deviations.
These deviations could be due to issues in the power plant’s energy storage, unwanted
leakage, breach in the plant or something else.
The “Fit Model Parameters Summary” provides a summary of the learned model. In this examples,
we’ve used linear regression, so the model has coefficients and intercept. (Note that
a large coefficient does not imply that a field is important.)
On the “Load Existing Settings” tab we can compare models trained with different
parameters using statistics such as R-squared and Root Mean Squared Error, etc. and reload
the one that worked best. Once we are happy with the validation statistics, we can set
the training set to 100% and deploy our model. We can also apply our configuration to new
data. These assistants not only walk you through the steps required to train, validate, and
deploy a machine learning model, but all of these tools can be reused and provide a variety
of options for operationalizing our model. You can view the SPL commands used at every
step, schedule the configured setup of the model, create alerts based on the output generated
or use one of the validation panels on your own dashboard.
To learn more about the Machine Learning Toolkit, including the other Assistants, browse through
our ML videos.