Splunk has hundreds of apps that extend its core capabilities.
The Machine Learning Toolkit App delivers new ML specific search commands, visualizations,
and other tools for performing machine learning to your Splunk data.
In addition, the app has ML Assistants that guide you through the process of building
custom models and operationalizing them on the Splunk Platform.
In this video, we’ll explore the “Predict Categorical Fields” assistant, which helps
to predict the class or category of an event using other fields in the event.
This task is commonly referred to as classification.
There are countless uses for classification, from predicting customer churn to the success
or failure of any event.
Note that most of the input and output panels in this assistant have tooltips, giving us
more information about the particular panel.
Hover over a panel title or field label to reveal the tooltip.
For this demo, we will use the track day dataset that is packaged with the app, where we will
analyze the similarities and dissimilarities between the different car’s performance
on track.
This information could help us in filtering out the customized cars and finding a close
replacement for any car.
Using the search bar, we first load the dataset to get the ‘Raw Data Preview’ panel below,
which shows the fields in the dataset.
Our next step is to create model for which we need to select the Algorithm to be used
for classification.
Splunk’s ML App includes several classification algorithms, each of which are better suited
for different kinds of data.
You should experiment with the options to see which one works best for your data.
For this example, we will use Logistic Regression.
Next we will select ‘vehicleType’ as Field to predict and rest of the fields as ‘Fields
to use for predicting’.
Once the “Field to predict” and “Fields to use for predicting” are selected, we
split the data into training and testing set.
We split our data into two sets because we want to test the trained model on a set of
events the model hasn’t already seen.
Let’s pick a 70:30 split to withhold 30% of the data for testing.
We also have an option to fit intercept in the models equation or remove it.
To remove intercept, we have to be sure that when all the predictors are having value zero,
then to be predicted value should also be zero.
For example, in case of mobile logs when the battery percent and no. of apps open are zero
we can say that screen brightness should also be zero.
Therefore, intercept fit can be unselected.
Note: Do not unselect the fit intercept option when you are not sure.
Next, we have the option to save this model as per a naming convention that we can find
on the “Load Existing Settings” tab; we’ll revisit this tab later in this video.
Once we click the Fit Model button, the classification model is created.
Results of applying the model to the test data are loaded in ‘Prediction Results’
panel next to the actual values.
We also get a Confusion Matrix, which is commonly used to validate the classification results.
In this matrix we have two kinds of values
False Positive is the no. of cars for which the model type was not predicted correctly.
True Positive is the no. of cars for which the model type was predicted correctly and
these values are highlighted in the matrix.
We can also check the quality of our classification models using the accuracy and F1 statistics.
Closer to one is better.
Once we get acceptable statistics results, we can investigate the cars which falls under
false negative group to see if there is any issue in the car or any extra modification
done.
We can also see the false negative values to deduce which car shows similarity in performance
with other car.
For example, in our results 4.7% of Audi RS5s were incorrectly predicted to be Chevrolet
Corvette, this could be because of the similar performance statistics between these two car
models.
On the “Load Existing Settings” tab we can compare models trained with different
parameters using statistics such as Precision, Accuracy, Recall, etc. and reload the one
that worked best.
Once we are happy with the validation statistics, we can set the training set to 100% and deploy
our model.
We can also apply our configuration to new data.
These assistants not only walk you through the steps required to train, validate, and
deploy a machine learning model, but all of these tools can be reused and provide a variety
of options for operationalizing our model.
You can view the SPL commands used at every step, schedule the configured setup of the
model, create alerts based on the output generated or use one of the validation panels on your
own dashboard.
To learn more about the Machine Learning Toolkit, including the other Assistants, browse through
our ML videos.