Hi, I'm Philip from the HANA Academy.
In this series of videos, we're looking at the Predictive Analysis Library, which is native to SAP HANA.
In this video, we're going to continue our look at Support Vector Machine for outlier detection.
Now, in the previous video we performed the training phase, we built a model.
In this video, we're going to try to do some predictions to identify on new data whether or not there are outliers.
Now, you can find more information about that in the reference guide, you know where to get that
help.sap.com/hana_platform
then go and find the PAL reference guide in the reference guide section
and if we just scroll down and have a look we can see we've got the One Class SVM.
We already used 'SVMTRAIN' in the previous video, but if we go down a bit further we'll find 'SVMPREDICT'
and this is the one we're actually going to be using to make our calls.
So, you can see we've got the input data, which is pretty much the same.
Just be careful here because actually SVMPREDICT can have four, so different types, just like SVMTRAIN
so make sure you've got the right types of data tables
input and output with the right kind of columns and the right data types in the right order.
So just be careful of that.
For example, you can see that the probability is only relevant to SVMTRAIN when 'PROBABILITY' is set to '1'.
In our case, we're going to be doing predictions using One Class SVM, so actually our result for each input row
we've got the idea our result will show us -1 for outliers or a value of 1 for normal data points.
That's something that you need to be aware of.
So, let's go and have a look at an example.
Now, we go to the GitHub repository, you know where this is by now.
We're in the code snippets for the PAL repository of the SAP HANA Academy
and we've got number 143, which is 'Support Vector Machine Outlier Detection Predict'.
So that's the code we want to now go ahead and run, so I'm going to just copy and paste that script
and I'm going to paste it into Eclipse, which is where I'm going to be running from.
So let's just create a new SQL window in Eclipse and paste that code in.
Now, just as a reminder, this is our original training data, we've got customers with an ID and the four indicators
so we're going to need those four indicators plus the ID for our new data.
Then we've got the model.
This is the model that we created in the previous video when we used the SVMTRAIN algorithm
so that model is going to be input data as well.
So, in our code it's the same procedure that we've seen pretty much every time.
We're going to use the 'AFLLANG_WRAPPER_PROCEDURE_CREATE' to create a stored procedure one-time only
which we'll then be able to call as many times as we like.
It's going to be SVMPREDICT, we saw that just now in the reference guide
and we're going to have three input tables this time:
we're going to have the new data, we're going to have the parameter table, and also the model.
This was an output table in the training phase but it's an input table when we do the prediction phase
and, of course, we'll need a results table to port our predictions.
Is this input row an outlier or not? That's what it's going to be predicting for us.
Now we need to make sure we set up the tables.
Obviously we've made table types
we have a signature table and then called the 'AFFLANG_WRAPPER_PROCEDURE_CREATE' with the signature table to build that procedure.
Then we've got the data and results tables
we'll create empty ones, and then for the runtime we actually have the different options.
So we're going to specify the thread numbers we've got and whether or not we might want verbose output
that's an option we could have with this algorithm, again, that's all in the reference guide.
We're going to create some data.
I've just got a couple of rows here
you'll notice the first row, with ID 151
the values kind of correspond to what we have in the original training data
whereas the values we have for the second row, 152, don't, so that one I would expect, probably, to be flagged as an outlier.
So we're going to call our algorithm.
So call 'P_SVP' with our input data table, our parameters, our model input table
and then we've got our results table, which will have the results in it.
Remember to clean out the results table each time you run it
otherwise you're going to get to see previous results, because it will insert new rows each time.
So, let's go ahead and run that.
It ran, and it was pretty quick, and we can see we've got our results, so for ID 151 that is outlier 1.
Now, be careful, it's just the name I gave that column, maybe, is a little confusing.
A value of 1 means it's normal data, and for 152 we can see we've got a value of -1.
We saw earlier in the reference guide that -1 does indicate that this row is probably an outlier.
So, we've been able to use predictions on our data to identify outliers.
So, what we've seen in this and the previous video is a good look at Support Vector Machine
and how we can use that in an unsupervised way called One Class SVM
to basically train against some data for what is typically normal data
so that then we can provide new data to it later and it will try and identify where we might have outliers.
So a good way of doing outlier detection is to use this Support Vector Machine option.
This is available with HANA 2.0 SPS 01.
You can find more HANA video tutorials on our YouTube channel.
If you'd like to be informed as soon as new video tutorials are published, please subscribe to the channel.
You can also connect with us on LinkedIn, follow us on Twitter, and we have pages on Facebook and Google+.
Code snippets for all of our tutorials are published to GitHub.
Thanks for watching.