The result of a machine learning project is either a one-time output like for
example, a clustering specification or a deployment which performs ongoing
scoring as part of a business process, such as credit scoring or pricing
optimization. With RapidMiner you have two options to create models and put
them into production. You can use the design view of RapidMiner Studio to
create, run and schedule processes or alternatively you can also use the more
guided approach of RapidMiner AutoModel in combination with the Model
Operations feature. In this tutorial we are in the latter path and in particular
we are providing an introduction and overview to the Model Operations feature
which is all about deployments. You can access model operations directly through
the 'Deployments' button. When you start it for the first time RapidMiner Studio
asks you to create a 'deployment location' which is basically a dedicated folder
where where things are stored. So when you create your first location you
should consider if you will only do things locally and on your own - then a
local 'location' will be fine. However, if you want to collaborate, use other
computation resources than just your personal computer and maybe even have a
web-service available to integrate your scoring with other systems, then you
should choose a remote location. In each of your locations you can have multiple
independent deployments, so your projects could be spread across multiple
locations and deployments. Note that a remote location means things will be set
up on a RapidMiner Server, so you are required to have access to one. Now
enough of this, I'm sure you got it! For the rest of this demo I am switching
over to a setup where I have a full-blown deployment in a remote
location. This deployment is about forecasting
on-time arrivals of flights in New England. Knowing if a plane will be on
time as early as possible helps with planning ground and passenger logistics,
and so this is an example of a predictive process that will be run
continuously. The 'Dashboard' which you see here is the usual place where you would
check for the latest details or alerts when managing an active deployment.
However, for the purpose of this introduction the better place to start
is the 'Models' tab because this is where you see the models which were created
and added to this deployment via AutoModel. The one listed at the top is your
active model and below are the challengers. We can also see details like
the error rate during training or metadata like 'creation date' and 'author.'
Every time new data is sent to this deployment for scoring all models are
executed by default which allows you to keep track of their performance. If one
of your challengers starts to outperform the active model you can just switch
them. You don't have to come in here and check the performance periodically as
you can create an email alert in the 'alerts' tab. This deployment has been
activated so the predictions could be integrated
through a web service and could be consumed by another application.
The 'integrations' tab has all the details you would need for that including an option
to test things out. Now to resume our tour. On the models tab we can take a
look at any of the models and all its details. For example we can see the
actual model here, but also the training data that was used for training, the code
of the RapidMiner process and all other results and details.
We provide and show this to you in such detail here because it is really
important to have this available for governance and compliance reasons.
For example with 'the right to explanation' in GDPR or any anti-discrimination law you
may be required to document that your model is not biased, how it was built and
how the class for a certain request was determined. Okay now with 'models,' 'alerts,'
and 'integrations' covered let's look at the remaining tabs. First some words
about drift and the 'Drifts' tab. Drift can happen when the characteristics of your
input attributes change over time as a consequence the prediction error may
increase. In the scenario of flight delays it may be that the model was
trained with data of stable, nice weather conditions, but in locations with a
seasonal cycle the weather changes and so the 'weather delay' may experience
drift. Drift can also be an indicator for bias in the selection of the training
data so it is good to monitor it. If the drift is getting too large then the
model should be retrained. By the way, drift monitoring with alerts can also be
set up as shown before in the alerts tab. Another feature built into 'model
operations' which you may already be familiar with from 'AutoModel' is the
'Simulator.' It can be used to test 'what-if' scenarios which helps to get a better
feeling for your model. As an example I can test how delays will impact my
on-time prediction for a certain flight. Alternatively you can use 'optimize' for a
prescriptive approach. In our scenario the question could be: "if the plane is
provided too late for a certain flight then how do we have to influence taxi
time and the departure slot, to still maximize our chance that the plane will
arrive at its destination on time." On the 'scoring' tab you can score unseen data or
upload actuals manually. Feeding back the actuals is a crucial piece of
maintenance work so you can keep an eye on your models ongoing performance over
time. Speaking of performance, there are two ways to see what's going on with
your deployment. The first is the more general one, namely the 'dashboard.' It
gives you a lot of useful, mostly aggregated numbers and details for a
timespan of your choice. Now for the full details the best place to look is the
'performance' tab which is the heart of the model management. To have this tab
show you all the analytics make sure to provide the actuals where you have them.
As first item on this tab you can see the current active model and you could
set your charts to show only that if you want. The main charts are showing you how
many data examples have been scored, what the error rate was and most importantly
how much you gain by using the predictions. For this introduction we'll
leave you with this teaser, for more details on the charts we recommend you
have a look at the help and refer to our other materials on model management.
We hope you will enjoy our Model Operations as much as we do, thank you very much for watching!