Hello.
My name is Bernhard Suhm, and I'm the Product Marketing
Manager for the Statistics and Machine Learning Toolbox.
I collaborated on this demo with Drew Halberstadt, Manager
of the Apps Team in the Machine Learning Toolbox.
For quite a while now, you've been
able to train and assess models, transform features using PCA,
even automatically tune hyperparameters interactively
inside the learner apps.
However, some other commonly applied techniques still forced
you to leave the learner apps.
Partition your data so you can evaluate your model
on held-out test data.
Automatically rank features to avoid overheating, or reduce
model size.
And resume your session after performing additional steps
outside the learner apps, such as taking steps
towards deploying the model.
Finally, the learner app still blocked your desktop session,
unless you had PCT.
This video shows how we've closed these gaps,
in the context of building a condition
model for monitoring manufacturing processes
or industrial machinery.
We have data from vibration sensors--
three channels, and distinguish two conditions.
The data is the same as used for the anomaly detection demo
that the Predictive Maintenance team published last summer.
You may already be familiar.
And there, they used the Diagnostic Feature Designer
to extract 12 features.
And we are using those same 12 features in this demo.
The model building workflow begins by loading the features
into Classification Learner.
In addition to choosing different ways of validating
your model, you now can also set aside a percentage as test.
Next, we'll train a bunch of models.
Training all models would take a long time
with this large data set.
Instead, we select several, including a neural network,
which were added in R2021a, in case you missed that.
We train the models in parallel, since we
have Parallel Computing Toolbox and multiple cores available.
To identify the best model, we can sort them
by validation accuracy, and K Nearest Neighbor and Ensemble
come out near the top.
Next, let's assess the size of models
to determine which ones are suitable for deployment
on memory-limited hardware.
We'll save the session, since we need to do
that work on the command line.
To assess models further, we export them as compact models,
and then save them to disk.
You can see that the K Nearest Neighbor model is
much larger than the Ensemble.
That's because the K Nearest Neighbor
needs to store the whole training data
set, which is more than 17 key data points, in this case.
Let's go back to our model training session
and explore whether we can build smaller models.
Good that we saved our session so we
don't have to start all over.
We select features interactively,
by clicking Feature Selection in the tool strip,
and choose MRMR as one of the popular feature
ranking methods available.
Once features are ranked, select the top features
or pick them individually, like before.
After Apply and Save, we retrain a couple
of models with the reduced feature set.
Accuracy remained almost unchanged,
which suggests there was a lot of redundance among the 12
original features.
To further optimize the model, we
can apply hyperparameter tuning.
While we wait, due to the background pool,
our MATLAB desktop session won't block,
and we can revisit the size of these models.
We see the Ensemble just got a tad smaller, while the Gaussian
SVM shrunk by a factor of 8.
Now that hyperparameter tuning finished,
we see it did not significantly improve accuracy.
We can compare models side-by-side
before making our final selection.
Turns out the Gaussian SVM and ensemble
have almost identical accuracies.
While the Gaussian SVM doesn't come up exactly at the top,
it's essentially as accurate as the best,
and it's significantly smaller.
Therefore, we can feel good about deploying this model.
Validation accuracy may not accurately
reflect performance after such extensive iteration
on the models.
Therefore, as final step before deployment,
we measure accuracy on the test set
that we held out at the beginning of the session,
by clicking test all.
With this demo, we showed that you do not
have to toggle anymore between command line and learner apps
to apply advanced techniques.
We took features extracted from the Diagnostic Feature
Designer.
Then we partitioned data three ways,
and we built a couple models.
We saved the session, realized we
needed to make the model smaller,
we resumed the session to select features.
The machine learning apps now support also feature selection,
evaluating models on held-out data,
and later resuming a session, to name the top recent three
enhancements.
You can find this demo in the Statistics and Machine Learning
Applications section of the Documentation page
of the Statistics and Machine Learning Toolbox.
Scroll down to the section for the Industrial Automation
and Machinery Industry, and click on the Build Condition
Model for Industrial Machinery and Manufacturing Processes
Demo.
And that's going to walk you through the steps
I've just shown to you.
I hope you enjoyed watching this video,
and learning more about how you can build machine
learning models in a no-code workflow in MATLAB
Thank you for your attention.