(mellow music)
- Hello and welcome to this new episode
of Machine Learning Essentials Series.
I'm Nishan Thacker
and in this video, we'll talk about MLOps.
Now, the simplest way to understand MLOps
is to look at it as the application
of Dev Ops principles and practices
to the machine learning workflow.
The goal of MLOps is very simple:
one, faster experimentation and model development,
two, faster deployment of updated models and to production
and three, quality assurance.
MLOps comes into play to streamline the ML process
and define a seamless handoff between data scientists
and the ML engineer or the developer
who's working to take the models to production.
To get to know MLOps better,
we'll try and break it into three core concepts.
Why do we need envelopes? What is MLOps?
And what are the benefits of MLOps?
Let's get started.
So why is MLOps needed? To answer this question,
let's take a look at the typical machine learning workflow.
To build a machine learning model, you start with data,
which traditionally takes up the most amount of time
to clean up and get in shape.
Data can come in different formats and different sources.
The better the quality of data,
the better the quality and efficacy of your model.
Thus, as you keep getting
increasingly better quality of data,
you want to use that to build your new model.
This is our first need for MLOps.
Version of the source data
and its attributes like quality, et cetera
so that you can draw lineage
to the underlying datasets that helped you build the model.
The next step in the ML workflow is to build the model.
This comprises of several sub-steps
like feature selection/generation, algorithm selection,
hyperparameter tuning, fitting the model, etc.
This is also called experimentation
as you're in a trial and error sort of experiment
with various combinations of features, algorithms,
and hyperparameters until you get the combination
that generates a suitable model for you and your business.
But in this case,
the trials that do not reap the desired result
are also important
as they inform the next set of combinations to try.
This is where another set of needs arise for MLOps.
First, to track metrics of the experiment runs
so that one can look back
to determine what to attribute to tweak further.
In addition to that, since all this experimentation
requires some amount of code to be written,
there is also the need to source control the code,
its environment, and any other dependencies
for reproducibility purposes.
Also, in most cases,
as data scientists iterate upon these steps,
a lot of the sub-steps remain the same
and don't need to be run again.
MLOps can ensure that by enabling
the use of concepts like ML pipelines,
you can essentially checkpoint these sub-steps to only run
if something warrants a change.
All these tasks from experimentation, to optimization,
is what is contained in the Build pipeline as part of MLOps.
Now, once the data scientist has been able to create
a model which has acceptable efficacy,
this model needs to be deployed
so that users or applications can start leveraging it.
But before deploying the newly created model,
it is important to validate it
both technically and from a business standpoint.
This validation may even need a staged deployment process.
Again, MLOps can help accomplish this in an automated
and predictable way, by creating release pipelines
that evaluate, test, and package the models into containers
so that they can run anywhere.
It also allows for a QA phase to be created
where all required validation can happen
in a low-cost testing environment
and then allowing for gated or controlled rollout
into production to allow for alpha/beta or red/blue testing.
Here, it is important to note that alongside model metrics,
validation may also involve
assessing the model's most influential features
and the model's fairness.
These responsible AI practices
ensure that models that are created
follow ethical standards.
These also ensure that such automated processes,
eventually also have provisions for human-in-the-loop
and that there is appropriate audit awareness
on the behavior of the models.
Now, you may think that once the model is in production,
that is the end of the process,
but no, that is just the beginning.
The job of MLOps now shifts to monitoring the model.
This monitoring not only happens
from technical metrics perspective
like performance, and response latency
to support high query through put but also for model drift.
Wait, what is Model drift?
Well, a deployed model is based on a definition
of what the business case needs. These needs may evolve.
For example, if you were detecting credit card fraud
and the business now evolved its thinking
around what is a fraudulent transaction,
it will need a rethink or retraining of the model.
This is a simplistic example of model concept drift.
Another drift that is important to understand is data drift.
This is when you train a model
on the demographics of a set of users
and now you`re observing
that the population it is being utilized on
doesn't match that same demographic.
If you're from the data analytics or the warehousing world,
think of these as slowly changing dimensions.
Other examples of data drift include changes in the data
due to seasonality, changes in consumer preferences,
the addition of new products, et cetera.
MLOps can listen to these changes
and trigger automated retraining of the models
so that the new model can now cater to the new requirements.
Well, that was a pretty high-level view of the ML process
and where MLOps is needed as part of it.
Let's now look at what is MLOps
and how it accomplishes these tasks.
At the core, MLOps is a process that enables data scientists
and IT or engineering teams to collaborate
and increase the pace of model development,
alongside continuous integration and deployment
with proper monitoring, validation
and governance of the machine learning models.
As you must have observed
when we talk about the need for MLOps,
it is not just a set of steps you follow
or a product you deploy, but it is a process you engage in
at a depth that depends on your specific needs.
If your scenario just warrants you to engage
at the experiment tracking and automated deployment,
that is all of the process you follow.
If another scenario warrants tracking code, data, and drift
to automatically retrain based on certain criteria,
the engagement in the process is more involved.
All this while, we've been talking about MLOps as a concept.
let's spend a few minutes talking
about how you can engage in MLOps
with Azure Machine Learning.
First, you can create reproducible ML pipelines.
Machine Learning pipelines within Azure ML
allow you to define repeatable and reusable steps
for your data preparation, training, and scoring processes.
This allows for the same checkpointing capabilities
we spoke about earlier.
Second, it enables the creation of reusable ML environments
for training and deploying models.
This again, helps with reproducibility.
Third, it provides capabilities to register, package,
and deploy models from anywhere.
You can also track the associated metadata
required to use the models.
It helps with maintaining a centralized repository of models
and deployments irrespective of where they were created.
Fourth, it enables capturing governance data
for the end-to-end ML lifecycle.
The logged information can include who is publishing models,
why changes were made, and when models were deployed
or used in production.
Fifth, it has provisions for alerting on events
in the ML lifecycle, events such as experiment completion,
model registration, model deployment
and data drift detection.
These can help trigger retraining pipelines
or other actions.
Next, it helps monitor ML applications
for operational and ML-related issues.
Compare model inputs between training and inference,
explore model-specific metrics,
and provide monitoring and alerts
on your ML infrastructure as well.
And finally, it provisions automating
the end-to-end ML lifecycle with Azure Pipelines
or GitHub Actions.
Using these, you can frequently update models,
test new models, and continuously roll out new ML models
alongside other applications and services.
Now this is the most comprehensive set of MLOps capabilities
offered anywhere.
The purpose of these capabilities
is to facilitate data science teams
to collaborate effectively with ML engineers
and build a strong MLOps practice.
What's more, Azure ML also integrates with Mlflow
to enable you to leverage a platform-agnostic
ML lifecycle management service and bring MLOps capabilities
to MLflow pipelines.
Now that we understand MLOps a little better,
let's look at what benefits it provides
and there are quite a few.
To start with,
it enables ML Models scalability and management.
Depending on scope, thousands of models
can be under control of MLOps.
Next, MLOps provides reusability and reproducibility
of ML pipelines, which is terrific.
And it is not just about data scientist
and ML engineer productivity,
but also audit and regulatory requirements.
Also, it provides effortless CI/CD
to serve up-to-date ML models with lineage,
tracing back to the data it is trained upon.
In addition to these, MLOps helps maintaining
ML Model's health and governance
with simplified management of the model after deployment.
And finally, it advocates for responsible AI practices
ensuring model interpretability and fairness.
Ultimately, MLOps is a useful technique for people,
processes and technology to come together
to optimize the creation of Machine Learning solutions.
Now, given there are so many moving parts,
it is common for teams to take time
to adapt to the MLOps way of thinking.
But keeping the rigor and sticking to the process
leads to a robust, scalable
and enterprise-ready ML practice.
Now, as a bonus for sticking with me
on this video for so long,
let me leave you with a set of best practices
for MLOps on Azure.
Create models with reusable ML pipelines
using the Azure Machine Learning pipeline components.
It will help you iterate faster.
Automation is key for robust MLOps.
Use GitHub actions or Azure Pipelines
to automate the full deployment and monitoring
and retraining process.
Monitor performance, not only for the efficacy of the model
but also the underlying infrastructure like memory usage
and query throughput etc.
Monitor data drift and utilize the insights
to retrain the model
depending on business-specific thresholds.
This helps keep the model always updated.
And enable automatic audit trail creation
for all artifacts in your MLOps process
to ensure asset integrity and meet regulatory requirements.
Well, that's it for this video.
Hope you got a good understanding of MLOps now.
To try MLOps on Azure,
please spin up a free Azure Machine Learning workspace at
aka.ms/aml-trial
and download the 30-day learning journey
from the data scientist resources page at
aka.ms/data-scientists.
We also have a dedicated GitHub repository
with MLOps templates and samples at
aka.ms/mlops.
That's it for now, see you in another episode
of Machine Learning Essentials.
Thank you.
(mellow music)