>> Hi everyone. I'm David Aronchick,
head of open-source machine-learning strategy here at Azure,
and I'm going to be talking to you today about
machine learning operations or "MLOps" as it's called.
MLOps is a way to recognize
what it means to actually
bring your machine-learning to production.
Today data scientists basically
start on their laptops and experiment,
and they have to experiment a lot
because machine learning is great,
but it's really hard to find the solution you're looking for.
So you may stay on your laptop quite some time before you ever
get to a formal Dev environment or bring it to operations.
The reality is that data scientists today will often
start on their laptops
because that's the environment they know best,
and they work and experiment on
those machine learning solutions in order to get
the answers that they're looking for that their businesses need.
But today too often those solutions are thrown
over the wall to
development environments or operations environments,
and violate a lot of best practices when it
comes to good software engineering.
That's where MLOps comes in.
MLOps or ML operations or DevOps for ML is a way to
bring machine-learning to great software best-practices.
MLops gives you a ton of advantages over
just packaging up a model and throwing it over the wall.
It first it gives you reproducibility and auditability.
You know that model was built from
a very specific pipeline and checked
into code that you will always be able to go
back and audit if you need to in the future.
Second, because you're using DevOps,
you'll be able to do automated validation
and checking to make sure that
it's profiled to the exact environment
that you need before you roll it out.
You're also able to bring in important tools
around eliminating bias or enabling expandability.
Finally, it enables automation and observability.
What you want to do is as you roll it out to production,
you want to wire it up with
the correct tools to make sure
that things are operating correctly in production,
and you want to pull all those information
back to your models to train better in the future.
Today what you'll see is something like the following.
On the lower left hand side,
you have a data scientist,
who takes their model and throws it over the wall and
an app developer who ends up
cascading it all the way down to production.
But that data scientists doesn't know what's going on.
They don't see drift,
they don't see performance,
oftentimes they're often their
unsaleable working on their next model.
What you really want to do is start
making the process more reproducible,
instead of just throwing it over the wall,
you check a model into
a model store that can be
compared against various different models.
Then you'll add validation steps,
and you'll want to validate not just after you check the model in,
but as you roll it out to production to
make sure that things haven't changed.
Finally, when you roll it out to production,
you'll want to use sophisticated deployment tools
like AB testing and Canary,
and then feed all that information back
into the original model store for future use.
A complete MLOps solution
will handle all of these various components with
rich services that handled deployment
anywhere not just to the Cloud,
but the edge and the Intelligent Edge.
MLOps on Azure is built to bring your production,
and at Azure we're very serious about MLOps,
and have been investing in a number of
very important services to make MLOps easier.
Already today, we have a whole set of
solutions to make MLOps much, much easier.
First, we allow you to describe your infrastructure as code,
so you can declare exactly what size
deployment for training, or inference,
or any other solution that you want in
code using SDKs and the CLI.
Second, we have the Azure DevOps Pipeline,
which richly handles the ability
to push through from code to production,
and finally we have testing releasing services
that are ML aware including support for things like R and Python,
and ML flow, and Qflow.
What I want to talk though about is
a few key components related to Azure DevOps.
Azure DevOps already supports a number of MLOps solutions,
and it is used millions and millions of times a
day across the spectrum of technology.
We support any language,
any platform, any Cloud,
it's very extensible and allows you to
write raw code if you would like,
and we support rich Cloud native solutions
like containers and Kubernetes.
But as we move forward,
we've also invested in making Azure DevOps ML aware,
and we're going to allow you to support a number of
the key elements of an MLOps pipeline with Azure DevOps.
For example, we give you a whole set of
first-class model training tasks
including things like installing your con environments,
unit testing, rolling out to Azure ML,
or rolling out to any deployment you'd like.
Second, we offer gating and
provisioning based on validation and profiling.
So you can look at your model and
determine what the best amount of cores,
CPU, memory are in order to run your model.
Finally, we help you with automated deployment.
We allow you to package your model using containers,
validate and profile after you've containerized,
and the roll-out to things like ACI and AKS,
some of our richest most Cloud native technologies.
All together, the components of Azure DevOps and Azure ML work
together to make rolling out
an MLOps solution more elegant than ever.
It fits in with our overall rich portfolio
of machine learning tools.
All the wave up from
the most sophisticated pre-trained models down to
machine learning focused and deep learning focused infrastructure.
End-to-end is entire cycle.
Azure MLOps is designed to
help you bring your machine-learning to production.