>> You're not going to want to miss this episode of
the AI Show where we talk about MLOps,
Part one of three,
managing your assets, managing
your experiment, managing your models.
It's all pretty cool. Make sure you tune in.
[MUSIC].
>> Hello and welcome to this episode of the AI Show.
We're going to learn a little bit about MLOp,
I have with me Shivani Patel. How are you doing my friend?
>> Hi, how's the going?
>> Fantastic. Tell us who you are, and what you do?
>> Hi, I'm Shivani. I'm a program manager
on the Azure Machine Learning platform team.
I specifically work on MLOps features.
>> Fantastic. So tell us what we're talking about today.
>> So today I'm going to talk a little bit about assets,
artifacts, and code management
with MLOps on Azure Machine Learning.
I'll show you some examples of how to do that, end to end.
>> So let's start with why is this
something that's important and how can it be helpful.
>> For sure. So it's really important because in
order to reproduce what you're doing on your local machine,
you need to be able to track all the components
that you are creating so you can pass it onto your team,
be able to reproduce
that same model in the context of a different environment.
So it gets really important when you're trying to
scale out your MLOps workflows.
>> Fantastic. So how can you do
this with our tools? Why don't you tell us?
>> For sure. So I'm going to go ahead and
switch to my screen here to show you.
So here we have the Azure Machine Learning Studio.
This is basically where you can see and visualize all your runs,
your artifacts, and so on and so forth.
So right here we basically have
these datasets, experiments, and models.
So what I've done is I've ran
a training process here in my notebook,
and these samples are available in
our GitHub repo for Azure Machine Learning Notebooks.
So what I've basically done here is
set up my development environment,
connect to a workspace.
Then as I'm creating my experiment,
doing my training process,
I'm registering everything. It's right here.
I'm creating an experiment to track everything that I'm creating,
all the runs that I'm going to create.
I'm attaching some compute here
as well as registering the datasets that I'm using.
So this is really important so that you can track
which datasets were used to produce a model.
So you can go back and see if there's any bias in the dataset,
and then I'm going here and
running through my classic training process.
What's really great is that you can go back and visualize
here in your datasets and see that, okay,
here's a dataset that I registered and
see all the information about it and how to
consume it if you want to use it in different model.
Additionally, I can see right
here if I've created any models from this particular dataset.
>> So let's talk a little bit about why it's important under
MLOps to have these things versioned, registered, visualized.
You talked about datasets,
experiments, and what else did talk about? Models.
>> About models.
>> So why is it important to do that?
>> So it is really important first for
a team of datasets to be able to share what models you've created,
how you created them,
what environment you used to build these models.
You can share that information across a team, saving time,
saving effort, and reproducing
a similar model for a similar scenario.
The other piece here is that teams are
generally rotating across some large companies.
We've talked to many customers where they
had issues where one data scientist went ahead and
created a really great model
to solve a business problem just on their laptop.
But being able to pass on that information to another team gets
really difficult if you're not tracking each of these components.
Additionally, on an end-to-end scale,
if you're trying to understand and have visibility, auditability,
and a view on what's actually happening in your pipelines,
you need to be able to have the environment,
the code that you're using properly vetted.
So if you don't have each of these components registered,
there's no way you can have
that lineage or that validation to show that okay,
this piece of the code was registered and validated,
or this model was actually vetted using this dataset.
>> So one of the things that's interesting when we talk about
MLOps or DevOps in general is code.
How do we keep track of code?
>> For sure. So I actually have a little example right here.
So if you are using
get your running your Notebooks or
cloning them from a Git repository,
what you can basically do here
is we'll go ahead and automatically log that information for you.
So right here I finished my run and I can scroll down here,
and when you do a Git.details,
we're logging all the information.
It's a little difficult to see,
but right here we actually track which
repository or which branch you're
using to manage all of that code.
So basically what you want to do is every time you're
developing your training process and notebooks,
you want to make sure that you're pushing those into Git.
It's not necessarily a tedious process.
As long as you're pushing that into master,
we're able to track which
notebook the run or the training scripts are being used in.
>> That's awesome. So if someone already has
a strategy for managing their own code,
for example, let's say they use GitHub,
that automatically will flow
into what they're doing with Azure Machine Learning Services.
>> Exactly.
>> Okay. So we talked about managing a dataset,
we talked about managing experiments,
we talked about managing models,
which are like data science assets.
We also talked about managing code.
What we really did, we just said
manage the code the way you manage your code.
The interesting thing to me is,
if a developer is watching,
the interesting thing that I
think we should categorize managing datasets,
because that's generally not something that's
part of the regular development process.
Why is managing datasets important,
and what can you do in Azure machine learning
service to manage them properly?
>> Right. So reason managing datasets is so
important so that you have these validated datasets.
I've been gone through a cleansing process
that you know that these datasets have been vetted,
they have no bias and I can use them end-to-end for
testing or testing when I'm
pushing my model in a container into production.
So the datasets are not
only important in the model development process,
they are also really important when you're going in and
touching it in the context of the environment.
You want to be able to flow in these new datasets that are being
created so you can go ahead and re-train the model.
So when you have different versions of the dataset,
let's say you start with one set of dataset,
you run the model and new production came in,
a new version of that same dataset comes in,
you want to be able to automatically pick up
that information and track it right here.
So you can basically manage your file,
all your file datasets in a centralized place.
>> That's cool because then
when data scientists are working together,
they're all working with the same code
coming from the same Git Repository,
for example, the same datasets,
and they can view each other's experiments and the outputs of
those things as well as work out with
the artifacts and model management service.
>> Exactly. So one fun thing is you'll go into your experiments.
Here's an experiment that I just ran.
You're able to see the information not only about the run,
but actually see which dataset was used here.
So you have the input dataset right here that gives you
that full lineage to look back and see what's being used.
Believe it or not, I've talk to some customers who
actually pass on their datasets through e-mail.
So this is significantly better way to manage datasets.
>> I've heard of people passing over
zip files in a share somewhere,
which is equally frightening.
Well, this has been amazing.
We learned about how to do MLOps by managing your code,
wherever you are managing, and we
talked about managing experiments.
We also talked about
managing models as well as datasets. Anything I'm missing?
>> No. That's pretty much of it.
>> Fantastic.
Well, thank you so much for spending some time with us.
Thank you so much for watching.
We've been learning all about MLops,
the very first part.
This is the first part in a series of three videos.
Thank you so much for watching.
We'll see you next time. Take care.
[MUSIC]