>> You're not going to want to miss
this episode of The AI Show where
Shivani Patel shows us how machine-learning open-source,
and Azure Machine Learning Service go
really well together. Make sure you tune in.
[MUSIC]
>> Hello and welcome to this episode of The AI Show.
As you recall, last week we got to talk
to my new BFF Andy who works on Scikit-learn.
He's a core contributor there,
and we signed an action briefly
in some notebooks with Azure machine learning.
I have here someone who I'm going to ask the hard questions to,
Shivani how you doing my friend?
>> Good how are you?
>> Better. For those that don't know you,
tell us who you are and what you do.
>> I am a PM on the Azure Machine Learning platform team,
and I specifically work on some fun open-source work,
and also integration MLOps, all that stuff.
>> Let's start with the hard question.
Andy, last week showed a little bit of Scikit-learn,
but how integrated are
the open source machine learning
frameworks inside of Azure Machine Learning?
Is it something that you think about?
>> Yeah, I'm really glad you asked. I actually have.
Let's switch over to a slide real quick.
As you can see here,
we do focus on supporting open-source frameworks.
Our whole goal as a platform is to be open and interoperable.
So that means is that we do have support and
make it a lot simpler to submit training jobs
as well as do model deployments with
these pre-installed packages and libraries
for all the common Open source frameworks that you're using.
We definitely have a strong focus on
making sure that we're supporting these Open source standards,
as well as the frameworks.
>> That's awesome. Generally, when I see Open source things
running in a Notebook for example,
it's always running in the context of the Notebooks.
Do you have an example or examples of
Open source frameworks and how they
interact with Azure machine learning?
Then for those that don't even know what
Azure Machine Learning is can you give us an elevator pitch?
Let's start with that one first.
>> Yeah, definitely. Azure Machine Learning Platform is basically
a platform that enables you to track,
monitor all of your training, runs your experiments,
manage your data as well as manage
the models as well as the deployments that you are creating.
So pretty much we support end-to-end ML life cycle.
>> That's an awesome elevator pitch.
But again, like I just said earlier,
when I'm running these TOSS things, usually, all the computation is
happening on the Notebook itself.
How hard is it to move this open source stuff
into Azure ML and not just be running
on my local host Notebook for example.
Do you have any examples of that?
>> Right. Definitely, I can show you some examples there.
I'm going to switch over to
my screen and pop up my example here real quick.
Let me actually first show you what I'm running.
We have a set of these examples in Notebooks that provide
these curated environments that will
essentially have all the requirements to run,
whichever framework you have.
If we click into one of these for example,
we just talked about Scikit-learn, so let's pop into there.
You have all these packages and dependencies already installed.
So you're going from your local environment
and now, you want to run it on let's
say, Azure compute instance or use GPU CPU.
As long as you are calling these environments,
you'll have all those packages already there ready for you,
and I can show you an example of that. Do I make sense?
>> Yeah, so let me see if I'm understanding right.
Basically, what you do is in order to run
pretty much anything it looks like
because now that I'm understanding it,
if you have this notion of a thing called an environment,
and you explicitly declare that environment,
you can use that environment and run it like carbon,
like to stamp out that environment and run the things
inside of those environments, is that right?
>> Yeah, exactly.
You could go ahead and use ones that we support.
Like I said, we pay attention to what
common frameworks people are using,
we want to make sure it's as easy as
possible to go from local to Cloud.
We do have a curated,
that's what we call them curated environments,
or you can go ahead and create your custom ones,
we support that if you have a different framework.
>> This is awesome but it's unclear to me.
I understand the YAML file and describing it.
It looks like basically a
requirements.txt file which you'd
find but it's a little bit different.
How do you translate this YAML file or this basically
requirements.txt file into something that's
running in a different compute contexts?
That part isn't clear to me.
>> Definitely, and I totally understand that and I will show you
an example of it that'll make it a lot easier to see.
Right here, basically I have this
diabetes Scikit-learn, super common example.
So I have launched this Notebook in VS code,
and these are the same examples that I showed you right here.
These are just the examples that I cloned into this workspace.
Now, all I'm doing here is calling
the workspace that I'm working in, fun stuff.
I am grabbing that GitHub repo and then
I'm calling the environment file that you just saw.
So just for a reference,
here's environment file, here is the Scikit-learn example.
I'm literally just calling that file,
and then I'm defining the environment as that,
great, and then defining whatever compute target.
For this one, I'm just going to use
CPU and now you can see what's going on in here.
I have my script, all that fun stuff.
Now, that I defined the environment,
I am basically calling that environment here,
I have this run defined,
the compute target I chose,
and I'm just going to run it.
>> I see. Let's see,
if we can back out for a second,
to see if I'm understanding.
So basically there's this notion of
a workspace where all the work happens because it's
a workspace, these are the jokes.
So it's a workspace and inside
of that workspace you have any number
of things that you call experiment's running,
and each of these experiments need something to
run and an environment run in. Am I getting this right?
>> Exactly.
>> Where do these runs actually live or go?
>> When you see all of these runs,
there's a link here and I've already pulled it up right here.
This is the workspace that I was talking about.
Soplease zoom in a little bit here so you can see it.
We have all the artifacts that you would generally create in
your ML life cycle where as I mentioned we're
tracking end to end here with Azure Machine Learning.
If I go into my experiments and I
see I've ran this quite a number of times,
you can go in and see the latest run that I
just submitted right here,
and you can see the compute target that was submitted by me.
It was a script,
and then I have the script name,
information about it, which
Git Commit it came from as well as any other properties.
So you can track all those runs in this single workspace.
>> As I'm starting to think about this,
one of the biggest problems that we have in
machine learning is this notion of reproducibility
because I might come up with
the best model in the universe on my machine that
had my particular Conda environment with my particular things,
and not being able to replicate it can be a huge problem.
But in this case,
is this running just in a generic compute context?
How is this actually running in its own special environment?
>> That environment that we're creating,
that shared environment that you're creating,
we actually log those environments.
So when we're creating this environment up here,
we scroll back right here,
so we're defining a environment.
It's not only just the environment file
but we're registering it as
type of an asset or artifact.
It enables you to snapshot all those packages,
all those libraries so that someone else could go in and reuse it.
>> That's awesome. If I'm understanding this right is that
in the workspace, you're also able to
collaborate with other people.
It's not just you that can log into that workspace. Is that right?
>> Yeah, exactly. Again, another emphasis
here on the RunConfiguration is saving that environment.
So the actual run that you've created is saving that environment.
But yeah, right here, you can see in the example that
not only me but a few of my colleagues have
also submitted their runs here.
You could pretty much go in and see that information.
This was created by a service principle, that is not a name,
it's another concept but essentially,
someone else would come in and create these runs.
As long as you have access to this workspace,
your admin has given you access,
you can go in and resubmit this run.
>> Awesome. Do you have examples like for example,
because you just showed a Scikit-learn,
do you have something in PyTorch or TensorFlow?
>> Yeah.
>> To give us the same concepts over again to
help them solidify in my head?
>> Yes, for sure.
I can actually extend this one and show you
a little bit how the deployments out of the world works,
and make sure that we support those frameworks as well.
Right here, we have a PyTorch model,
going through the same actions.
You can see you've got the same workspace,
I've got this time a GPU cluster that I've defined.
So I have that same PyTorch example, again, right here,
"Environments" and pretty simple.
We have all these Torch version,
I don't have to worry about reinstalling it,
fun stuff, and then I have that environment defined.
Great. So I will just click around here,
and then defining that
this run is going to have that environment registered with it.
When you go in and do a nice little run here,
and so I have completed this runs.
I can show you in the workspace for that like exactly looks like.
Here, we've got the PyTorch example.
Again, numerous runs, working really hard here.
So you see the one that I just queued.
So here's a previous one that I had completed.
Again, you can see where all this information came from.
I also have the arguments defined here,
and you can see all this information.
I'll go look, didn't log anything there.
You can actually go in and see all the logs,
all information as well as the output model.
Now this is where it gets pretty interesting.
So if you go ahead and see,
this one doesn't have any registered models.
But if we go back,
I can show you an example
of one that does have a registered model tied to it.
So here's another example of that sklearn run.
So I see the run ID here.
So you can see that this run had an output of this model.
So you could do the same thing without PyTorch run.
This is just an example.
So it works the same way.
>> This is really cool.
It's like almost like DevOps
but for machine-learning types of services.
>> Yes, exactly. MLOps.
>> This brings me to the next question because I'm familiar
with frameworks that generate models for machine learning.
But then there's other frameworks that
manage the flow of how things go in machine learning,
and that was like the greatest segue way of all.
I'm thinking about MLflow,
which isn't necessarily something
that you would do to build a model.
It doesn't really fit in the context of like experiments and runs,
but it does because it spans across them.
Could you describe what MLflow is and then maybe tell us
that's something that Azure Machine Learning can support?
>> I see you spying on my little browser here.
No, for sure.
That's why I mentioned we support open-source standards.
We're not over here creating our own thing.
We understand that there's a lot of Open Source packages,
services out there, and people are already using them.
One of the most common ones is MLflow.
So I'm going to just switch over to the slide so
you guys can easily visualize what I'm talking about here.
So MLflow's basically another Open Source platform,
and it enables you to manage
your lifecycle as Azure Machine Learning does.
So similar to what I showed you in the workspace.
We track experiments, we track deployments,
we track models, very similar in that sense.
So they have four components to it.
They have their tracking for their code, their experiments.
They are again very similar to what we do,
the way that they submit runs, there's projects.
So you can think of that as our
run configuration that I just showed you.
Then they have MLflow models,
so you have registered models like we do.
So instead of battling it out,
we are working closely to make sure
that both of these can integrate seamlessly.
So what that means is that if you are someone that it's running
your experimentation or running
your training jobs in multiple Clouds.
You are able to use something like MLflow to track in Azure,
your Azure Machine Learning runs as well
as if you're running it in some other Cloud.
You can use them MLflow to maintain that source of truth.
So whatever you are tracking with MLflow and Azure ML,
you can see both of those runs in both of those services.
>> How does that work? I mean,
you already showed me some code where you did some stuff.
Is it pretty invasive to introduce it or if you already have it?
How does it work now?
>> Yeah, I can actually show you right here.
So let's take the existing example we have.
I know MLflow for those that aren't very familiar,
it's a little bit of a confusing concept,
but they essentially do the same thing that we do with tracking.
There are scenarios where
using both MLflow and Azure Machine Learning
provides our data scientists more of
an end-to-end view of what's actually
going on in their whole process.
So let's take that same PyTorch example,
and I'll show you how you can start tracking with MLflow.
So what you basically do here
is point this MLflow tracking your IT, your workspace.
So what that enables is that all the data, or
all the runs that are happening in
Azure ML will also get tracked in MLflow.
So you're pointing to the specific workspace.
Then I'm doing the same activities.
The only difference here is that I'm also
setting the MLflow experiment name so
that it knows that this is
the experiment that I want to track in MLflow.
Very similar concepts and context.
So you don't really have to learn a lot of
different concepts to use both of the services together.
>> My sense is that that's actually really cool because,
maybe you want to do experiments somewhere else and you want to
do model management and registry with Azure Machine Learning.
Using something that's Open Source like MLflow
allows you to pick and choose where you want to do your stuff,
for example Azure Databricks.
>> Right. Exactly. We have tons of customers that
use all 3D services;
Databricks, MLflow, and Azure ML,
because their scenario is so complex in that sense.
They need that flexibility across the different services as well
as the ability to trace across Databricks Azure.
So MLflow is the answer pretty much for that,
and so it makes it a lot simpler in that case.
So right here, we see that
we've run that experiment with the experiment name.
Everything else is pretty much standard.
We define the run here.
You have to wait for the run to complete.
One thing I wanted to highlight here on the deployment side,
it's very, very similar to the way that you
deploy in Azure ML right now.
So what we've introduced here is doing an MLflow Azure ML deploy.
So in this case,
after the run is completed
we're just pointing to the path of the run.
Even though it is MLflow track,
you don't have to go in and separately register.
The MLflow.AzureML deploy will take
that model that you saw in the run,
let me just show you here.
So if we see the outputs here.
So very similar, I mean,
I'm using this as an example here
but it will look the same in PyTorch.
You have this models folder.
So it'll take the path of this and understand the syntax,
the schema of the model here,
and then go ahead and deploy that model.
So right here, we show you,
pop back over here. Let's look at the endpoints.
So that one was the PyTorch example,
so I can show you the MLflow PyTorch example.
So right here, you can see that I have that model ID,
the model output that was created from that run,
as well as this nice metadata that is being tracked.
So we don't just enable you to track your runs in MLflow,
we want to make sure that you have that view
of the metadata and
the properties that MLflow also tracks on there.
So you'll have the URI,
the runs, data you're using.
This is in the endpoints.
In the models, we do the same exact thing.
You click into let's say that same PyTorch model,
you have all this information already filled for you.
>> So this has been awesome.
You basically showed us how
with Open Source you can create environments.
Well, you create environments
so that open-source can live in them.
We've looked at sklearn in PyTorch,
and there were some other demos in there we'll
put the links below so everyone can see them.
Then you showed us an Open Source framework that looked at
a different aspect of machine learning
that combined multiple things,
not just training runs for example,
and that was ML flow.
That was pretty awesome. So where can people go to find
out more or if they want to learn more about these things?
>> Yeah, we have tonnes of examples
and additional examples that I showed you.
We have some MLflow examples that go a little bit
deeper into [inaudible] or TensorFlow doing auto logging.
I will put the links in the description
below and you can try those out yourselves.
>> Awesome. Well, thanks so much
for spending some time with us, Shivani.
Thank you so much for watching and learning all about
Open Source frameworks inside of Azure Machine Learning Service.
Thank you so much, you have an awesome day.
[MUSIC]