>> All right. Last one,
are you ready? All right.
I think this is going to be the last one
we're going to be talking about.
There's been three other videos,
where we've talked about how to set these things up.
We've talked about how to actually
do some data preparation.
You have to do some modeling.
We've talked about logging, et cetera.
In this episode, we are going to be looking at how
to actually take this model and deploy it out.
>> That's right.
>> Okay. Show me how to do that?
>> Yeah. So, essentially,
like you mentioned, eventually,
whenever you have a machine learning model,
you want to actually make it
available to be used in some application.
So what we are going to see next is
whatever train model which we thought is the best,
in this case, it's a random forest classifier,
is exported into pickle file.
Basically, it's a serialized file,
and then you can deploy it in the cloud.
>> I see. So this is where
because we've already looked at
the experimentation service.
We've already worked with the DSVM.
Now it's time to look at
the model management portion
of it. Okay. All right let's dive in.
>> So in this case,
it's the same Jupyter Notebook you see.
I just export that particular model as an output,
and it's available in my assets folder.
>> And you used pickle,
but you can use whatever you want
because it's just something,
but then later you remember saying.
>> That's right. So once you have the model,
you would want to score it, right?
So, essentially, what I have
over here, it's a scoring file,
and I tell it that, "Okay, this is my model,
and this is a sample dataframe.
Now generate a JSON file."
The reason you need this JSON schema
is you can tell your Web service that,
"This is how I expect the input,
and go ahead and give me the output."
>> I see. So this thing right here,
and I want to be clear, is maybe
there's some confusion with me.
This file is effectively like
it will take a model that you've built,
and it will score it or do
the inference or do the prediction.
>> That's right.
>> Those are the three synonymous things.
>> Yeah.
>> And the way you're doing it is there's
just a run that's going to
take the input as expected the way you trained it.
And then in the main, it actually
if you run this file by itself,
it will create like a schema that says,
"This is what I'm expecting.
This is what I'm outputting."
>> That's right. So once you have that,
you will again see that file in your project assets.
So the schema is already available over here.
Now let's go ahead and actually create the service.
So deploy it, right? So you remember last time,
I had shown you that you can use the command prompt.
>> Inference.
>> So we go to
the same command prompt which has the Azure demo in CLI,
and then you actually
give the details about
how you want to set up your cluster.
>> I see and what is this cluster?
This is where we're deploying the model string.
>> That's right.
>> Okay.
>> Now you have two options either,
you could deploy it in a Kubernetes cluster
which we created for you,
which is managed by our container service or you could
deploy it locally on
your desktop if you already have Docker installed.
>> I see okay and so what's the difference between
this cluster that's being set up and
the model management piece? What's the difference?
>> So the model management,
it's a single place where
your earlier versions of
the models that's stored, and I'd say-.
>> It's just a physical model.
>> Yes and how it is stored,
that's where we decide that
whether it's a cluster or it's a local deployment.
>> I see.
>> And it gets dragged back.
So think of it this way,
like you have a model in this case which we
call it as income prediction and
then the underlying is how,
where is it actually running?
It could be running locally or it could be running in
the cluster because when you're trying it out,
you may not want to use
the cluster you just want to locally,
but when you're actually putting it in production,
you use the cluster so you can scale it out.
>> So where does the actual model live?
Does it live in the model management service
or does it live in the cluster
or is it living on your local machine?
>> It lives in the cluster or on the local machine,
depending on what you decide.
In this case, we are going to decide
the cluster. So it lives in the cluster.
>> I see and model manager just
keeps track of the ones that you've built.
>> That's right.
>> And it has information about
that model but doesn't have
the actual physical pickle file.
>> No.
>> Okay, okay.
>> So in this case,
the input which I'm giving is
basically set up a particular cluster.
In this case, this is the name of the cluster,
the location where you want to set it up and
this parameter dash dash cluster
that's the magic like we do all the hard work for you.
So essentially, what we do is we create the storage,
we create a container registry,
we create the Kubernetes cluster.
Right before it we use two nodes and then we
also create an application insights.
Because essentially when your model is running,
you actually want to
use how many API calls are there in this cluster.
>> And so if I have multiple different models,
I can set this cluster up once and then
push multiple models to that same cluster, is that true?
>> Yes, you can do that.
>> So this might be something that you do one
time and then later you scale
out your cluster to add
additional nodes to run additional models.
>> Yes.
>> Okay cool.
>> So once you have done that,
you can see that okay,
you can actually see what's the environment like.
So in this case,
you see that the,
this is my clustering which you saw earlier
when was it deployed the cluster size.
Now we also give you an option to autoscale it.
So based on like how much utilization
is there, you can scale it up and down.
>> I see and this is interesting right,
because the model file is self-contained,
you could actually replicate into
several different containers that
are running the same thing and it
scales horizontally really well.
>> That's right.
>> Cool.
>> Yeah. Once you have that particular environment setup,
you actually with just one line,
you can actually create the,
so here what I am giving here it's,
I'm telling it that okay.
You remember the scoring file we did?
>> That's right.
>> Yeah. So, that's the file
and this is the model which we
created and then the decent
schema which was output of that scoring file, right?
And then this is the name of the application.
So I'm just calling it a simple name.
It's a Python-based stuff and then the conda file.
>> Which we saw in a couple of videos before and
it's interesting right because when
I'm looking at this you have,
these three files to define the service,
how to run the inferencing?
What you're running the inferencing on?
And what the shape of the actual inferencing model is?
>> Yes.
>> And then it just sets up and sort of thing.
>> Yeah so it's like one command
and it can take like 5-10 minutes,
but it will do everything for you.
So the output of this is
essentially a service which is now running in
that cluster and there will be
an HTTP URL which you can call in your application.
>> I see and can you show us a little
bit like because my guess is
that this is like almost we're making
our own cognitive services with our own custom model.
>> Yes.
>> Is there like a rest end point?
How can we see what available calls there are?
>> Yeah perfect. So actually,
we show you all the options which are available.
So there is a usage command and this is
my deployed model which is just going to run that.
So the output of the command is
essentially telling you how many ways you can call that.
So this is the IP address of the container which is
deployed in Azure right now.
This is the way like how you call the header and
we believe in security as you know, right?
So essentially, there are
authentication keys which you need to
generate the first time which you will
use in your code when you're calling this API.
>> And effectively when you're doing a GET or a POST
I think you just add in the header or bearer token.
>> That's right.
>> And you get that by running this particular command?
>> That's right, that's right.
>> Cool.
>> And the other way is,
you can even call this application using
the CLI or PowerShell or any other way you would like.
We give you a sample output over here.
>> And so if I want to run this right now I just run
this top command right here and it
even gives you sort of shape of what it is.
>> That's right. So in fact,
why don't we go ahead and try this in
the CLI and then I'll show you how to call HTTPS one.
>> So it looks like false?
>> Yes, that's right. So the income is
less than 50K for this particular person.
>> That's fantastic and
because of the way that it's formed you can actually
return maybe even a confidence for or a tuple of
something else or you can return anything
you want and that way you can have more,
but this is an easy way to show it as well.
>> That's right. You're right. So, depending
on the machine learning experiment,
the way you have set it and output, you can do that.
>> Fantastic. Well, what else you've got to show?
We've gone through like soup to nuts almost.
>> Yeah. So one last thing I think
probably I can show is the HTTP-API.
>> Oh of course, let's take a look at that.
>> So again because I already
have the Python environment in my ML Workbench,
I'm just going to call my Python code.
So in this case, I'm providing the URL which you
saw earlier the API key and then I'm just passing it,
and I can run it over here. When it's running,
so what I did was essentially gave an input DataFrame
and to that URL and then asked my workbench to run it,
but you could run it outside in command prompt anyway.
>> Sure.
>> And then I'm doing a print output.
So essentially, in this case,
it was a different person and it's
telling me that income is greater than 50,000.
>> And this I mean, if you're looking
at this Python code this is
just any kind of HTTP call via C# dev,
and just an easy call.
You're doing a Ruby call or JSON call or JavaScript call,
it's just a GET command that
you're doing a but you're
passing in some headers with it.
>> That's right and we give you
some sample examples on our documentation site,
where you can know how to call it extra.
>> Fantastic. Okay so I think we've gone from
overview all the way to
actually operationalizing a particular model.
You've got it. Why don't we go
through just a quick overview,
just to summarize what we did?
>> Okay perfect and in fact I
can show you visually how, what we are doing.
So here what we did was we brought in,
the goal as you know,
it's to actually predict the income of a person
given a set of demographic data.
So we have stored this data in
Azure Blob storage and then we
used Azure Machine Learning workbench in
order to create this experiment then we created
a machine learning model which we
exported as a pickle file or it could be any other file,
and then we deployed it in the cloud.
For the experimentation itself,
we use a data science virtual machine which we
associated it with the notebooks within workbench.
Now you could use
actually Insight or things
like that but that's how it works.
And then for the deployment,
again because it's dockerized we use
the Azure Container Service of Kubernetes based cluster,
but you could even use local or you could even
deploy it in IoT Edge or
things like that, or the SQL Server.
>> I mean, it's pretty amazing.
I know it's been a couple of videos,
but we went through really fast
data science process where we are probably
going to spent the most time in the experimentation area.
Trying to make sure things work with Dataprep,
et cetera, and then obviously,
like some of these commands we
made it go a little bit faster,
but it takes a while to create cluster.
So why don't you give some expectations
on how long some of these things might take?
>> Sure. So that's a good point.
So the first thing is creation of
the data science Virtual Machine itself,
it can take up to five minutes depending on what it is.
Again time can vary so that is one part.
The second thing is the cluster deployment essentially,
so it can take up to 20 minutes because we
are creating the entire environment for your storage.
>> You're making is essentially like a whole,
back in the day we used to have these buildings with
all these computers that we'd have to like install stuff,
you're literally creating a data center.
A mini data center for this.
>> That's right and we also
associate scaling capability and
insights so that can take up to 20 minutes.
The other time-consuming thing could
be the creation of the Docker image itself.
So, as you remember we associated
that virtual machine with our notebooks experience.
So we're there, we're actually creating a Docker image,
so that can take 5-10 minutes as well.
>> But once you do these things one
time usually it's pretty fast.
>> Yeah. It's pretty fast.
>> Because I imagine that you're not going to be creating
a cluster for each model that you're going to put out.
I also imagine that you may have multiple VMs,
but not very many
and once they're started they're good to go.
And then creating these things once sets everything up.
>> Yeah, and the other thing is also the pricing,
like you can control how
much you are charged because now you can
stop those virtual machines
or stop the cluster when you are not using.
>> Fantastic.
All right so where can people go to find out more?
>> So I have the link over here on the slide
and essentially we walk you through starting videos,
whatever we described in this video series.
So the setup part,
how to run your experiments,
how to deploy it and
everything you need to know it's all in there.
>> Awesome. Well, thanks so much for
this whirlwind tour of
the Azure Machine Learning Services.
There's a lot to enjoy and to use.
Make sure you check it out, we'll put links below.
Thanks for watching and we'll see you next time.
Take care.
>> Thank you very much. Yeah.