>> You're not going to want to miss this episode
of the AI Show where we do
a whirlwind tour of what's new in Azure Machine Learning.
Don't miss it, tune-in. We'll see you there.
[MUSIC].
>> Hello. Welcome to this episode of the AI show.
We're going learn all about what's
new in Azure Machine Learning with Chris,
my good friend. How are you doing, bud?
>> Excellent. How are you?
>> Good. Tell us who you are,
what you do for those who don't know.
>> I'm Chris Lauren. I'm a PM
on the Azure Machine Learning Server.
>> Fantastic. So my understanding
is that Azure Machine Learning is
supposed to be the one-stop shop for data scientists from data,
all the way to deployed models, is that true?
>> Absolutely, almost.
>> Okay.
>> So in any machine learning experiment,
about 80-90 percent of the work is absolutely finding,
discovering, and working with data.
>> Correct.
>> You've got to find it, prep it,
then you can train and deploy your Machine Learning Model.
So in Azure, we integrate a bunch of different products
together to make this work as seamlessly as
possible across data engineers,
data scientists, and engineers that are going to
deploy web services or there applications.
>> Awesome.
>> So I'll show you how Azure Machine Learning
enables this end to end lifecycle.
>> Fantastic, and show us a little bit about what's new, right?
>> Oh, for sure.
>> All right, let's get started.
>> Sweet. So this is Azure Data Factory,
and one of the things that
Azure Data Factory enables is connecting
a bunch of different data sources like whether it's in S3 buckets,
or on-prem data sources and such.
So we've fetched are data from a bunch of different locations.
On-prem, maybe prep the data in Azure Databricks,
and then we'll save it in Azure data storage,
and then we'll fundamentally train and
deploy a machine learning model using Azure Machine Learning,
which is what we're primarily here to talk about.
So I'm going to show you how to train and deploy
your Machine Learning Model once you've prepared your data.
>> Let's pause right there. So that was Azure Data Factory?
>> Azure Data Factory.
>> I See.
>> Integrated with Azure Machine Learning.
>> That's where you're like almost,
because we have other things that we can
pipe it into Azure Machine Learning, this is one of them.
>> That's right.
>> Fantastic.
>> Yeah. So for example,
I'm going to show you how to train models to
calculate whether a flight is likely to be delayed.
>> Okay.
>> So there are some types of datasets that you may not
have on-prem or readily available,
and so we've released Azure open
datasets to enable discovering that sort of stuff as well.
So we can search for weather data,
we can look at the NOAA data,
for example, and then we can identify
what kind of columns or features are available in that data.
>> I see.
>> Then we can even see how to get access to this data.
It generates a code using the Azure ML Open Datasets.
So you can use this Python package anywhere on
your local machine or in the Cloud to be able to get access
to data so that you can enrich
the datasets you do have with datasets
you don't have to get better more predictive data.
>> I see. So when I started dataset at first, I was like, "Well,
this is very useful to train novel
models," but you're saying that we use this data
to augment the data we
already have to make it even better models with what we got.
>> Yes.
>> I love it.
>> That's right. Absolutely. So now,
one of the things that our customers
have told us a lot is they really like
the shift in Azure Machine Learning to provide
the Python SDK and the Jupyter Notebook experience,
but a lot of people in their companies really like
the drag-and-drop UI experience that we
had in Azure ML studio as well.
So now one of the things I'm proud to
tell you this morning is that we've integrated
that visual interface with
the new Azure Machine Learning service so that
no matter the level of experience you have with data science,
you can train and deploy
great models using Azure Machine Learning.
>> Now, when you brought this in,
are you integrating it with all the other stuff
that was already in Azure Machine Learning?
So like stuff you shared?
>> Absolutely.
So you can use the same set of compute resources,
you can discover the same set of
datasets that are all available within
one cohesive Azure Machine Learning workspace
that you can provide Role-Based Access Security for,
so you can control who has access to what,
and enable your data scientists and engineers
to work together in one cohesive experience.
>> Fantastic. So the drag-and-drop experience is
coming integrated into Azure Machine Learning.
>> That's absolutely right.
>> Love it.
>> Now, that Azure Machine Learning workspace
enables bringing together your compute,
your data, and your model's all-in-one UI.
So you can see I've got some Notebook virtual machines here.
This enables running those Jupyter Notebooks which are
the bread and butter for most data scientists these days.
We also have auto scalable GPU or CPU clusters in the Cloud.
So when you're not using them,
they'll automatically shrink to zero machines.
You're not paying for compute that you're not using.
But notice that it's all available right here.
So we can provision and manage all of our assets,
whether it's compute, data etc,
all in one place.
So we can look in our data stores, for example here,
and we can see the list of data that's available
across Azure Blob or SQL etc in here as well.
But once you've trained your models,
you can store those models in
the Azure Machine Learning Model registry,
and we can see that these models are versioned,
and we have full lineage to see which experiment
was used to train those models as well quickly and easily.
>> The thing I like about that is
sometimes when a model is deployed,
I don't know where it came from.
Is there a way to trace back to
an experiment to the actual data that was used as well?
>> Yeah, absolutely.
So when you drill into each one of these experiments,
then you can see a whole bunch of details about the metrics,
and child runs, and the links back to the individual,
the input dataset as well.
>> That's cool.
>> Let me show you how to train a machine learning model
using one of the other really neat capabilities
called the Automated Machine Learning.
>> Cool.
>> So as a data scientists,
sometimes you just want to determine quickly
whether dataset has any predictive power at all.
>> Right.
>> So you can use the Python SDK in
this Jupyter Notebook to connect to the workspace,
so that's what we were just looking at.
Create an experiment and experiment is just like a project.
It'll keep track of all your attempts to train a great model.
Then we can explore the data,
get a quick profile to understand
the distribution of the different values in your dataset,
and then using open source libraries,
we can certainly understand
the distribution of values plotted visually as well.
>> So this stuff is already available but
the main difference is that I could be running this
on my surface goal because it's connected to
a Cloud VM doing the Notebook?
>> Absolutely.
>> That's cool.
>> Or the Linux Machine Learning desk.
>> Oh, that's right. There's a machine desk too.
>> Yeah, do that. So the Automated Machine Learning
enables taking all those different features
that are available in that dataset
and then identifying different combinations
of algorithms and hyper parameter values
for those algorithms to identify on how to train our best model.
>> Got it.
>> Including data pre-processing or augmentation.
Because sometimes,
extracting individual values are
transforming that input data to create
new columns in your dataset
can help increase the predictive power of your data as well.
So all that happens automatically just by
providing a simple script to get your input data,
and then deciding where you want to
run this Automated Machine Learning job.
So in this case,
we're going to automatically create
a CPU cluster because we're using tabular data,
we don't necessarily need GPUs for us so we can control our costs,
and then we can submit this
using an Automated Machine Learning config file here.
Now, what this does is this enables easily repeating
this experiment over and over in
a Docker container with the same set of the Python libraries,
the right versions so you can make sure that
your code always runs properly, right?
>> So Automated Machine Learning basically is like a for loop
that's smarter that runs over
a bunch of for example scikit-learn models.
>> Yeah, but not just scikit-learn models,
even model ensembles including all of the pre-processing.
>> That's cool.
>> So pretty much any way you could achieve your goal,
it'll iterate over all of
those possibilities to create the best model possible.
>> Got it.
>> You can see here,
we specify the goal by telling it I'm
doing a classification tasks in this case,
is the flight going to be late or not?
Then we can set some parameters like
how many times am I willing to
try this just so I can control the costs.
One of the new things that we've added here
is the experiment exit score.
Basically saying this high-quality bar,
I want to be like 80 percent confident in this case.
>> That's cool.
>> As soon as I get there, just stop.
So they may not go all the way to those 50 runs for example.
>> That's cool.
>> We've added the model explain stability as well,
which is super important in certain industries to understand
why a model predicted the way that it does.
So I'll show you how that works in just a second here.
So we'll go ahead and run this experiment on
that compute that I mentioned above.
Then we'll see right in line here,
how well the different combinations
of preprocessing steps and things work.
As we mouse over in the widget,
we can see the different hyper parameter values and such.
>> I think those things on its own and we didn't do anything.
>> Absolute.
>> Here's some data, let's see if any of you done.
Because usually what I do when I do a machine learning model,
I start with a dumbest models first
like decision tree or Naive Bayes,
logistic regression usually in that order,
and then if it works great, if it
doesn't, then I have to do something else.
But it's a lot of work for me to set
that up myself, this does it all on its own.
>> That's right. Yeah, absolutely.
Then we can see right here in
the widget how well the different attempts are working,
so you don't have to go anywhere to
monitor how well the experiment worked.
>> This feels like a lot of
typing stuff and is there a way I can do this a lot easier?
>> Yeah, absolutely.
So in the UI,
now we can simply create an experiment and we can give it a name,
and then we can select the compute target we want to run on.
>> See Chris, you should elaborate this.
>> Well, but then it doesn't have the same [inaudible].
>> That's true. So we'll select our data.
Now we can preview this data.
We're going to see all the different columns
that are available in this tabular dataset.
We can even click profile to get a quick understanding of
the distribution of the different columns that are in the dataset.
>> That's cool.
>> Then again, specify that this is a classification task.
Notice we've also added forecasting now which is pretty cool.
>> That's cool.
>> Then we specify the column that we want to target,
and then we simply hit "Start" if we wanted to run it,
but we can even have additional controls like
specify if I wanted to choose
a different metric for my target here.
If I wanted to block some of the algorithms.
>> I see.
>> So don't even bother wasting any compute resources on those.
>> If you don't like the researcher that
invented that particular model, don't use it.
>> Absolutely.
>> You don't have to?
>> Absolutely. So that's
how we get started quickly in the Auto ML UI.
>> That's cool.
>> However, there are certain types of things
like vision object detection type of scenarios,
or audio signals, etc,
deep learning scenarios that just don't work well here yet.
We'll work on enabling those someday, but for now,
most classic business problems work with tabular data,
this is an awesome solution.
>> I'm a vision person.
So now you're getting to my area
because everything looks super nice,
but I don't know how to automatically say
how many internal neurons you need for your deep learning network,
and so is there a way we can do that here?
>> Yeah, absolutely.
So using the open source
deep-learning libraries like TensorFlow or PyTorch,
we can run experiments using our Jupyter Notebooks.
Again, powered by the Notebook VMs
that are contained within that workspace,
make it really easy to bring all this together.
So again, we'll connect to our experiment.
We'll load up some training data.
I can even enumerate the different types of
GPU virtual machines that I want to
include in a cluster because I can never
remember the names of these type of virtual machines.
So then we can create again,
an auto scalable cluster using the latest in video GPU VMs,
these are V100s which are pretty awesome,
but they are a little bit pricier than CPU machines.
So again, with a minimum nodes of zero,
this is going to shrink down,
so I'm not paying for it when I'm not using it.
>> I loved the option of having the ability to do it in code,
and also in the new UI as well.
>> Yeah, absolutely.
So then just like before,
I'll go ahead and call experiment.submit to run this in the Cloud.
They'll automatically provision.
>> Scroll up there because I'm looking at this estimator here,
it's literally called TensorFlow,
so there's first-class support for things like
TensorFlow and PyTorch in the SDK.
>> That's absolutely right.
>> That's cool.
>> Yeah. Then we can simply specify there train.py script,
which can be any arbitrary TensorFlow or PyTorch script.
So you don't have to add any special magic
in there to be able to use Azure Machine Learning.
>> So if you already have something that's working,
just put it up there and we got it.
>> That's exactly right.
However, if you want to make some small changes like include
the run.log mechanism in our SDK,
then that will take lost metrics or
any other metrics and it will store them in the Cloud with
that run so then when you have that lineage back
from the model looking at what experiment was run,
then all the key metrics get stored as well.
>> That's awesome.
>> Along with all of your log files.
>> That way when you hire your brand new data scientist,
Sally, who is a rockstar data scientist,
she can come in and look at what
the organization has done
historically and be able to move forward.
Generally, a data scientist comes in and they're like, "All right,
I got to start from scratch," but
here they have all the history of everything ever run.
>> Yeah.
>> That's cool.
>> Yeah, absolutely.
>> But here's the thing, I like TensorFlow,
and we didn't rehearse this at all,
but I like to use TensorBoard,
do you have something for that?
>> I do.
>> What?
>> Absolutely. So you can see it from using Azure ML TensorBoard,
we can export all of those key metrics to TensorBoard format or
use the existing TensorBoard logs if you're
logging into TensorBoard in your experiment,
and then that'll start a TensorBoard session.
>> So that the regular TensorBoard
that I know and love is all here?
>> Yeah, absolutely.
Now, sometimes to get the best quality model,
we need to iterate over
multiple hyperparameters and try and figure out the best values.
So we have an automated
hyperparameters sweep built-in here as well.
So we can see we run multiple attempts to find
the best values to get
the highest quality model before deploying that to production.
>> That's cool. Like different learning rates,
different batch sizes, stuff like that.
>> Yeah, that's right.
Now to facilitate deploying these models to production,
we've automated this process with
Azure Machine Learning and Azure DevOps to
facilitate bringing together the best of what data scientists
do with the best of what engineers
do and enable them to collaborate easily.
>> That's cool because generally, most models die.
The model is great, but then nothing happens,
I wanted to see what we're doing to go the last mile and this is
what you're talking about Azure DevOps
and Azure Machine Learning together.
>> That's right.
>> How does it look?
>> So we can see here I've got
a build which is simply going to run
an Azure Machine Learning pipeline
to train the machine learning model,
and then register that model
in the model registry that we saw before,
different models in the different versions.
So to create that pipeline,
then we'll go through the process of
creating a process whereby
we have the scoring script and the animals script.
This will facilitate creating
a Docker container to make the deployment super repeatable.
>> I see. So for the inference part of the thing,
we can have our own scoring mechanism.
We just have to adhere to whatever protocol we have.
>> Yeah, that's absolutely right.
>> If I remember right, it's just literally
an init method and a run method.
You just have to have those two and you're good.
>> Yeah.
>> Cool.
>> Yeah, absolutely.
So then we can facilitate deploying
those using a release pipeline.
So we can trigger the release to say an Azure Container instance,
run some integration test cases,
and then if they pass,
then we can facilitate deploying that to
the Azure Kubernetes service which is fully
integrated with Azure Machine Learning as well.
>> That's cool. So this is all nice and good.
Can you show us how we can use the model natively?
>> Yeah, absolutely.
So oftentimes,
people simply want to enrich
their data with the predictive power of the model,
and there's no better place to do that than in Power BI.
>> Of course because the pictures are beautiful.
>> They absolutely are.
So in this case,
I have a dashboard that shows how to use
that machine learning model in Power BI,
and we can see the predictions of
flight delays by different airports or by carriers,
and we even used a new key influencer widget
here to see what characteristics of
that data has the strongest predictive power to
determine whether the flight is going to be delayed or not.
>> How do you hook this all up though,
it seems to magical to me?
>> Yeah. So with a Power BI data flows,
we can incorporate these new AI insights which have
either the Azure Cognitive Services
or Azure Machine Learning models,
built right into the data flow,
so it enriches all the rows in
your dataset with the results of that model.
>> That's awesome.
>> So here I can discover all the different deployed models that I
have in Azure Container Instance or an Azure Kubernetes Service,
map the fields of my dataset,
and then I can see the results right here in Power BI.
>> That goes directly into the visualizations.
>> That's right.
>> All right. Well, that was kind of a whirlwind tour.
Anything else you'd like to add?
>> Fundamentally, Azure Machine Learning is
the best place for
enterprises to bring together all of their data,
their compute to empower their data scientists
to collaborate on training great models,
and then deploying those with
the best practices for what we call MLOps
to ensure that those models can actually get out to
production and have high value for your business.
>> Fantastic. Now if you felt like we went really fast,
take a break because that's all that's
new in Azure Machine Learning from data,
all the way to deployed models.
Thank you so much for watching.
Thank you for being here.
We'll see you next time. Take care.
>> Thank you.