>> You're not going to want to miss
this episode of the AI Show where we talk about
running Azure Machine Learning anywhere with Saurya.
Make sure, you tune in.
[MUSIC]
Hello and welcome to this episode of
the AI Show where we have a special guest,
Saurya Das, to talk all
about running Azure Machine Learning anywhere.
Why don't we start with introductions?
Saurya, why don't you introduce yourself?
Tell us who you are and what you do, my friend.
>> Sure. Hi, everyone.
My name is Saurya Das,
I'm a Product Manager in the Azure Machine Learning thing.
I focus primarily on the hybrid strategy
and I do that through the integrations with Kubernetes.
>> Fantastic. Tell us
a little bit about Azure Machine
Learning for those that don't know.
>> Azure Machine Learning is the super product
from Azure that allows you to do everything machine learning.
Machine learning is big.
There's way more in Machine Learning. It's too huge.
There's data preparation.
There is training models using multitudes of frameworks.
You have PyTorch, TensorFlow,
MPI, and then there is serving, goal inferencing.
There is reinforcement learning.
The space keeps evolving.
There are pipelines, MLOPs.
There are open source projects out there like
[inaudible] and others, Airflow.
It's all there and we have a suite of
products that help you solve these problems.
>> Looking at the title, it says
running Azure Machine Learning anywhere,
are folks restricted to
only Cloud Compute when
we're doing things in Azure Machine Learning?
>> No, it's not just the Cloud.
The idea is to move beyond the Cloud so allow you
to have a consistent experience in the Cloud,
and then anywhere you have Kubernetes clusters.
Standardizing on Kubernetes, which allows you to extend anywhere.
Like I said, just includes AKS,
GKE, AWS, there's container services in the Cloud.
Then you've got the on-prem solutions.
You could have Azure Stack Hub or Azure Stack Edge,
or you can have any Kubernetes cluster on the on-prem,
which would bring to Azure through Azure Arc for Kubernetes.
I can give you a bit about Azure Arc Kubernetes if you want.
>> I think you're going super fast because I think
you said that you can run
Azure Machine Learning jobs on GKE,
Amazon, and even on-prem.
Did I hear this right?
>> Yes, that's correct.
>> How is that even possible?
I know you said Arc, but can you
explain what Arc is and how this makes it possible?
>> Azure Arc is this umbrella project and that has
a specific line of product for Azure Arc enabled Kubernetes.
What Arc allows us to do
is you have Kubernetes clusters like I said,
across Cloud and on-prem.
You install Kubernetes and then you
install the Azure Arc connected [inaudible] .
What that allows us to do is give it
an Azure Resource Manager ID and
allows us to track these on-prem
clusters or other Cloud clusters in a single pane of glass,
as I will show you in the Azure Portal, centrally govern them,
manage them, apply policies, monitor.
You can also see
security hazards and all that. Everything is controlled
>> Well, listen, I feel like we need to take
a look. Can we do that?
>> Yeah, sure. What you see here is the Azure Portal.
I've customized this to show me "Resource Groups,
Machine Learning, and Kubernetes Services."
But you can customize this to show anything.
Let's go to Kubernetes services
because that's the most [inaudible] part here.
As you can see, this
UI shows me all types of Kubernetes assets that I have.
If I go and group them by type,
I will see two types of Kubernetes clusters.
One is Kubernetes service.
This is Azure's managed service.
Azure Kubernetes Service, I've got two clusters in the Cloud.
The other is Kubernetes Azure Arc.
This is "Azure Arc enabled Kubernetes."
These are clusters that are elsewhere and we brought
them to Azure by giving them an Azure ID or an Arc resource ID.
This includes AKS engine clusters, there are BDR clusters.
There's a GKE clusters,
and there is my desktop.
I installed a Docker Kubernetes on my engine,
on my laptop, and I brought it to Azure.
[inaudible] Go ahead.
>> Let's pause right here. Basically, the two Kubernetes
at the bottom are just standard,
AKS, Azure Kubernetes Services.
The ones above, represent
Kubernetes services that are running outside of
the context of Azure
including one that's running on your laptop. Is that right?
>> Yeah, a couple of them are AKS engine cluster so I spun
up VMs on Azure and I deployed Kubernetes on them.
>> I see. It's like a home-grown, you made your own.
You spun up three VMs and did a Kubernetes cluster.
>> Yeah.
>> Fantastic. Now that these are here,
how does one access them?
Look, as a machine-learning guy, I like hardware.
I just don't like messing around
with it. You know what I'm saying?
How does a machine-learning person access
these clusters to do training?
Do they have to know anything about Kubernetes?
>> They don't have to know anything about Kubernetes.
We have built the cluster such
that the IT operators in organizations can set up
these clusters for their data scientists and
data scientists just need to know
their targets, so I can show that quick.
What you see on my screen is the
Azure Studio, the Machine Learning Studio.
This is primarily a place where
one or more data scientists come in.
They keep all their assets here including experiments,
models, datasets, pipelines and whatnot.
They keep everything here.
They do their training jobs here on serving and whatnot.
This is the centralized place for all the assets.
I just showed the idea operator goes in,
creates these clusters, sets up these targets.
They enable the Azure Arc bricks on the cluster.
They enable Azure Machine Learning.
I will just switch over to my "CLI" and I have a cluster.
Let me show you what this cluster
is. Let me just clear up the screen.
This is a three-node cluster that I have,
Azure Arc cluster, and then
what I have deployed on this is going to be important.
To bring this cluster to that view that I had,
you have to install Azure Arc bits.
This installs a bunch of bits and make
sure we can see it in Azure and simply govern it.
Then the second part is installing Azure Machine Learning bits.
This will allow you to use this cluster,
to deploy cleaning jobs onto the specific cluster.
You will see that in the specific name space, Azure ML.
You've got a bunch of operators deployed.
You will see that we have some MPI, PyTorch, TensorFlow operators.
Then we've got Azure ML operator.
We've got Java exporters.
Of course, we're using [inaudible] metrics
report back to Azure so we can show them,
show them to you in a centralized location,
but this is what a cluster looks like once
an IT admin sets it up for the viewer data scientists.
Now, data scientists can go to the Azure Machine Learning Studio.
>> Just pause right there. All of that stuff
happens by someone that's
a Kubernetes-person date because if you're
running these things on your own Kubernetes cluster ostensibly,
probably you have somebody that's literally running that stuff.
But now to actually get that running,
you don't have to do anything.
You just have to cook it up. Can you show that to us?
>> In the Azure Machine Learning Studio,
you have the Compute tab.
When you go to the "Compute" tab,
we have two sets of compute.
One is all of these as Managed Compute.
These are compute that
people use and you don't care about using,
updating the infrastructure and all that,
security patches and all.
The other part is the Attached Compute and
that's where we've integrated Kubernetes.
You will note that excess Kubernetes service.
This is a simple pane where you come in and give it a name say,
my target. Then you choose a Kubernetes cluster.
This Machine Learning Studio is in
the same subscription as my portal.
You will see all the exact same clusters here. I
had two AKS clusters and I had six Arc clusters.
You will see all of them.
You can probably select any of them as a target and say attach.
I already have one attached for you.
It takes a couple of minutes to attach and you're all set.
These are all the targets that are for a data scientist.
You can target any of these for running your training jobs.
Now this step, could also be done by
the ID operator but it's up to you.
Then we have provisions where you can select
an entire cluster and target the default namespace,
or you can provide a specific namespace that will
allow you to deploy to that namespace,
typically and ID [inaudible] operators, split
their large clusters into namespaces and
hand them out to data scientists teams and dev teams.
You can do that as well.
>> That's interesting. Effectively, if someone has
on-premises Kubernetes clusters that already have GPUs,
that already do all the things machine-learning folks want.
But you want to take advantage of Azure Machine Learning for,
let's just say it's dashboards and logging,
you can do that now?
>> Yeah, use the excess capacity anywhere you
want on the fly.
>> That's awesome, man because
some people are afraid to move to the Cloud because they're like,
"Well, we already have a bunch of clusters."
But Azure Machine Learning is actually really,
really good at the exhaust
of machine learning like logging and everything.
Now, you can use actually both.
Where can people go to find out
more unless you have something else to show us?
>> Yeah, I have a demo of showing how
the data scientists should
not have to make a whole lot of changes to run a Notebook.
What typically a data scientist does in
the day to day is they have sample Notebooks.
This is an image classification, traditional example.
I've got this Notebook.
I wanted to call out that there are a bunch of steps here.
You import a bunch of packages, Python packages.
You install the Azure SDK.
You connect to this here what is called
as a workspace on a project.
There's view as a workspace where you keep
all your assets and you're going to put this workspace.
You create a sample experiment.
This is the only change as a data scientist you want to do.
You got to import this package and you give
it one of the compute targets that you set here. That's it.
Then you are just all set to use AML,
just like you were using before.
We want you to be productive.
We don't want you to be wasting time,
learning Kubernetes on anything.
Everything has been set for you.
You can run through the entire script all by yourself.
Nothing changes and I will just submit a trial,
a sample job and show you how it works.
This is the last step of submitting a job.
After you set a compute target,
I will run it and then I will go to the "Experiments" tab.
This is the entire image classification experiment.
I will show you that these are
all the jobs that have run on this cluster.
This is the one that I just submitted. It's been queued.
It will run. Yeah, you click on any experiment you want.
You can familiar view of the properties of the run that you have,
the metrics, the images that have been used.
The child runs or the output logs.
The output logs tell you what is going on when you submit a job.
You can track it. You can see that it's now starting apart.
From a data scientist's perspective, nothing changes.
You can see it's already gone into running stage
and it will complete right now.
Let's go ahead and take a look.
The IT operator can go back to the CLI,
their favorite tool and take a look if everything is working fine,
the cluster has not been destroyed in
any format so they can do great jobs.
They will see that
this job was scheduled for [inaudible] , it's all good.
They can look at the logs.
They can use [inaudible] CTL, command line.
They can look at the logs described and see what the events are.
They are basically allowed
to slice and dice the cluster just the way they want.
There's also a corresponding
[inaudible] that's also been spun up and is running.
It would run to completion and you will have your own.
One of the things that I want to point out is this compute target.
You can very quickly change
this compute target in your Notebook and target any cluster.
It could be any cluster,
on-prem or in the Cloud
as set by you and the target. Yeah, that's it.
>> That's really cool. Basically, all you have to
do is change the compute target and you're done.
>> Yeah, very clear. That's all you need to do and you're all set.
>> Well, this is awesome.
Where can people go to find out more?
>> There is a link to a blog that we wrote.
We are in GitHub public preview so sign-up sheet is there.
You go sign up and we are
slowly onboarding customers to try this out.
>> Awesome. Well, this has been amazing.
We've been learning all about how to use
on-premises Azure Machine Learning clusters using Azure Arc.
Thanks so much for watching and we'll see you next time.
Take care.
[MUSIC]