>> You're not going to want to
miss this episode of the AI Show,
where we look at Azure Machine Learning service,
and all of the awesome developer tools that help
us be productive with AI in the Cloud.
See you then. Hello. Welcome to
this episode of the AI Show,
where we're going to look at
Azure Machine Learning Service,
but specifically the tooling behind it,
I have a special guest today.
Chris how are you doing buddy?
>> I'm doing awesome. Thanks Seth for having me.
>> Why don't you introduce yourself.
>> Absolutely. So, I'm a Program Manager
in Azure AI Team,
focused on developer tooling,
and specifically today we're going to talk
about Visual Studio Code Tools for AI,
and how you can do AI2,
locally and in the Cloud.
>> Fantastic. So, let's start out
with what's Azure Machine Learning Service,
we'll do like a quick overview,
and then we'll look at the tooling that's available.
>> Sure.
>> Let's do it.
>> Absolutely. So, Azure Machine Learning
is a service that runs in
Azure to keep track of all of
your Machine Learning experiments, your data,
your compute targets, your models,
and your deployed services,
that use those models like
an intelligent function that you can call via a REST API,
and get back a result from your training model.
>> Awesome.
>> Now, you can train those models anywhere,
and deploy them in Azure using
Docker containers that run on Kubernetes service.
So, that it's highly load balanced,
and super easy to use, and super performant.
>> So, if you're a Data Scientist,
do you understand all this
Docker container Kubernetes stuff?
Because I have a hard time with
Kubernetes and Docker sometimes.
Do you have to know what that
is in order to make this work?
>> Absolutely not. I'm not actually going to show you how
using Visual Studio Code Tools for AI,
you can generate a new Azure Kubernetes cluster,
which is right-clicking something,
and you can even generate
your Docker image that has your model
and deploy it and get
a REST API again, just right-clicking.
Almost no-code, except for just your Python code that runs
your model which you would run anywhere in
any Jupyter Notebook or in Visual Studio Code.
>> Fantastic. So, let's get into the actual service,
what are we looking at right here?
>> Sure, absolutely. So, this is the Azure Portal,
and in the Azure Portal,
then you can see we've got a bunch of experiments.
You can see "Compute" targets, "Models",
"Images", these are Docker images and "Deployments".
So, those are your actual running services.
On the right you can see I run a couple of
experiments within the context of this Workspace.
As we scroll down you can see that there's
multiple different runs for the same experiment,
because I always try the same thing
over and over to try and get a little bit better,
a little bit faster,
a little bit better quality results.
So, quickly and easily you can
see all the different experiments that I
run and you can see an overview of the "Duration".
But if you drill into any given one of these,
then you can also see
the different quality metrics that
I logged during that experiment.
>> Fantastic. So, let me see if I understand this.
This whole thing I'm seeing is called
Machine Learning Service Workspace.
What is a Workspace? How should I think about it?
>> You should think about it as
just a big container or a box of stuff.
Inside that box of stuff,
then I've got different experiments.
I've got my models that I
can keep track of where they are.
I've got different compute target.
So, if I've got, instead of having
just a bunch of virtual machines scattered around
Azure and all the data scientists on
my team not knowing where to find them for example.
>> I see.
>> Then we can put all these different things
inside a Workspace.
In that Workspace, then we can
find them and share them and get access to them,
just by granting access to the Workspace,
then I get access to all the different compute targets.
So, I can not just find them,
but also actually use them.
>> I see.
>> So, I don't have to go and grant
permissions to all the individual things.
>> That's awesome because generally my trail
of sadness when doing AI is
I'm basically changing code over and over again and I
can't see what did I do three times ago.
So, this allows you to remember
how you went about actually solving the AI problem.
>> That's absolutely right and where you did it.
So, if somebody wanted to reproduce
it using the same set of machines,
then they can absolutely find
those in the Workspace as well.
>> Fantastic. So, this is a super good overview
of Azure Machine Learning Service.
But as a developer,
usually data scientists are going
to be using the things that they like to use.
How do we support them in that?
>> Sure. So, I personally don't
like to use the Azure Portal for actually writing code,
that makes no sense to me as a Developer.
I prefer to use Visual Studio Code.
So, let me show you how with
Visual Studio Code Tools for AI extension,
that's available in Visual Studio Code.
Not only can I see all my different code,
so I've got a Python script
here to train a machine learning model.
But if I click the "Azure Activity Bar",
then we can see I've got this "Azure Machine
Learning View" here as well and
notice this same Azure Machine Learning Workspace
is available within the subscription.
Now, I can filter to
all the different Azure subscriptions that I have so I
don't have to see all of them all at
once in the VS Code View,
because that's just clutters things out.
>> Of course.
>> Then within that Workspace,
then those same experiments
that I was looking at earlier,
I can see them here.
Now, this "Folder of Code" that I have in VS Code,
I need to attach it to the the "Folder to an Experiment".
So I say, this code I'm working on,
on my local machine,
I'm going to attach it to
this experiment in Azure Machine Learning.
>> That's where you get all those cool metrics
that we saw before.
So, if I'm a Data Scientist,
I already have my own,
let's just say MNCs for sake of arguments,
because it's the easiest one for everyone to understand.
I already wrote my own code,
do I need to change anything to make it
work inside of Azure Machine Learning Service?
>> You don't need to change a whole lot.
What you do need to do is, you need to look at,
define the "Run.GetContext",
this is an Azure Machine Learning run.
So every time I run my experiment, it's a new run.
>> I see.
>> So, then I'm going to get a handle to
the run if I'm running
in the context of Azure Machine Learning,
and if so, then I can use the
"Run.Log" method for example,
to log the metrics that I care about.
>> I see. So, if you want the cool visualizations,
you can log, but if you don't log in it will
still run in the Cloud as is.
>> Or it can even run locally
and just log all the metrics to the Cloud.
>> I see.
>> So, as I'm stepping through
debugging my code on my local machine,
then I can still just keep track of
my experiment using Azure Machine Learning.
>> That's pretty cool. So, look as
a Data Scientist usually what I
do is I don't know what I'm doing,
and then I usually pull up a notebook,
try some things out.
Okay, this works, I cut and
paste the code into something like this.
How are you going to support me with Jupyter Notebooks?
>> Sure, absolutely.
So, a lot of Data Scientists
get started early in their process
using Jupyter Notebooks and I'm no exception.
Now, we've added support in
Visual Studio Code for Jupyter Notebooks,
such that you can use
the Azure Machine Learning Python SDK
to keep track of your experiment
from code if you'd like to or use
any other Python libraries
inside Jupyter notebooks as well.
Now, this is not just a viewer of the notebook,
you can actually step through and execute
all the individual commands just like you normally would.
>> So, you find that Jupyter Notebooks
Service and you're running it just
like I would any other time except
it's in the context of Visual Studio Code.
>> That's right. Which is
great for me because personally,
once I step through and get
an idea as to whether
an experiment is really going to work or not,
then I want to take the code out of the Jupyter Notebook
and usually put it into individual Python files,
so I can check them into source control, share them,
version them, debug, refactor.
So, working on both the notebook
and the code all in VS Code,
really helps me to be super productive.
>> Awesome. So, let's take a look at
the Azure tab again to see what's available.
>> Absolutely.
>> So, when I'm looking at this "TeamWorkspace",
everything that everyone on the team has done,
you'll be able to see from their experiments,
from what they're computing et cetera.
>> Yeah, that's right.
So, you can even see this "Compute" node.
So, we talked about what an experiment is.
The "Compute" node, I can easily create
more powerful "Compute" contexts
in Azure if I would like to.
So, let me show you quickly,
we can create a compute.
We can say, create an Azure Kubernetes service
if we wanted to deploy my models to use in production.
Or if I wanted to create a Batch AI cluster for example,
I think Seth, you probably deserve
a GPU cluster of your own.
>> I do, at least on two or three machines.
>> So, I'm just going to go ahead and
create Seth's GPU cluster.
>> Of course.
>> I'm going to give it a name. Now, I've
got a bunch of different sizes of
virtual machines and I honestly can
never keep track of exactly what the name is.
So, I'm just going to go ahead and
search for something with a GPU in here.
I find, oh well,
the V3 sounds pretty good,
it's got a GPU of V100
and it has four of them.
Do you think that would meet your need?
>> I feel like I need some of those.
>> Yeah, okay, great. So, let's go ahead and create that.
Now, I've added this config file,
you see one of the interesting things about
the Batch AI service is that,
it actually can automatically
scale to the certain number of nodes,
these are the number of virtual machines in the cluster.
So, by default you are going to have zero machines,
which means that when no jobs are running,
the cluster actually doesn't exist.
>> So. I'm not going to be charged for this stuff.
>> You're not paying for anything.
>> That's awesome.
>> But then as you submit jobs,
then it'll create the virtual machine,
it'll put your Docker container there,
it'll run it and that's awesome.
You can set some controls on here,
I'll never have more than four virtual machines
at a time for example.
We selected the GPU of four V100s.
So, if we had four virtual machines with four V100s each,
you actually would have up to
16 GPUs in this cluster or you could have,
if I really liked you I could get you a whole lot.
>> Wow.
>> I don't like you that much.
>> That's a lot of digits we're going to be
recognizing in the Cloud, I'm pretty excited.
>> Absolutely. So, we can go ahead and click
"Finish" and create the machine.
The cluster now I already have one
here for the purpose of the show today.
Now, within the context of a particular compute context,
you need to have what we call a run configuration file.
The run configuration file enables repeating
an experiment over and over
if you'd like in a consistent way.
So, that says what script do
you want to run as your primary script.
What parameters do you want to
run and that sort of thing.
So, you can just simply right-click you
can create your "Run Configuration" file,
and you can again see that down here
we have multiple "Run Configuration" files.
>> Okay. So, let me see if I understand, this went fast.
So, when you're looking at your local folder
that now you've attached to an experiment.
>> Yeah.
>> There are "Run Configurations" that allow you to
submit jobs to the GPU cluster.
>> That's right.
>> I'm seeing two other ones there; Docker and local.
Are those just "Run Configurations"
that exists by default?
>> That's right. So as soon as I attach
my folder to the experiment,
then VS Code will automatically
create these two "Run Configurations"
for you and set
up all the default settings automatically.
>> All right.
So, let's see if I can summarize what we've learned.
So, the first thing is we looked
at the Azure Portal and what
an Azure Machine Learning Service Workspace was.
It's a configuration where you can have
a bunch of people
collaborating and have a bunch of assets,
it's your AI toolbox for machine learning.
>> That's right.
>> The second thing we looked at is
Visual Studio Code Tools for AI that lets
you run notebooks within
the context that you know and you love.
You can run Azure Notebook or not,
I keep saying Azure Notebooks because we
have an Azure version of Jupyter Notebooks.
>> They're so easy to use in Azure Notebooks,
they've become synonymous for you.
>> So, Jupyter Notebooks, you can run it
in there and then test your code.
Then you can move it into
regular Python files that you tend to run and you can
submit them into
the TeamWorkspace experimentation service by targeting
different compute clusters or
where the Run Configs will
tell you how you want to submit them.
>> Or you can even just run them on
your local machine and log the metrics to Azure as well.
>> Awesome.
>> So, you can experiment quickly,
step through and debug on your local machine,
make super productive in Visual Studio Code.
>> So, it was awesome, I feel like I want to
still go through an actual example of it.
But I feel like this is a super good overview of what
AI Developers can do on Azure and Visual Studio Code.
>> Yeah, absolutely.
>> Well, thanks so much for spending some time with us.
Thanks so much for watching,
and we'll see you next time, take care.