Hey everyone, I'm Sid, I'm a program manager on the PyCon
AI & ML tools team and today I'm going to be
showing you how you can supercharge your Azure ML Development through
Visual
Studio code. So let's get started. Just a little
bit of an overview.
We are going to first start by creating
some Azure Machine Learning compute resources were then going to
seamlessly connect to these resources from within VS Code,
and then finally we're going to run and interactively debug
a local training job directly on the remote compute that
were connected to.
So let's start by first going to VS Code
and talking a little bit about the project that we're
working on.
So I'm in VS Code now you'll see I have
this project open.
It's called pycon-mnist and just a little bit
of background as to what we're doing here.
We're going to be training a neural network with Tensorflow
to classify handwritten digits,
and we're going to be training this neural network using
the famous MNIST data set that many of you may
be familiar with if you've done or dabbled in machine
learning before. And so here I have a training script.
I have some utility script with some helper methods,
as well as an environment YML file that I can
do that.
To define the environment that I'm going to be working
in. Now
one of the options that I have is I can
actually do this training directly on my local machine,
right? So I can run this Python script in the
conda environment that I have here.
However, right now I'm on my laptop and my laptop
unfortunately is a CPU based machine with very
little memory and I think it's four cores and
So what I expect to happen just given the memory
that I have and given the memory that
I have and the cores that I have available
is that I expect this to actually take quite a
bit of time,
so instead what I want to do is I want
to create a compute resource through Azure Machine Learning,
while again just being within VS Code here and I'm
going to do so through the Azure Machine Learning extension
and once I have that compute resource I can use
that instead to run my training job instead of my
laptop as I mentioned.
What I can do is I can navigate to the
Azure tab on the left hand side.
You'll see that when I open this tab,
I'm presented with the Azure Machine Learning pane.
Here I have a list of subscriptions that are tied
to my Azure account and then for these subscriptions I
have a list of workspaces.
So just really quickly for all of you that may
be unfamiliar with Azure ML,
Workspace is a top level resource in AML that
is used to manage and organize all of
the artifacts and resources that I create through the service.
So with this expanded you'll see that I
have a list of artifacts here,
such as datasets, experiments, computing resources, environments, again,
all related to the Azure Machine Learning
service and grouped under this workspace.
So now I want to get started with creating the
compute.
I can list compute instances which you can think of
as managed VMs for me to make use of.
Here I have a list of compute instances already.
You see, some are stopped, some are running, one is in a restart
failed state.
That's probably something I should take a look at after
I have finished with this demo.
But what I can do to create a compute is
I can right click on this
create compute instance node. I can choose the
create compute instance option.
I can then provide it with the name,
so I'm going to call it PyCon 2021
demo.
And now here I can search for the compute or
the VM size that I'm looking for based off of
different things that I know.
So if I knew the name
of the VM size,
I can just search for it directly.
You'll see that I know that I might want
an NC6 so I can find it that way.
Alternatively, if I know that I want a GPU,
I can filter for all of the computes that have
that have a GPU available.
And then finally,
if, for example, I want something that's 16 cores,
right? So four times what I have on my laptop,
I can just search for cores: 16 and then you'll
see here that I'm presented with a list of options,
right? So I'm going to choose the Standard_D 14 here,
and I'm going to say that I don't want to
enable SSH access. Now
remember that I mentioned earlier that we're going to be
connecting to this compute,
so keep that in mind here.
I'm not enabling SSH access,
I'm not doing anything in terms of opening ports,
changing any network configurations. But eventually I should still be
able to connect to this compute instead,
so I'm going to say no,
and now this compute is being created.
It is going to take a little bit of time
to create this compute
luckily I have one from before that I'm going to
use instead,
so I'll show that to you right now.
So I have this pycon2021 compute and what I
can do to connect to this compute is I can
just right click on this node,
choose the connect to compute instance option,
and then you'll see here that a remote window is
now being opened where this remote window is
connecting me from my local machine to the remote compute
instance.
Now I talked a little bit about SSH access or
the lack thereof here.
Well, what's happening behind the scenes now is there are
Websockets.
There's a Websocket connection that's being established between my local
machine in the compute instance and that Websocket connection and
all of the information that's going to that through
that Websocket connection is also going through the Azure
ML control plane.
So the way in which kind of authorization and access
credentials work is that it goes through the Azure ML
control plane.
I'm authenticated using my AAD credentials.
What's awesome about this when you think about it in
comparison
to SSH is I can audit
this,
right? I can't share my AAD credentials with someone else
that's tied to my Azure account and so I don't
have to worry about things like leaked passwords or leaked
certificates and so I'm establishing a connection to this
compute instance in a highly compliant and highly secure way,
and the beauty of this now is that once I'm
connected here.
It's honestly just the same as it would be
in regular VS Code,
right? So I can do,
a number of things here,
such as interact with the terminal as you see here,
interact with the file system,
managed, get repose, use the debugger,
use different kinds of extensions,
and so what I'm going to start off with is
I'm just going to go to my personal directory,
right? So you'll see, that's the users siunnith
Directory,
and then I'm going to Git clone the repo that
I plan to work in,
right? So I'm just running this Git clone command.
It's going to clone the repo in this directory.
That was really fast. Now if
I drop this down in the File Explorer,
you'll see that that's there immediately,
so I can just copy the path to this directory
and then I can open this folder
now in VS Code and so you'll see that this
new folder is opening and we're establishing that remote connection
again.
But this is all happening really quickly because we're making
use of existing servers that we've installed on the compute
instance and the existing Websocket connection that
we have.
Alright, so now that I'm on the compute instance again
and scope to this folder to my project folder,
what I can do is I can open up my
training script and now I can run this training script
as a job tracked through Azure ML.
But directly on this compute instance.
So the way that I would do that is I
would right click in this file
here I can choose this Azure ML
run as experiment in Azure command
and then
now I'm prompted to decide whether I want to run
this remotely or locally,
I'm going to say that I'm going to run this
locally.
As I mentioned, I'm going to say that yes,
I do want to debug my experiment,
and now I'm asked for an experiment.
So what is an experiment you can think of it
as a way as just a placeholder or
a way in which I can group a number of
different training runs. So I'm going to use this default
experiment that I have.
I now have to choose an environment,
so just really quickly on environments an environment in AML
can be thought of as a
way to package different conda or pip dependencies that
you have and these environments when ran remotely or sometimes
locally can be either materialized as a conda environment or
a docker container. So in my case I'm going to
materialize this environment as a container in order to do
this local run and be able to debug and I
instead of using one of these curated environments that you
see here.
So those are all the ones that are prefixed with
Azure ML.
I'm just going to choose one that I've created from
before,
right? So it's called PyCon Tensorflow debug.
It contains all the right packages that I need.
And in this case for this demo,
I'm not going to use a data set is outside
of the scope of the demo,
but if you are interested in datasets and wondering how
you can use them,
there's a ton of documentation that you can take
a look at there and so now here,
finally, I just have this little json that gives me
a bit of an overview as to what I'm going
to be running.
So again, its tied to my default workspace,
I'm going to run it locally.
It's going to contain these packages,
so Tensorflow, Azure ML, debugpy,
and then I can just hit submit,
experiment and voilà. Now you'll see that the experiment
is being submitted and that my debug session is being
set up,
so my debug session is ready.
I can now set breakpoints. So what I'm going to
do is I'm going to set a breakpoint here just
to make sure I'm getting the run context correctly.
Then it set a breakpoint at the MNIST data
set download step,
and then finally I'm going to set a breakpoint at
my epoch,
right? So I'm going to these are
my training cycles and then I'll just bring that up
again and hit start debugger and so now
when I hit start debugger,
the Azure Machine Learning extension,
what it's going to do is it's going to run
my experiment as you see here.
This is running in a Docker container on my compute
instance that I'm connected to seamlessly
seamlessly from within VS Code,
and I immediately hit that breakpoint that I set to
see the run context.
So I want to make sure that I'm getting the
run context correctly.
I am, that's awesome, and then what I'm also going
to do is I'm going to output something to the
debug console,
so for example, I'm going to say print.
Nope, that's not the debug console.
This is the debug console,
so I'm going to say print "Hello PyCon" and I'm
going to put this to the debug console.
You can imagine that I want to print that because
I'm curious as to... or because I'm looking to get
more information about the run
that I'm doing and then I'm
going to hit play, and what's happening here is that it
does take a little bit of time for the data
to get prepared, but what I want to show you
is,
as this job is running,
I can actually track it through the Azure Machine Learning
extension,
so you'll see I have that experiments node.
If I just search for default,
it's going to find the default
experiment that I have here and then you'll
see that this is the run.
This is the most recent run, run 71, that's happening
right now.
I can actually take a look at logs associated with
this run,
so if I go to the logs tab,
find Azure ML logs, double click on this.
This is going to stream this 70 driver log directly
in my console,
which is really cool too,
so not only can I interactively debug,
but I can also run this experiment and then stream
logs directly in the console here.
And yeah, now I just wait for my
for all of the data preparation steps to take place
and then eventually I should hit the epoch breakpoint that
I have set here. So you'll see that I hit
the...
I hit this breakpoint that I set for the epochs
so I can just step over and then now
I can
debug the training cycle that's happening and So
what I'm going to do is I'm going to unset
these breakpoints and I'm going to hit play before I do
that. One thing I forgot to mention before,
remember how I typed "hello pycon" in the debug console
that's actually now part of my driver log,
so that's really cool, because what I can do is
I can debug variables
I can write what I want to the debug console and
then I can have that
get saved as part of the artifacts associated with my
run here,
so I'm going to hit play and I'm just going
to let this run and so you'll see
this is just going to run through
all of the different cycles.
One thing I also want to mention is
I can also set a breakpoint kind of as this
is running right because this is just looping as you
see here,
so I'm going to put one on the batch size.
For example, if I see something going awry with my
validation accuracy of my training accuracy.
I could set a breakpoint there and now I can
investigate what was happening
and then I can step through this code here and
then when I feel like I've diagnosed the issue that
I'm seeing,
I can unset that break point and then just continue
and so this is going to run and then once
this completes I can actually... I can get outputs for
this run, so I'm going to show that for a
completed one you'll see here that I have outputs and
then this is just the model that I can then
use to serve as part of our Azure ML on
for example in inferencing cluster.
So just a quick recap of
what we've done here.
I created a compute through VS Code.
I seamlessly connected to this remote compute,
which is much more powerful than my laptop using the
Azure ML extension as well.
And then finally on this compute I ran a local
training job that I interactively debugged.
And so in terms of next steps,
I would recommend checking out our newly published blog
post where we talk a little bit about this.
The compute instance and compute instances and how you can
seamlessly connect to them,
as well as some other entry points.
For example, coming from the Azure ML Studio for all
of you that are familiar with that,
then I would recommend installing the AML extension and giving
it a try,
especially if you're an Azure
user, someone who is starting to dabble kind of in
the machine learning world want to get your feet wet
a little bit.
Try this extension out and then please provide feedback to
the team.
We'd love to kind of know how you're using the
extension,
where we can
improve this extension and then yeah.
And then we'd love to reach out to you directly
as well if possible,
so that's it from my end.
Thank you so much. I hope to see you at
the next PyCon.
Cheers.