>> Hey everyone. I'm Cassie Breviu filling in for
Seth today on the AI Show
and you are not going to want to miss this episode.
We are going to be talking about Azure Containers
for training with PyTorch and Azure DSVM. Let's take a look.
>> [MUSIC]
>> Joining me today is Parinita.
Why don't you tell us a little bit about yourself?
>> I guess it's nice to be here.
I'm Parinita Rahi.
I am a PM in the Azure platform team.
I'm so glad to be here to talk about the new Container
for PyTorch that's now available on Azure.
>> Cool. Can you tell me a little bit
about what this Container is?
>> Sure. Data scientists,
ML engineers, etc,
generally when they have a training workflow for large models,
one of the challenges that they face is,
what are the different dependencies,
binaries, etc, I need to get installed to have ease of setup?
This container helps developers,
scientists setup, develop,
and accelerate training workloads meant
for large workloads like GPUs for the PyTorch models.
It's now available on Azure as a Container,
as a curated environment as well as the DSVM.
It's available for like ease of download and you can
get started very easily for any of your training work needs.
>> That's really cool. Basically,
I'm able to create
a DSVM or Data Science Virtual Machine or use the Container.
Is the Container in part of Azure Machine Learning?
Can I use it there as well?
>> Yeah. It's something that you can go to
the Azure ML Studio and in the curated environment section.
You can easily go and look it up and get started.
It's a pretty simple workflow that you can now use with Azure ML.
>> Cool. You mentioned some of the people that are
using this are engineers,
so it's like the ML engineers, data scientists.
Who are the main roles
that are going to be leveraging this Container?
>> As I mentioned earlier,
it's primarily for data scientists.
It's also for large enterprises who
have training workflows on PyTorch.
Imagine somebody who has like
an NLP modeling workflow for language models,
text and speech models who generally use PyTorch
and have dependencies to get that setup.
Those are the ideal candidates who
will benefit from this Container.
>> What are some of the benefits of using
this pre-configured Container versus
something that they would set up themselves?
>> When you get started,
especially with the different versions you have of let's say
PyTorch or Python or CUDA,
and other essentials that needs
to for you to set up your training workflow.
Figuring out which version is compatible with what,
what are the dependencies,
all of that can take,
a lot of back and forth in figuring out bugs, etc.
This Container, what it does is it
has your essentials required for training.
It has the version of
Ubuntu setup with CUDA or ROCK depending on which driver you
use with Python versions and
the latest PyTorch versions, pre-validated and pre-installed.
It's again tested and
with a suit of Microsoft internal 1P workloads.
It has been pre-verified.
It basically takes away a lot of
the pain that scientists, engineers, etc,
have to go through while starting
off with their new training scenario.
You just go get it from Azure ML and get started.
That's pretty much all you need.
In addition, what it also provides is
a bunch of these training optimization libraries.
You might be familiar with ORT which is
used for accelerating training of
large transformer models in PyTorch.
You can leverage ORT because it's
pre-packaged within this Container.
You can also leverage deep speed
which is a deep-learning optimization library,
which makes training easy and effective.
There is also like MSCCL,
which is a collective communication library
that Microsoft open-source provides
that's bundled within this Container.
These are just a few that I'm naming.
There is a whole lot that you can use.
It's packaged with us looking at
these 1P training scenarios and what does it
take for a normal workload to be effective,
to get started easily to have optimization
especially since you're running these on GPUs you
would look at training optimization.
All of that is bundled in or for ease
of set open and acceleration.
>> Awesome. I'm going to be able to now train my models faster and
set up my training faster and then maybe
leverage this on Azure Resources.
Then there's that cost savings that comes along
with it as well because it takes time to set up these Containers.
There's less time for that,
but then also you're going to train your models faster,
so you're going to save money on
your Cloud computing bill as well, right?
>> That's true. As I mentioned these are optimized
faster training so your time to
compute or time to train goes down significantly.
We have seen 45-85 percent improvements
with some of our hugging face models.
Like you said, it helps you reduce your compute costs as well.
>> It has all of these pre-built packages
configured together ready to use.
Can I customize it to add
some additional packages that might be
specific to my model training?
>> Yes. I think Jessica will give a deep dive in a later session,
but there are two ways in which you
can use these curated environments.
You can use them as is like if you just want to start with
something and you don't have other dependencies that you need,
which is like quality not captured within the workflow.
Then you can just get started with
the curated environment from the Azure ML Studio.
However, you can also use this as a base to
get started with any additional dependencies you may have.
You can get this environment, download it,
and then install any additional dependencies.
It definitely minimizes your steps however here.
>> Is there anything else you want to share about this container?
>> Generally if you're looking for faster training,
if you have workloads with larger models,
you want to get started without having
to spend a lot of time for setting
up because this is
an Azure's pre-installed package and it
can also help you scale
because you can share within the enterprise,
you can easily share it with
a lot of your engineers to get started
without them having to worry about these dependencies.
Then if you have a workflow on Azure,
I think these are definitely some of the criterias wherein
you need to consider this Container.
In addition what it also provides is,
because this is on the latest PyTorch version,
our teams will continue to provide support and export
troubleshooting and guidance for the latest PyTorch build.
If you're thinking about having any questions
addressed as you start off your training journey or Azure support,
all the goodness that comes with it
is provided through this container.
The other thing is, as I already mentioned,
all of these ML packages that are
open source are tested and
against a suite of ML workflows within Microsoft.
You are basically leveraging all of
the goodness and saving time and saving costs,
so all good things here.
>> Thank you so much for joining us today, Parinita.
Next up we're going to talk to Jessica,
and Jessica is going to show us
some demos of this Container in action.
Now we are joined by Jessica and she's going to tell
us a little bit about how Azure Container for PyTorch works.
Jessica, why don't you tell us a little bit about yourself?
>> Hi and thanks for having me.
My name is Jessica Cioffi.
I am a Product Owner on the Azure AI Platform,
working with the training group.
For this effort, I've been really focusing on connecting
the Azure Container for PyTorch and our curate environments
that we currently offer on Azure AI.
>> Awesome. I'm certainly excited to
see this in action since we've been talking about it.
Let's get your screen shared.
Awesome. Why don't you show us how it works?
>> Sure. When you enter
your workspace in the Azure Machine Learning Studio,
you can go down to
this environments option in the menu on the left side,
and clicking that will bring you to
a view of the different environments that we have.
At the top here we have
both curated environments and customer environments.
Curated environments are environments that we have put together.
They contain a lot of dependencies and accesses that you guys
would want to have when you're
running your deep learning training,
and we put that all together for you so
you don't have to worry about going and trying to find, oh,
does this version match with this version
of the software that I want to run one around?
We have all that for you
and you are able to use that pretty easily.
Initially, when you do come to this view,
there is a very long list of
all the different options that we have here.
We have different versions of different OSs that are there,
we have different CUDA versions that you could use as well.
Searching for Azure Container for PyTorch,
or if you do ACPT for short,
you will see our two newest options here.
The first one here is for CUDA version 11.3,
and the second one would be for CUDA version 11.5.
Currently, we have those two offerings out
for the ACPT curated environments.
>> Awesome. These were just
added and so they're brand new for people to try out.
Then there was also like you show there's a lot of
other environments there that I can use,
so I can use these within Azure Machine Learning.
Can you show us a little bit about how I might use these?
>> Yeah, absolutely. When you want to run a job,
you can opt to choose an environment to run that job with.
If we go here to this jobs option,
it will bring up a menu where you can run jobs.
If you click New here and then go to jobs down at the bottom,
you'll be brought up with a Create training job menu
where you can go through and select your options.
I will click to do
a Compute cluster and click my cluster that I have here.
Then moving over for
the environment type as the second option you can choose
between curated environments and
the options will populate below like we just discussed.
Or you can choose a customer environment which would be
something that you'd customize to your specific needs.
For this one will go with are curated environment.
If you expand the option here you
can see all of the different information.
Since this is our CUDA 11.3 GPU version,
you can see that it is Version 1 because it is brand new.
I will select this one as our environment.
Clicking Next, you'll be able to configure
your job settings so you can change the name of your job,
your experiment name, and then upload your code accordingly.
I will upload a folder that I have
here and then I will click the folder,
upload my code, click Upload again
and it'll upload the code that I have in that folder.
Then to start the job,
you have to enter the command.
I will just type in python train.py to make sure that file runs.
There are additional options at the bottom for
any inputs and outputs you want to add training applications,
environment variables, all these things
that you can do to customize your job.
I'll click PyTorch here and then click Next.
Then this last page here in the menu is just where you can
review the job specification to
make sure that you set everything the way you want.
Your compute is set up and what you were expecting,
the different settings that you set so the name,
the experiment, things like that,
the command that you want to run to run
the training and the environment you can see
here is our new Azure Container for PyTorch the CUDA 11.3 version.
Going with that, I will click Create and just like that,
your job will be started.
>> Cool. Is there
any updates that I need to make to my scripts that
I upload there or can I take just the script to that I was using,
maybe locally for testing and put it up on this container?
>> You should be able to.
The only thing that would be important to note is
that if there are specific libraries
that you want to make sure
that align with the code that you're running.
You could actually use this image that we have as a base image and
customize it to make sure that the libraries
are included for your specific project.
>> Cool, so I can take this and use it
as a base environment and then
install the additional libraries that I
need to customize this curated environment?
>> Yes, exactly.
>> Cool. Are we going to see
that, are going to tell us how to do that?
>> Yeah, let's do it.
>> Awesome.
>> I will bring us back to the Environments menu.
Like I mentioned earlier,
we have the curated environments option at the top,
which have worked all of our preset options.
Then I can open up one here and just show
a little bit more of the details in the Dockerfile here.
You can see the properties lined
out for you here on the left and on the right you can
see the base image at the top and
then some of the dependencies that we include.
If these are all that you need
to run for your project and nothing is missing,
you can use it just as is.
But if you would like to customize it for yourself,
all you would need to do is copy this here.
Then you would be able to go
to Custom environments and click Create.
For here, we pick a name I'll just name it my-environment.
You can add a description if you'd like,
or from the Select environment type,
you can either start from
an existing environment if there is
something that you already have running on our system,
or you can create a new Docker context and use the image.
From the customized menu
you can see some pre-populated information here.
All you would need to do is remove the image that we have
here and replace it with the ECPAT image.
As you can see, it's the CUDA 11.5 version PyTorch.
Then underneath here you
see we have run pip install azureml-mlflow.
You can change this ML flow to be whatever you need.
If you need transformers, you can pip install transformers,
and here you can adjust all of the dependencies that you
have on top of the base image that were already providing.
You're able to customize this however,
you need to best fit your training experiments and experience.
>> That's awesome. You can leverage
all of this work that's been done
to create this container that's pre-ready.
But then when you just need to add that little tweet
to what you need, it's really easy.
You just create a custom environment and import
that container and then add your pip installs.
>> Yeah, and you use it the same
exact way when you're setting up a job.
Instead of just selecting curated environment,
you switch to the custom environment and
pick the one that you named. It's that simple.
>> Super cool. What are some of the reasons I might use
the curated environment instead of an environment
that I was using previously or the new environment?
>> This new environment comes with a lot of performance speedups.
By utilizing the best of Microsoft technology
for deep learning and distributed training,
such as the speed on X runtime,
even fair scale, the combination of
those options and enabling
those when you're using your environment,
you can actually see performance feeds from 48 to even double,
sometimes from your previous runs.
I actually do have an example
of this performance increase on some previous runs.
Here is a GP2 run of a NLP training operation.
You could see it was completed in
four hours and 37 minutes using the baseline.
If we look at the deep speed
by using our curated environment and you can
see that the total onetime decreased to an hour and 42 minutes.
>> That is amazing.
>> It went from over four hours
to a little over an hour and a half.
That's about 2.7 times speed increase for
the same training job utilizing our new environment.
>> That's awesome, so that's time-saving
if you need to retrain things
and money saving on the compute resources.
>> Absolutely.
>> Which compute was used for this training?
>> Let's double-check. I should be able
to go to Compute, go to the clusters.
We can see this is NC24S_V3.
I should be able to click here and
see some additional information.
Yes, so this is NVIDIA V100.
>> Wow. That some impressive performance improvements just
by updating to the new curated environment.
Is there anything else you want to tell
us about the new environment?
>> Yes, so the environment is currently in public preview.
You can learn more at
our Microsoft Azure Machine Learning documentation.
>> I'll share that link below here,
go ahead and check out those documentations to
learn more and Ignite is coming up as well is
there anything exciting happening
with these environments at Ignite?
>> Yes, so we were actually talking a
little bit more about some of the updates and
the improvements that the environments have
seen at Ignite during our deployment.
Definitely check out at Ignite to learn some more.
>> Great. Well, I will be setting my calendar to be at Ignite,
I'm speaking at Ignite so I will also know that it's happening,
but everyone else should set their calendar
and go check out the session so
they can learn more about these curated environments.
Thanks so much for hanging out with us today, Jessica.
>> Thanks so much for having me.