(laid-back electronic music)
- Hello, and welcome to this new epis
of the machine learning essentials se
where we talk about some fundamental concepts
of machine learning in the cloud.
I'm Nishant Thacker,
and in this video we'll talk about
training ML models at scale.
Now, when it comes to machine learnin
training is a job that takes an enormous amount
of time and resources.
As this is the phase where the machin
is actually learning from large quantities of data.
Now there are two concepts to underst
and optimize for, here:
one, it's managed versus bring your own compute,
and two, the right compute configurat
keeping in mind the intent of the use
Let's look at each of these in a little more detail.
Machine learning has evolved over the years
with a lot of contributions
from the open-source communities.
As a result, machine learning framewo
and libraries now support multiple compute frameworks,
including Dask, Apache Spark, Kuberne
and many more.
When it comes to Azure,
the managed compute offerings are based on Azure Batch.
This means there are dedicated VMs assigned as compute,
with network isolation
and security protocols that cater to the most
stringent of regulatory standards.
We'll talk more about managed compute
within Azure machine learning in a mi
The other compute option is bring your own compute.
Why is this needed?
Well, great question.
Some users might prefer to run their jobs on Apache Spark
or Kubernetes clusters.
This could be for a variety of reason
Having ideal capacity in existing clusters,
preferring specific compute engines over others,
standardizing across multi-cloud or hybrid environments,
and others.
To ensure that such requirements can be met,
Azure machine learning allows users
to attach compute from other Azure services,
like Azure Databricks, Synapse Analyt
Azure Kubernetes Service,
or even Azure HDInsight.
This allows users to leverage their choice of
compute frameworks to train models
in their choice of environments.
Here's a pro tip:
only use these attached compute optio
if you understand these engines really well.
Azure machine learning can optimize a job to
run on managed compute,
but it relies on you to optimize the
bring your own compute options.
Now let's talk a little more about th
managed compute options.
There are two offerings here:
one, compute instance,
which is a single-instance interactive work station.
This is best suited for getting start
or dev/test-type workloads.
Compute clusters, which is the second option,
is a multi-node distributed cluster
for large training jobs.
This is best suited for production-level training
of complex models on very large data
Here, we're just talking about training compute.
There are other managed compute offerings for inferencing,
which we'll discuss in another episod
of the ML essentials series.
Coming back to the training compute,
in addition to provisioning these compute environments,
with commonly used tools and librarie
Azure also enables admin experiences
for enterprise readiness,
so that these can be managed centrall
Some of these admin experiences are:
scheduling, to automate the provision
and guaranteeing its availability,
creating an on-behalf-of capabilities
which allow admins to provision on behalf of users,
to keep deployments in check.
Quota management, to ensure compute environments
do not hog up all the resources,
monitoring, to keep track of jobs
on these compute environments,
utilization tracking, to ensure that compute resources
provisioned are not sitting idle,
and of course, security, supporting secure RBAC
to sophisticated network isolation.
Both compute instances and compute clusters
come with a lot of the above admin experiences.
Let's dive a little deeper into each of them.
First, compute instance.
A compute instance is a fully-managed
cloud-based work station,
optimized for your machine learning development environment.
It provides the following benefits:
one, these enable productivity.
It comes pre-installed with Jupyter, JupterLab, Rstudio
and it also integrates seamlessly wit
Azure machine learning notebooks
and Visual Studio code.
Second, these are pre-configured for machine learning,
so you can save time
and set up tasks with pre-configured
and up-to-date ML packages, deep learning frameworks,
GPU drivers, and many more such aspec
Third, of course these are secure.
Compute instances provide robust management policies
and secure networking configurations,
such as, you can auto-provision these from ARM templates
and Azure machine learning SDK,
you can bring them under Azure role-based access control,
you can enable virtual networks for t
you also have SSH policy to enable or disable SSH access,
and these come with TLS 1.2 enabled.
Another benefit of compute instances
is that it is fully customizable.
Broad support for Azure VM types, including GPUs
and persisted low-level customization
such as installing packages and drive
make advanced scenarios a breeze.
Here's a pro tip: in general, a compute instance
is specific to a single user for interactive workloads,
but it can allow batch jobs from other users too,
if the owner permits.
Next, lets talk about compute cluster
Compute cluster is a managed-compute infrastructure
that allows you to easily create a si
or a multi-node compute environment.
A compute cluster can be shared with other users
in your workspace.
Compute clusters can run jobs in
a virtual network environment,
without requiring enterprises to open up SSH ports.
The job executes in a containerized environment,
and packages your model dependencies
in a Docker container.
The main purpose of compute clusters
is their ability to scale out.
The compute cluster scales automatica
when a job is submitted,
depending on the heuristics of the training job;
thereby, distributing tasks to individual nodes
to be executed in parallel.
Now this applies to both data paralle
where data is divided into partitions
and the number of partitions is roughly equal to,
or potentially a multiple of,
the total number of available nodes
in the compute cluster,
and model parallelism,
which we also refer to as network parallelism sometimes,
where the model is segmented into different parts
that can run concurrently in different nodes,
and each one will run on the same dat
You can use a compute cluster to distribute training,
or even a batch inference process
across a cluster of CPUs or GPU compute nodes in the cloud.
Here is a pro tip:
compute clusters are designed to cater to long-running jobs,
and as such, can recover from node failures
or preempted nodes,
in the case of low-priority VMs.
Thus, always use compute clusters for
production-level training jobs.
Well, we've discussed a lot of concepts in this video,
let's quickly recap.
We first looked at two forms of compu
managed compute, which comes with a lot of added benefits
like security, administration, quota management, et cetera,
and then bring your own compute.
This provides the flexibility of compute engines,
alongside the ability to utilize spare capacity.
Within managed computes, we discussed two types:
compute instance, for getting started quickly
or dev/test-type workloads,
and compute clusters,
for large, distributed production-scale training jobs.
All these compute environments can su
enterprise-grade security, like RBAC and VNets.
To try these,
please spin up a free Azure machine learning workspace
at aka.ms/aml-trial.
Also, download a 30-day learning jour
from the data scientist resources pag
aka.ms/data-scientists.
That's all for now,
see you in another episode of
the machine learning essentials.
(laid-back electronic music)