(upbeat music)
- Hello and welcome to another episode
of the "Machine Learning Essential" series.
I'm Nishant Thacker.
And in this video,
we'll talk about hybrid and multi-cloud machine learning.
Now, there are a lot of definitions
for hybrid or multi-cloud machine learning.
What we'll cover is how you can enable
machine learning solutions
to leverage your on-premises
and third-party cloud infrastructure.
But first, let's start with an understanding
of why we need such a capability.
Well, there could be a variety of reasons
depending on what your domain is
and use cases are.
You may be on your journey to the cloud
and not all your data's there yet
so you wanna run machine learning on-premises.
Or you're creating a solution
for a highly regulated industry
and need data to remain on-premises
or on a sovereign or private cloud.
You might already have data on other third-party clouds
and now you want to leverage
the machine learning capabilities from Azure
to enrich your solutions there.
Alternately, if you wanna utilize
your unused compute infrastructure,
either on-premises or on third-party clouds
and vice versa
to implement "cloudburst" scenarios,
where you start small in your local environment
and now you wanna expand to the cloud
because you're running out of compute infrastructure
in your local environments.
Whatever be the reason,
hybrid and multi-cloud deployments
are becoming more and more the norm
rather than an exception.
And enterprises are looking for this capability
to hedge their bets for the future.
Let's take a deeper look at how Azure enables
hybrid and multi-cloud machine learning deployments.
There are two key concepts to understand.
One, what are the considerations
for hybrid and multi-cloud deployments?
And two, how does such a deployment work?
Let's look at each of these one by one.
First, let's understand the considerations.
When we think about deployments
that span hybrid environments,
we have to make some important choices
about our solution architecture,
especially around compute options
and deployment apologies.
The reason being,
anything you choose needs to be available
across your target environments.
One such compute option is Kubernetes.
It is available to be deployed on-premises
and as a service on all public clouds.
It also brings a well-understood
and widely adopted deployment topology with containers.
Machine learning has adopted the container typology
to package the models and their dependencies
so that these can be run anywhere.
Here's a pro tip.
Kubernetes cluster support both GPU
and CPU compute options,
enabling machine learning work loads
to utilize the right compute options as needed.
This makes Kubernetes a very good choice
for hybrid and multi-cloud deployments.
The other consideration is management
both at the infrastructure deployment level
and the machine learning lifecycle management level.
At the intro level,
you need to ensure that it can seamlessly interact
with the deployment targets
whether it is on-premises infrastructure
or public cloud infrastructure
like Azure, AWS or GCP.
This allows it to maintain
the health of the deployments
and assist with automation,
tracking,
security, et cetera,
from a centralized management console.
With Azure, this is made possible with Azure Arc.
Azure Arc offers simplified management
and faster deployment.
And it helps standardize visibility,
operations and compliance
across a vide range of resources and locations
by extending the Azure control plane.
Since it can ship and manage solutions from Azure
to any Kubernetes distribution in any location,
it enables you to securely build and train
machine learning models anywhere.
The other aspect that needs to be managed
is the machine learning life cycle.
Again, you need a tool or a service
that can run anywhere
and provide a management plan
for the end to end life cycle.
It also needs to integrate seamlessly
with Kubernetes as an engine.
With Azure, this is accomplished
via Azure Machine Learning.
Azure Machine Learning provides an SDK
that can run anywhere,
including on-premises
and other public clouds
to provide that single life cycle management service
across all the deployment targets.
Here's a pro tip.
Azure Machine Learning also supports ML flow
so that even existing deployments
can be configured to join Azure Machine Learning
and enterprises can centralize
the machine learning workflow entirely.
Understandably, customers can now run
and manage their machine learning training
on Kubernetes clusters.
In Azure, GCP,
AWS, Edge devices,
or even on-premises
through Azure Arc-enabled Kubernetes.
This allows customers to increase operational efficiency.
Now, you must be wondering,
how does this work?
Great question.
Let's tackle it by looking at
the two personas involved here.
First, how administrators utilize
the common Azure Arc plane
and provide resources for data scientists.
And second, how data scientists utilize these resources
for distributed model training.
Let's understand the administrator's job first.
With a few clicks,
administrators can enable
the Azure Machine Learning agent
to run on any open-source Kubernetes cluster
that Azure Arc supports.
This along with other key design patterns
ensures a seamless setup of the agent
on any open source Kubernetes cluster
such as AKS, RedHat OpenShift,
managed Kubernetes service
from other cloud providers, et cetera.
There are multiple benefits of this design
including using core Kubernetes concepts
to set up or configure a cluster
running cloud native tools
such as GitOps, et cetera.
Once the agent is successfully deployed,
administrators can either grant data scientists access
to the entire cluster
or a slice of the cluster
using native concepts such as namespaces,
node selectors,
tolerance, et cetera.
The configuration and life lifecycle management
of the cluster,
such as setting up autoscaling,
upgrading to new Kubernetes version
is all transparent, flexible,
and continues to remain the responsibility
of the IT operations team.
Next, let's understand
what the data scientist needs to do.
Does the data scientist need to know Kubernetes?
Naturally.
One of the core principles we adhere to
was splitting the administrator persona
and the data scientist one
with separate roles and responsibilities.
Data scientists do not need to know
anything about or learn Kubernetes.
To them, it is just another compute target
that they can submit their training jobs to.
Within Azure Machine Learning,
they will be able to see this
as an option for "Attached Compute"
and just select the relevant Arc-enabled cluster
as the compute option for training.
This enables them to use familiar tools
such as Jupiter notebooks,
TensorFlow,
PyTorch, et cetera,
and spend their time solving machine learning problems
rather than managing the infrastructure
that they are running on.
Here's a pro tip.
Do not expect data scientists to be able to understand
and manage execution environments like Kubernetes.
Their time as best utilized
building and optimizing machine learning models.
Now, Kubernetes comes with its own set of challenges
around security, management,
and governance.
Azure Machine Learning
and Azure Arc-enabled Kubernetes come together
to ensure that not only
is an administrator able to centrally monitor
and apply policies on your workloads
on Arc infrastructure,
but also ensure that the interaction
with Azure Machine Learning service
is secure and compliant.
This along with a consistent experience
across the cloud and on-premises clusters
no longer requires you to lift
and shift machine learning workloads,
but seamlessly operate them across both.
You can choose to just run in the cloud
to take advantage of the scale
or just run on excess on-premises capacity
while leveraging the single pane of glass
Azure Arc provides
to manage all your hybrid
and multi-cloud infrastructure.
Hope this gives you a good understanding
of an enterprise ready hybrid
and multi-cloud machine learning deployment.
To recap, we looked at two core concepts.
The considerations,
norming on an execution environment
and management service
that can be made available anywhere.
And the solution.
With Azure Arc and Azure Machine Learning,
you enable both admins and data scientists
to do their job to the fullest
and achieve operational efficiencies.
Well, that's it for this video.
Hope this inspires you to think
of innovative ways to utilize these hybrid
and multi-cloud capabilities for machine learning.
As always,
to try these on Azure,
please spin up a free Azure Machine Learning workspace
at aka.ms/aml-trial
and download the 30 day learning journey
from the data scientist resources page
at aka.ms/data-scientists.
See you in another episode
of the "Machine Learning essentials"
(upbeat music)