(upbeat music)
- Hello, welcome to this new episode
of the Machine Learning Essential series.
I'm Nishant Thacker and in this video we'll talk
about securing your Machine Learning environments.
Now, security is a topic that is very important
for organizations of all sizes.
Recent events have further renewed the rigor around ensuring
that security isn't being compromised.
Since it's one of the biggest reasons
Machine Learning projects get stuck,
it is important to think of security
from the very beginning,
instead of trying to patch it in at the end.
We'll taken the next few minutes
to understand the various aspects of security
within Azure Machine Learning
and how you can secure your resources inside Azure.
We'll break this down into four areas.
One, authentication or authorization.
Two, network security, three data encryption
and four policy and monitoring.
Let's get started.
The first thing to understand is how do we restrict access
to resources and operations.
Authentication to Azure Machine Learning workspace is based
on Azure Active Directory or Azure AD.
In general, there are three authentication workflows
that you can use when connecting
to the workspace.
Interactive, where use your account
and Azure Active Directory to either directly authenticate,
or to get a token that is used for authentication.
Interactive authentication enables you to control access
to resources, such as web services on a per user basis.
Then you have service principle
where you can create a service principal account
in Azure Active Directory
and use that to authenticate or get a token.
A service principle is used when you need
an automated process to authenticate
to the service without requiring user interactions.
For example, a CI/CD script that train
and tests the model every time the training code changes.
And then you have managed identity.
When using the Azure Machine Learning SDK
on Azure Virtual Machines,
you can use a managed identity on Azure.
This workflow allows the VM to connect to the workspace
using the managed identity
without storing credentials in Python code
or prompting the user to authenticate interactively.
Azure Machine Learning compute clusters can also
be configured to use a managed identity
to access the workspace when training models.
It can communicate with services
like Azure Container Registry or ACR
via these managed keys instead of requiring admin keys
for the ACR, which could be a difficult task
for most enterprises.
Regardless of the authentication type used,
role-based access control RBAC, is used to scope the level
of access allowed for the resources.
For example, an account that is used to get the access token
for a deployed model, only needs read access
to the workspace.
This is what we call authorization.
Let's see how that works.
In Azure Machine Learning,
you can create multiple workspaces
and each workspace can be shared by multiple people.
When you share a workspace, you can control access
to it by assigning specific roles to users.
There are three built-in roles,
owner, contributor and reader.
Of course their roles correspond to specific rights
to operations within the workspace.
Now here's a pro tip.
If the built-in roles don't meet your needs,
you can create custom rules.
These can help you define roles that are best suited
for your team structures and organizational policies.
Now custom roles are supported to control all operations
inside the workspace, such as creating a compute
submitting a run, registering or data store
or deploying a model.
Custom roles can have read, write
or delete permissions
on the various resources of a workspace
such as clusters and data stores and models and endpoints.
You can make the role available
at a specific workspace level,
a specific resource group level
or even at the subscription level.
These roles are used to secure compute targets and data.
Owners and contributors can use all compute targets
and data stores that are attached to the workspace.
Now that we understand the authentication
and authorization aspect, let's look
at network security.
To restrict network access
to Azure Machine Learning resources,
you can use Virtual Networks.
VNets allow you to create network environments
that are partially or fully isolated
from the public internet.
This reduces the attack surface for your solution,
as well as the chances of data exfiltration.
You might use a virtual private network
or a VPN gateway to connect individual clients
on your own network, to the VNet.
But network security is not only
about putting your ML workspace
inside a Vnet, a Machine Learning environment
typically interacts with a bunch
of other services like storage,
container registry, Kubernetes service, et cetera.
We should ensure that these services
can also communicate securely
with the Machine Learning workspace.
This is where Private Link comes in.
The Azure Machine Learning workspace
can use Azure Private Link
to create a private endpoint behind the Vnet.
This provides a set of private IP addresses
that can be used to access the workspace
from within the Vnet.
Some of the services that Azure Machine Learning relies on,
can also use Azure Private Link
but some rely on network security groups
or user defined routing.
This private IP, also sometimes referred to as,
no public IP is a must have for a lot of enterprises
to ensure there's nothing open to the public internet.
Azure Machine Learning provides the capabilities
to ensure end to end private IP deployments
for your Machine Learning environments
with Vnets and Azure Private Link.
Here's another pro tip.
The Azure Machine Learning team has a dedicated ARM template
with an Azure Quickstart templates that deploys
the secure configuration seamlessly.
You don't need to worry about setting this up on your own.
We've tackled users and networks.
Now it's time to secure the data.
As part of the Machine Learning process,
you may bring in data from a variety of sources
and register them as data stores or datasets.
All this data can be encrypted by customer managed keys,
but while these sources
may have their own security capabilities, when they interact
with Azure Machine Learning and share data to be processed
inside its workspace, there is a need for encryption
both at rest and in transit.
Here's another pro tip.
If your workspace contains sensitive data
that needs encryption at rest,
Azure Machine Learning provides
a flag called hbi_workspace
but should be set appropriately
while creating your workspaces.
It's now the hbi_workspace flag controls the amount
of data Microsoft collects for diagnostic purposes
and enables additional encryption
in Microsoft-managed environments.
In addition, it performs the following actions.
First, it starts encrypting the local disk
in your Azure Machine Learning compute clusters.
Second, it cleans up your local disk between runs.
Third, it securely passes credentials
for your storage account, container registry
and SSH account from the execution layer
to your compute clusters using the key vault.
And finally it enables IP filtering
to ensure underlying batch pools cannot be called
by any external services other than
the Azure Machine Learning Service.
Additionally, Azure Machine Learning stores
some metadata in an Azure Cosmos DB instance.
It is important to note that this metadata
just contain some operational information
such as run history.
It does not have any sensitive data such as train model,
train results or anything like that.
This Cosmos DB instance where the metadata stored,
is associated with a Microsoft subscription managed
by Azure Machine Learning.
All the data stored in Azure Cosmos DB
is encrypted at rest with Microsoft-managed keys.
If needed, the user has the option to configure
all of these with customer-managed keys as well.
This includes the storage, the container registry,
the container instance, or Kubernetes
or even the Cosmos DB instance where the metadata is stored.
To use your own customer managed key
to encrypt the Azure Cosmos DB instance,
you can create a dedicated Cosmos DB instance
for use within your workspace.
We recommend this approach
if you wanna store your data such
as run history information outside
of the multitenant Cosmos DB instance hosted
within the Microsoft own subscriptions.
Now these actions ensure your data is secure
and encrypted at rest.
Now let's look at encryption in transit.
Azure Machine Learning uses TLS 1.2
which is a cryptographic protocol designed
to provide communication security over a computer network.
Azure uses TLS 1.2 to secure internal communications
between various Azure Machine Learning microservices.
All Azure storage access also occurs
over a secure channel.
On similar lines, all external calls made
to the scoring endpoints also happen over TLS.
Here's another pro tip.
For production-grade encryption during training,
we recommend using Azure Machine Learning compute clusters
and similarly for production grade encryption
during inference, we recommend using
Azure Kubernetes Service.
The Azure Machine Learning compute instance
is a dev/test environment.
When using it, we recommend that you store your files
such as notebooks and scripts in a file share
and your data should also be stored in a data store.
The next important aspect of securing
your Machine Learning environment is the ability
to enforce policies and monitoring
the workspace effectively.
Azure Machine Learning integrates
with Azure Policy and the Azure Security Center,
which provides Microsoft-created
and managed definitions related
to different compliance standards.
There are several policy options with Network Security
and Data Protection that can be applied via Azure Policy.
For example, an organization can ensure
that no Machine Learning workspace is created
without a private link configured.
Enabling observability
across today's complex computing environments,
running distributed applications requires the collection
of operational data from every layer
and every component of the system.
You need to be able to drive deep insights from this data
and consolidate it into a single pane of glass
with different perspectives to support the multitude
of stakeholders in your organization.
Azure Monitor collects and aggregates data from a variety
of sources into a common data platform where
it can be used for analysis, visualization and alerting.
It provides a consistent experience on top of data
from multiple sources, which gives you a deep insights
across all your monitored resources
and even with data from other services
that store the data in Azure Monitor.
It can provide you the ability to capture both metrics
and workspace activity logs.
This ensures you have granular details on everything
that is happening inside your
Azure Machine Learning workspace.
Here's another pro tip.
Azure Monitors metrics data is available for 90 days.
However, when you're creating charts,
only 30 days can be visualized inside a single chart.
If you wanna visualize a full 90 day period,
you can break it into three charts of 30 days
within the 90 days.
We've covered a lot in the short video,
but security is a vast topic
and only one of the components of enterprise readiness.
Azure also provides capabilities
and templates for cost management, deployment planning,
differential privacy, and even confidential computing
but we leave that for another day.
To summarize, in this video we observed
that Azure Machine Learning provides easy to use
and seamlessly integrated capabilities
around access control, network isolation,
data encryption and policy and monitoring.
All these can be configured in automated way using
either the Command Line Interface or ARM templates
or even designing your own tools
with the REST APIs provided.
These are important
as with these automation capabilities,
organizations of all sizes can ensure
that they can enable their data science teams to innovate
and deliver value securely.
All this without having to get exceptions
from the IT and compliance organizations.
Well, that's it for this video.
As always to try out these capabilities on Azure,
please spin up a free Azure Machine Learning workspace
at aka.ms/aml-trial
and also download the 30 day learning journey
from the data scientist resources page
at aka.ms/data-scientists.
See you in another episode
of the Machine Learning Essentially series.
(upbeat music)