(bright upbeat music)
- Welcome to the Machine Learning Essentials series.
In this video we get to tackle data scientists tools
for Machine Learning and how they work on Azure.
Machine learning practitioners have a wealth of choices
when it comes to experimenting, building models,
collaborating, sharing, and even managing
the end to end process.
While we won't cover all of the tools available,
there are a ton, we will look at a few that work really well
with the cloud.
Let's start with experimentation.
This part of the process usually starts with exploring,
understanding, and visualizing the data in a way
that helps data scientists understand what they can
and can't do.
A tool of choice almost always begins with Notebooks.
If you haven't heard about Notebooks,
they are a powerful tool for both experimentation
and exploration.
The amazing thing about Notebooks is the versatility
that is afforded to the user.
You can test out code, visualize data,
and test out different hypotheses
to arrive at amazing machine learning solutions.
These Notebooks can be run locally using either Jupyter
or directly in Visual Studio code.
Using Visual Studio code is a powerful way to keep
all of your machine learning assets together.
With Azure, you also have the option
of moving the Notebook, compute and storage to the cloud.
Options include spinning up a data science virtual machine
or using an Azure machine learning workspace that has
compute instances that have Notebooks built right in.
And the best thing part about these Notebooks
is that they can be shared amongst colleagues.
Now let's talk about libraries and frameworks
for building models.
Building models is likely is the funnest part,
at least for me, of machine learning.
When it comes to building models,
there are several amazing frameworks that help.
Generally, the best thing to do is to start simple.
Scikit-learn is a powerful framework
that has several pre-built models that can be tried out.
Scikit-learn has built in models for classification,
regression and clustering.
The library also has functionality
for dimensionality reduction, model selection,
and even pre-processing for feature extraction
and normalization.
Now, if you need more fine-grained control
over model creation or the model shapes,
especially when building deep learning models,
the most popular frameworks are PyTorch and TensorFlow.
These amazing open source frameworks
also have vibrant communities that are excited to help.
Let's talk about tools for collaboration.
And if we're gonna talk about collaboration,
we always start with GitHub.
GitHub is a development platform
inspired by the way you work.
From open source to business.
You can host and review code, manage projects,
and build software alongside 50 million developers.
It is an amazing platform to not only share and review code,
but also to build consistent engineering practices
when it comes to continuous integration
and continuous delivery.
If you're doing machine learning,
you should think about using GitHub.
And now even more exciting, new to GitHub is Codespaces.
Now machine learning practitioners can share environments
where code can run code as well.
These environments can be packaged up with the repository
and run directly in the browser giving collaborators
near instant access to shared work.
It is a great way to get people to get their feet wet
with what you're doing.
Finally, collaborating is amazing
with Azure Machine Learning.
You can share everything from Notebooks to datasets,
experiment runs to models,
and it includes a bunch of robust tools
for building models quickly.
Things like automated machine learning,
basically select the models for you.
There is also a drag and drop designer
and the ability to do data labeling.
We'll talk more about Azure Machine Learning later.
Lets talk about some tools for managing
the machine learning end to end process.
Let's start with MLflow.
MLflow is an open-source library for managing the life cycle
of your machine learning experiments.
MLflow Tracking is a component of MLflow that logs
and tracks your training run metrics and model artifacts,
no matter your experiment's environment.
Locally on your computer on a remote compute target,
a virtual machine, or an Azure Databricks cluster.
This open source software also works really well
with Azure Machine Learning.
MLflow Tracking with Azure Machine Learning
lets you store the logged metrics and artifacts
from your remote runs
into your Azure Machine Learning workspace.
Any run with MLflow Tracking code in it,
will have metrics logged automatically to the workspace.
MLflow Projects allow for you to organize
and describe your code to let other data scientists
or automated tools run it.
MLflow Projects with Azure Machine Learning
enables you to track and manage your training runs
directly in your workspace.
The metrics and artifacts from MLflow logging
are also kept in your workspace.
You can view them anytime in the workspace
directly in Azure Machine Learning Studio.
You can also register and track your models
with the Azure Machine Learning Model Registry.
Azure Machine Learning models are aligned
with the MLflow model schema making it easy to export
and import these models across different workflows.
The MLflow related metadata such as run id
is also tagged with the registered model for traceability.
Users can submit training runs, register,
and deploy models produced directly from MLflow runs.
Finally, deploying your MLflow experiments
as an Azure Machine Learning web service
allows you to leverage and apply
the Azure Machine Learning model management
and data drift detection capabilities
to your production models.
Let's talk about Kubeflow.
If you would like more fine-grained control
over your cloud hardware and love Kubernetes,
Kubeflow is an amazing project.
The Kubeflow project is dedicated to making deployments
of machine learning workflows on Kubernetes simple,
portable and scalable.
Their goal is not to recreate other services
but to provide a straightforward way to deploy
best-of-breed open-source systems for machine learning
to diverse infrastructures.
Anywhere you are running Kubernetes,
you should be able to run Kubeflow.
Finally, Azure Machine Learning empowers developers
and data scientists with a wide range
of productive experiences for building, training,
and deploying machine learning models much faster.
Accelerate time to market and foster team collaboration
with industry-leading MLOps,
which is DevOps for machine learning.
Innovate on a secure, trusted platform,
designed for responsible machine learning.
And it has productivity built in for all skill levels.
Code with built-in collaborative Notebooks
and one-click Jupyter experience.
Use drag-and-drop designer or automated machine learning
for accelerated model development.
It also has robust MLOps capabilities that integrate
with existing DevOps processes and help manage
the complete machine learning lifecycle.
And now, when it comes to responsible
machine learning capabilities, Azure Machine Learning
can understand models with interpretability and fairness.
It can protect data with differential privacy
and confidential computing and it can control
the machine learning lifecycle with audit trails,
asset management and tagging.
And finally, it supports open-source frameworks
and languages including the very tools
we have just been talking about.
So to recap.
Machine Learning practitioners have a wealth of choices
when it comes to experimenting, building models,
collaborating, sharing, and even managing
the end to end process.
And with such a wealth of tools, let's get started
and build the next amazing machine learning model.
(soft upbeat music)