[MUSIC]
Hi, my name is Sherry Wang,
Senior Program Manager of Azure HPC and AI Team.
Today, I would like to introduce
Azure AI and AI infrastructure.
AI has reached a critical juncture;
it is experiencing quantum leaps in model complexity.
This complexity has resulted in
a corresponding requirement for IT infrastructure.
The need for AI-first infrastructure
has been established,
one that can not only scale up to
take advantage of accelerators within
a server, but also scale out to embrace
many servers distributed across a network.
Microsoft Azure is the only public cloud that
offers purpose-built AI supercomputers.
For example, massively scalable IT infrastructures
comprised of NVIDIA InfiniBand interconnected
NVIDIA A100 Tensor Core GPUs.
Azure Machine Learning facilitates uptake of this
AI-first infrastructure from the earliest stages of
development through
enterprise-grade production deployments
that require full-blown MLOps.
Scale-up-and-out HPC supercomputers
are both performance and power efficient.
HPC technologies have significantly advanced science
and engineering from innovations in hardware to software,
parallelism and communication have been
repeatedly leveraged to
advance the technology that
comprises HPC infrastructure.
Scale-up HPC takes advantage
of shared memory to introduce parallelism via threads.
When combined with the
vector-processing capabilities available from GPUs,
these SIMT devices have proven
extremely effective at processing arrays of data.
In fact, multi-dimensional arrays of data.
With the added capability of a high-bandwidth,
low-latency interconnect fabric,
scale-out HPC takes advantage of distributed memory
for parallel computing by interleaving
computation and communication across
a network of computer nodes.
Scale-up-and-out HPC combines the attributes of
vertical and horizontal system scaling to address
the most-demanding workloads from science and engineering.
Microsoft Azure in partnership with NVIDIA,
delivers purpose-built HPC and
AI hardware in the Cloud to meet
even the most demanding
real-world application workloads at scale
while meeting price-performance and
time-to-solution requirements.
And with next-generation machine learning tools included,
you can incorporate intelligence within your workloads to
drive smarter simulations and
empower intelligent decision making.
Scale-up-and-out infrastructures
based upon NVIDIA GPUs and
NVIDIA Infiniband interconnect fabrics rank
amongst the most-powerful supercomputers on the planet.
Azure placed within the top 10 on the top 500
and has four more placements with the top 35,
a remarkable and notable first for the public Cloud.
You can find more information from the link above.
The progress of AI has been
astounding from just a few years ago,
with solutions pushing the envelope
by augmenting human understanding,
preferences, intent, and even spoken languages.
AI is improving our access to knowledge,
helping us provide more efficient solutions
that fuel transformation beyond imagination.
With the rapid growth and transformation,
AI demand for compute power has grown by
almost four orders of magnitude, outpacing Moore's law.
With AI powering a wide array of important applications,
including natural language processing,
robot-powered process automation, and
machine learning and deep learning,
AI companies are finding new ways to get more out of
each piece of silicon using
tools like mixed-precision modes,
enabling them to do more with less.
The only way to satisfy the expanding ambitions and
imaginations of the AI research community
is through massive scale,
and the only way to scale up real-world applications
without slowing down performance is via fast,
low-latency connections such as
InfiniBand interconnected GPUs from NVIDIA,
for a programmatic exploitation of
interconnected GPUs as accelerators within
scale-up-and-out infrastructures to deliver
the kind of performance that today's workloads demand.
Back to 1956,
Artificial Intelligent was a broad discipline,
the value of which has yet to be realized.
Forty years later, machine learning is
the most highly relevant subset of AI in use today.
Nowadays, the most demanding
requirements originated from deep learning,
a subset of machine learning,
at scale for extracting insights from big data sets.
These terms bandied about interchangeably.
An AI-First Infrastructure and Toolchain for
Any Scale aims to address this challenge.
AI-first infrastructures and toolchains are proving
to be of value in most industries.
Here, we provide use-case examples from manufacturing and
retail as well as
a compelling customer story from healthcare.
AI-first infrastructures and toolchains are
having significant impact in manufacturing.
Based upon industry trends,
we've identified two use-case examples,
namely predictive maintenance, and product quality.
By using NVIDIA GPUs on Azure for
predictive maintenance and product quality,
customers can see a plethora of benefits in scaling,
speed, quality, and accuracy.
Of course, these benefits are the outcomes derived from
technology-based solutions that
address the nuances of each scenario.
Each of these examples requires
AI-first infrastructure and toolchain as
the scenarios under consideration are demanding ones,
for example, to significantly
reduce false positives and negatives in
predictive maintenance to account for
subtle nuances in ensuring product quality.
The results speak for themselves,
results that impact the manufacturer's bottom line.
[MUSIC]
We are a premium manufacturer.
140 00:07:32,030 --> 00:07:35,870
sets standards.
Our biggest challenge so far is the transformation
from the combustion engines
to the new electric drive systems,
which require completely different production processes.
We choose especially
Azure Machine Learning to be part of our solution
because this has a rich functionality and also with
the computation of the power
provided by individual GPUs.
For the trainings of our models,
we are using Azure Machine Learning and also NVIDIA VMs.
The no-code approach
is the key here.
The solutions are designed as a self-service.
The complexity lies in the backend.
The user does not need any specific IT knowledge.
After the robot has
taken the pictures,
they are transferred to Azure for the model training.
There we use our RCV
labeling tool and also the cockpit to create
a project to label the pictures and then
also train the model, and for
this connection to the model training we are using
Azure Machine Learning and also NVIDIA VMs.
The models are deployed on
NVIDIA GPUs on the shop floor to provide
the computational power that is needed
to realize the corresponding response times.
The solution supports all this evolution
until we get the higher,
robust and fully automated inspection solution.
Suitable spectral solution combines
the strengths and abilities of the included partners -
Microsoft, NVIDIA, robots, one in one robots.
The result in the end is
a highly robust and automated inspection solution.
It adds much benefit to
flexibility and agility of the old production.
Our production system
is cutting edge.
We drive innovation.
We rely on effective applications that can be
integrated cost-effectively and flexibly.
AI-first infrastructures and
toolchains are having a significant impact in retail.
Here, we've used
deep demand forecasting and
the connected store as use-case examples.
Primary benefits include improvements
in performance, training times,
TCO, ROI, and shrinkage,
all key areas for companies in the retail industries.
Like in manufacturing, these benefits are the outcomes derived
from technology-based solutions
that address the nuances of each scenario.
Each of these examples requires
an AI-first infrastructure and toolchain
as the scenarios under consideration are demanding ones
For example, to significantly improve
the timeliness and accuracy of demand forecasting for
retailers to deliver
a frictionless experience to customers in
a retail storefront scenario.
In both manufacturing and retail,
use is made of Azure Machine Learning to build and
operationalize deep learning models
that execute on NVIDIA GPUs,
whereas model training makes use of
the N series virtual machines on Azure,
inferencing is often shifted to the edge.
These data-driven solutions require sophisticated
deep learning models, models than are much more
sophisticated than those offered
by machine learning alone.
In turn, this sophistication
demands an enabling AI-first infrastructure and toolchain.
But this infrastructure is not only
being used in manufacturing and retail,
but also the healthcare industry.
[MUSIC]
At Sensyne Health, we work
with some of the largest pharmaceutical companies
delivering research insights based
on questions that they pose on real-world data.
The aspect of COVID for us
was a massive amount of uncertainty.
The one thing that we did know at the time was that
diagnosis and diagnosis at scale
would be a major part of what was going to be needed.
Magnify is a solution which allows you
to turn a phone remotely into a diagnostic tool.
We worked out that we could incorporate
our image capability using
very powerful neural networks to,
actually, very accurately determine
what that reading actually was after you've
taken it, and the reason that's important is that
the human eye and the brain that interprets
the image is very varied.
Too many false negatives puts
an added burden on the healthcare system.
It means that more people
will be misdiagnosed and therefore
take incorrect action.
Magnify puts clinical grade diagnostics in the home of
any end user, not just a healthcare professional,
but a mum, a dad, an aunt, an uncle, it doesn't matter.
We were faced with the unique
challenge of supporting
around 500 transactions per second.
The way we did it
is using
Azure's capabilities and
GPUs and the Machine Learning behind it.
Azure has all of the
governance certificates that enable us to meet
those requirements from GDPR
through to some of the security standards.
We examined multiple GPUs that were
available to us under the issue of cluster.
The one which gave us the most advantage
was the Triton Inference Engine.
We were able to go from
a two-second turnaround time
from running all of these algorithms
to a .1 second and also enable us to deliver
very, very high accuracy results,
several times higher than other GPUs that we tested.
If we hadn't had
a similar technology offering from Azure,
it would have been really difficult
to achieve what we have done.
We've never had delivered
a diagnostic solution via
a camera phone using
machine learning technology had it not
been for COVID and
the partnership that we had with NHS and Microsoft and NVIDIA
We did something really
meaningful and that is
what makes all the difference in this world.
[MUSIC]
This is just the start
of the conversation for
AI and AI infrastructure, and
we invite you to read more in our AI-first
Infrastructure and Toolchain for
Any Scale whitepaper at aka.ms/AIWP.
Thank you for joining today's session.