(gentle upbeat music)
- Hi, I'm Steve Scott
Corporate Vice President of Hardware Architecture in Azure.
I joined Microsoft this summer
after a long career in the HPC space
and I wanted to provide my impression
of HPC and AI in Azure.
I've spent my whole career focused on building the largest,
most capable systems on the planet.
I joined Cray Research, the iconic supercomputing company
straight out of grad school,
rode through our acquisition by SGI
and subsequent spin out to become Cray inc,
and then I've been CTO of Cray since 2004,
with brief detours at Nvidia
and in the Google platforms group.
I'd loved building leading edge supercomputers.
So why join Microsoft? The answer is simple.
We have an unprecedented opportunity in front of us.
Microsoft has the scale, the talent and the resources
to do amazing things in the world.
And we've got the ability to optimize across levels
from Silicon all the way up to end services
and have broad and meaningful impact in the world.
Microsoft has the vision and the ambition
to take on some really big problems.
Here are just a few examples.
Project Palix is looking at DNA storage.
We all know about the massive growth in data,
and we need ways to reliably and durably store it.
Synthetic DNA offers the possibility of incredible density
up to an exabyte in a cubic millimeter.
Easily an exabyte in the palm of your hand.
It's non-volatile, it's reliable.
It's durable for up to millions of years
and it's eternally relevant.
Project Natick is looking at underwater data centers
from a sustainability and air reduction perspective.
We put hundreds of servers and FPGAs and 27 petabytes
of disk down into an inert nitrogen environment
and ran it, lights out for two years.
It used local renewable power, free cooling,
and had very low environmental impact.
And what we found was dramatically lower failure rates,
about 1/8 versus the land-based control group.
Lastly, the Microsoft Quantum program is pursuing
a novel type of qubit, a topological qubit that we believe
has superior scaling properties.
It has the promise of transformational computing power
in the longterm, but it turns out in the short term,
the algorithms that we are looking at for quantum computer,
so-called quantum inspired algorithms are proving
to provide dramatic performance improvements
on very challenging HPC applications.
So these are just a few of the examples of the sorts
of projects that Microsoft is taking on.
So when I came to Microsoft, what did I find?
Well, first of all, massive scale.
Azure has 61 regions worldwide,
each with multiple data centers, individual data centers
that have up to tens of megawatts of power.
This presents very similar problems
that we've been facing The HPC community.
Power and cooling of course are at the top of that list.
But interconnects are also very important.
For most of the workloads,
we have somewhat lower bandwidth per end point,
not so much in the AI space, but we have even larger scale.
It turns out we also need very high performance
and increasingly flash-based storage,
high performance storage that is.
And manageability of course is very important at this scale.
Now most of what's running in Azure data centers
is what we in the HPC space would call capacity computing,
but it turns out that there's a very sizable team
in Microsoft that's focused on true HPC.
I believe that Azure is at the very forefront of HPC
compared to other cloud vendors.
The HPC offerings here are custom built
for high performance computing,
not repurposed general cloud infrastructure.
We've got a variety of different HPC optimized node types,
some compute optimized,
some more memory bandwidth optimized, some based on GPU's.
And importantly, we have dedicated InfiniBand networks.
We're not just using a piece of the larger,
highly virtualized ethernet data center cluster.
We've got a variety of different storage options for HPC,
including high-performance Lustre offerings.
And we built up a very mature software stack with a variety
of different workload orchestration options,
a rich ecosystem of libraries and tools
and more services available than I could dream of
in my previous professional lives.
Think about databases and analytics and IoT, DevOps
and AI and networking, security, media,
identity management and so on.
We also have genuine HPC and AI expertise from the engineers
to sales, to service people that eat, breathe and sleep HPC.
And Azure provides different HPC optimized delivery models,
such as reserved instances.
Lastly, we're supporting an active set
of ecosystem partners in the HPC space.
Now, one of the most important topics in HPC today
is artificial intelligence. Everyone is interested in AI.
Azure Machine Learning is providing an enterprise grade,
machine learning environment to build and train
and test and deploy AI models.
Microsoft also has Project Brainwave,
which provides a highly performance platform
for real-time inference in the cloud or at the edge.
And we're deploying AI at increasingly larger scales.
The Turing Natural Language Generation model
is 17 billion parameters.
And we're currently looking at models that are well
over a hundred billion parameters.
We have trillion parameter and larger models
that are coming.
Microsoft has partnered with OpenAI and has deployed
a leading edge AI supercomputer that has 285,000 CPU cores,
10,000 GPUs and a 400 gigabit per node interconnect.
This would make it a top five supercomputer.
Microsoft AI capabilities can be applied
to build and deploy AI models in the cloud
or the edge in their own right to solve problems.
But in the HPC space, they can be used to augment
and enhance simulation and modeling.
Consider for example,
a CFD code where you're iteratively solving,
Navier-Stokes equations,
you can pre-condition that model with the output
of an AI model and dramatically improve the time
to solution even though you're still using
the traditional simulation modeling
to provide the final result.
Now there's no end in sight for the appetite
for higher AI performance.
Transformer models recently have shown tremendous success
and they keep getting better with scale.
So if a 10 billion parameter model is great,
a hundred billion and a trillion parameter, a billion,
a trillion parameter model will be even better.
There's really much more to come in the AI space.
And I'm personally extremely excited
by what I see us doing in AI.
So is this the right time to move HPC workloads
to the cloud?
I used to ask that question from the perspective
of an HPC system company.
And my answer then was, it depends.
Some workloads are better suited for the cloud,
some are better suited for on-premises systems.
And the answer is probably a hybrid cloud.
It turns out that that answer is the same
from my cloud provider perspective.
Though I do think that the balance is shifting.
On-premises computing does provide some advantage.
It will be less expensive if you can keep
your system highly utilized and the machine
matches your workload needs over its lifetime.
This means that you have to predict,
perhaps a couple of years in advance,
what your computing needs will be.
You need to choose the right technologies
to go into your system and over the lifetime
of that on-premise system, your workload matches
what that system provides.
On-premises computing can be better optimized
for capability HPC than a generic public cloud.
And sometimes you need to have
your computing local for whatever reason.
It turns out though that cloud computing
has some distinct advantages for HPC.
The first of course is the well-known argument
about scale agility.
The amount of computing that you need varies over time.
Again, you have to predict on an on-premises system,
how much computing you'll need a couple of years out,
but year to year, your needs may vary, month to month,
week to week or even day to day,
you might need different amounts of computing.
More important than scale agility though,
is technology agility,
the type of computing that you might need varies over time.
Different projects may require CPUs or GPUs
or different amounts of storage
or interconnect bandwidth, et cetera.
And cloud computing gives you the ability to vary
what technologies you bring to bear
for different problems over time.
Cloud computing allows you to pertain flexibility
in terms of your spending commitments,
the timing of that spending and the type of system
that you're using.
And of course there are a number of other benefits in terms
of the sustainability and resilience and security
that a cloud provider just provides for you
that you don't have to take care of.
Lastly, you get to benefit from this vast array
of Azure software and services
that I talked about previously.
So with Azure, you can also leverage
these pre-built purpose-built HPC clusters
to provide capability computing,
as opposed to having to rely on the generic cloud clusters.
So we've actually been running large workloads in Azure.
We've demonstrated many applications that are running
at tens of thousands, up to 80,000 CPU cores.
The Graph500 of course, is a very challenging workload
that's really looking at very network
and memory intensive applications.
At Azure we've recently submitted results
that would place Azure at approximately number 15 or 16
on the Graph500 lists for breadth-first search.
We ran this problem over 40,000 cores
and provided a G. TEP score of 1.15 million.
We're not actually aware of any other public cloud
that has submitted Graph500 results.
Star-CCM is a CFD application and at the launch
of the Azure HBv2 offering, Idaho National Lab ran
a Star-CCM on Seaman's largest publicly shared CFD model.
This is a 1 billion cell Le Mans sports car,
ran it on 57,000 cores and the scaling was very good.
It achieved 99% parallel efficiency at 44,000 cores
and 84% parallel efficiency at 57,000 cores.
WRF is a very well-known weather code.
Also at the launch of the HBv2 offering
we ran the largest publicly available WRF model,
which is Hurricane Maria on 80,640 cores.
This actually achieved 110% scaling efficiency,
which you can do because of cache and memory effects,
at 15,000 cores and at 80,000 cords,
it was still getting 72% efficiency.
This is really just limited by the model size.
We can compare that against the UCAR-published data
on the tax Cheyenne system,
which debuted in 2016 as number 20 on the top 500 list.
And in Azure, a 672 dual socket AMD Rome node
got 2.2 the wall clock performance
of a thousand dual socket Broadwell node Cheyenne system.
So this is getting credible,
highly performance, scalable results.
Lastly, we just recently provided some NAMD results.
We're working with the Beckman Institute
at University of Illinois.
This is a partnership that was formed
when we were donating time for the COVID-19 relief efforts.
Working with Beckman, we scaled NAMD to 86.4 thousand cores
on the 210 million atom tobacco mosaic virus.
This was 40% higher scaling
than the team had ever previously published.
That was on the TACC Frontera system.
And it achieved 76% parallel efficiency at full scale.
The largest published application scaling
that we've seen anywhere on computing clouds
is just 7,200 cores. So 12 times larger scaling on Azure.
All of these results were on the HBv2 AMD Rome nodes,
but we've also recently announced
that we're introducing systems with thousands
of A100 Nvidia GPUs.
These can be used for acceleration of both AI workloads
and traditional simulation and modeling,
rivaling the performance of the largest
on-premises super computers.
So what does the road ahead offer for HPC in the cloud?
I think we're going to see continued movement
toward the public cloud and toward hybrid cloud.
We're gonna see more diversity
and more high-performance technologies feeding
into the public cloud, both in the network space,
the storage space and various compute technologies.
We all know about the slowing of Moore's Law.
What this means is that we can no longer count
on rapid exponential growth in the underlying Silicon.
And we need to turn to specialization
to provide differentiated performance
on different types of workloads.
So we're going to see greater flexibility at Microsoft
to dynamically combine different types of network storage
and compute resources to solve the problems at hand.
We're going to see more robust
and mature software stacks as well.
Microsoft excels at providing great environments
for developers,
both in the general compute DevOps environment,
as well as machine learning ops.
And we're going to see that environment continue
to get better and better.
We'll see better security over time.
Right now it has switched over from having better security
on on-premises systems to really having better security
in the public cloud.
Microsoft has a real leadership stake here.
We're the first to introduce confidential computing
to allow you secure your data while in use,
as well as when it's at rest or in motion.
And we're going to continue to see more
and more confidential computing options
as well as general improvements to security.
But the area that I think that we'll see
the most exciting changes in the road ahead
is in this convergence of AI and HPC.
There's just tremendous potential there
to take traditional simulation and modeling
and make it perform better and faster.
I'm very excited about this?
So in conclusion, my impression having spent years
as the CTO of world's leading super computing company
is that Azure has real genuine HPC experience
and capabilities. Microsoft is taking it very seriously.
We built an experienced HPC organization.
We're building true HPC systems that are running workloads
and achieving very high performance at scale.
We're clearly ahead in providing true HPC capability
in the cloud.
And as Corporate Vice President
of Hardware Architecture at Azure,
I'll be focused on extending that lead,
bringing exciting new compute and AI technologies
over the coming years.
With that, thank you very much for your time
and have a great rest of your day.
(gentle upbeat music)