It starts with
AI infrastructure layer
and our approach to Azure as the world's computer.
You know, we offer the most comprehensive
global infrastructure with more than 60
datacenter regions, more than any other provider.
Being the world's computer means that we need
to be even the world's best systems
company across heterogeneous infrastructure.
We work closely with our partners
across the industry
to incorporate
the best innovation
from power, to the data center, to the rack,
to the network, to the core compute,
as well as the AI accelerators.
And in this new age of AI, we are redefining
everything across the fleet and the data center.
So let's start on how we power the data center.
As we build them, we are working
to source renewable power.
In fact, today
we are one of the largest buyers of renewable energy
around the globe.
We have sourced over 19 gigawatts
of renewable energy since 2013.
I mean, just to put that in perspective,
that's the equivalent
to the annual production of 10 Hoover Dams.
And we're working with producers to bring new energy
from wind,
solar, geothermal, and nuclear fusion as well.
And as we pursue this ambition to not just be carbon free,
but to even erase our historical carbon emissions,
I'm really excited to share that we are on
track to meet our target of generating
100% of the energy we use in our data
centers from zero-carbon sources by 2025.
Now let's talk about the network
that connects our data centers.
It's one of the most advanced and extensive
in the world already, and to meet the demands of AI
and the future workloads,
we are driving up the speeds.
Our breakthrough hollow core fiber technology
is delivering a 47% improvement in speed
because photons are able to travel
through these microscopic air capillaries
instead of through solid glass fiber.
This is really cutting edge technology.
In fact, we are manufacturing
this fiber ourselves in the world's
only dedicated factory for hollow core
fiber production.
Our first deployment,
in fact, is already live connecting our data centers
in the United Kingdom.
We are very, very excited about this.
And now
let's see,
let's just step right into the data center.
Today, I'm excited to announce
the general availability of Azure Boost.
You know,
it's fantastic to see this new system that offloads
server virtualization processes
onto purpose built software and hardware.
This enables massive improvements in networking,
remote storage, and local storage throughput,
making Azure the best cloud
for high performance workloads
while strengthening security as well.
Now let's go inside our servers.
We're tapping into the innovation
across the industry, including from our partners
AMD and Intel, and making that available to you.
For example, organizations like Vestas use
AMD on high-end compute and memory optimized service
in Azure to run
simulations on massive amounts of weather data.
And the largest SAP database deployments are powered
by our new Mvv3 virtual machines
running the fourth generation of Intel
Xeon scalable processors
supporting up to 32 terabytes of memory.
In fact, Intel is putting their own SAP instances
on these machines and it's great to see.
You know, as a hyperscaler, we see workloads,
we learn from them and then get this opportunity
as a systems company
to optimize the entirety of the stack
from the energy draw to the silicon
to maximizing performance and efficiency.
And it's really thanks to this feedback cycle
that I'm thrilled to introduce
our very first custom in-house CPU series,
Azure Cobalt, starting with Cobalt 100.
Cobalt is the first CPU
designed by us specifically for the Microsoft cloud
and this 64 bit 128 core ARM-based chip
is the fastest of any cloud provider.
And it's already powering parts of Microsoft Teams,
Azure communication services as well as Azure SQL
as we speak.
And next year we will make this available
to customers.
Yeah.
And when it comes to AI accelerators,
we're also partnering broadly across the industry
to make Azure the best cloud,
no questions asked for both training and inference.
It starts
with our very deep partnership with NVIDIA.
We built the most powerful AI supercomputing
infrastructure in the cloud using NVIDIA GPUs.
And OpenAI has used this infrastructure
to deliver the leading LLMs as we speak.
In fact, last week
Azure was the largest submission to ML Perf
Benchmarking Consortium with 10,000 H100
GPUs, three times more than the previous record,
delivering better performance than any other cloud.
And in the latest top 500 list of world supercomputers,
Azure was the most powerful supercomputer
in the public cloud and third, all up.
That made news.
What didn't make news is
we didn't submit the entirety of our supercomputer.
We submitted only a fraction of our supercomputer.
So I'm thrilled to be number three with that, as we
and by the way,
that's the only one that made the list
as a public cloud.
And as we build supercomputers
to train these leading large models, InfiniBand
gives us a unique advantage.
And today we're even going further.
We're adding NVIDIA's latest GPU
AI Accelerator, H200, to our fleet to support
even larger model instancing with the same latency,
which is so important
as these models become much bigger and more powerful
the ability for us
to have these new generation of accelerators
is a big deal.
We are also introducing the first preview
of Azure Confidential GPU VMs,
as you can run your
AI models on sensitive datasets on our cloud,
we codesigned this with NVIDIA,
so if you're doing
what is referred to as retrieval
augmented generation, or RAG,
you'll hear a lot about this
throughout the conference
running on this confidential GPU VM,
you can enrich, for example,
your prompt with very query place-specific knowledge
from proprietary databases, document archives
while keeping the entirety of the process
protected end to end.
And so it's very exciting to see us
not just lead with GPUs,
but lead with GPUs with confidential computing.
Now let's talk about AMD.
I'm excited to announce that AMD's flagship MI300X
AI accelerator is coming to Azure
to give us even more choice for AI optimized VMs.
With 192 gigabytes of
high bandwidth memory and 5.2 terabytes
per second of bandwidth, the MI300X offers
industry leading memory, speed and capacity.
Again, this means we can serve
large models faster using fewer GPUs.
We've already got GPT-4 up and running on MI300X
and today
we are offering early access to select customers.
And we're not stopping there.
We are committed to taking the entirety
of our knowhow from across the system
and bringing you the best innovation
from our partners and us.
Today we are announcing
our first fully custom in-house
AI accelerator Azure Maia.
Starting with Maia 100 design to running cloud
AI workloads like LLM training and inference,
this chip is manufactured on a five nanometer
process, has 105 billion transistors, making it
one of the largest chips
that can be made with current technology.
And it goes beyond the chip, though.
You know, we have designed Maia 100 as an end-to-end
rack for AI, as you can see right here.
AI powered demands required infrastructure
that is dramatically different from other clouds.
The compute workloads are, you know, require a lot
more cooling as well as their networking density.
And we have designed the cooling unit
known as the sidekick,
to match the thermal profile of the chip and added
rack-level closed loop
liquid cooling for higher efficiency.
This architecture allows us to take this rack
and put it into existing datacenter
infrastructure and facilities,
rather than building new ones.
And by the way,
they're also built
and manufactured to meet a zero-waste commitment.
So we are very, very excited about Maia.
With Maia, we are combining the state of the art
silicon packaging techniques, ultra-high
bandwidth networking design, modern cooling,
power management,
algorithmic co-design
of both the hardware and the software.
And we're already testing this with many of our own
AI services, including the GitHub Copilot,
and we will roll out Maia
accelerators across our fleet
supporting our own workloads first,
and we'll scale it to third party workloads after.
This silicon diversity is what allows us to power
the world's most powerful foundation models
and all of our
AI workloads from Copilot to your own
AI applications,
so that when I say systems,
this is the end to end innovation.
From glassblowing the next generation
fiber optic cables, sourcing renewable energy,
designing new approaches to thermal distribution,
innovating in silicon, our goal is to ensure
that the ultimate efficiency, performance and scale
is something that we can bring to you
from us and our partners.