Let's talk about high-performance computing. It is clear now that acceleration is going
to be the path forward for scientific and for high-performance computing. And as I mentioned
before, accelerated computing has four pillars. The first of course is the accelerator, the
advanced GPUs. The second is the stack, the acceleration stack for each one of the computational
domains. The third is systems and last is developers - ultimately, the applications
that we accelerate.
This year we did really great work. We've accelerated now over 700 applications. And
each and every single year, at every conference I show you our golden suite, the suite that
we track on a regular basis to make sure that we continue to engineer advances into libraries,
into the stack so that applications continue to improve in performance, even if we don't
introduce new GPUs. And as you see over the course of the last four years, we increased
application performance by 4X and the green bar is something that I'm going to talk to
you about. We're going to offer you a new platform and it's going to give high performance
computing a huge boost.
We also brought CUDA to ARM computing systems. ARM server CPUs are seeing adoption all over
the world. In hyperscale there's Amazon, Fujitsu, super computing KVM part of Marvell now a
new company, exciting company called Ampere Ccomputing or in China, Huawei. All of the
suite of NVIDIA tools and libraries are now available for ARM.
We also introduced a brand new SDK for IO processing this year we call Magnum IO.
Magnum IO includes all kinds of great things from RDMA, of course, to the ability to communicate
across multiple nodes and move data directly from storage to our GPUs. The suite of libraries
is going to continue to advance - Magnum IO is going to be one of our most important libraries.
Data processing and networking and storage
is going to become more and more important to data center scale computing over time.
We introduced two new stacks this year. NVIDIA Parabricks for genomics processing, the ability
to do variant calling at very high performances and a large body of work that we've been working
on for several years called NVIDIA RAPIDS for data analytics.
Machine learning has become one of HPCs grand challenges. The advances of machine learning
and the popularity of this approach has caused companies, institutes and data centers to
collect a vast amount of data. The machine learning pipeline consists of three things.
ETL, which creates the dataframe, does all the feature engineering necessary for the
machine learning algorithms to train on, which creates the model which is then put into operations
we call inference. These three stages of the pipeline have unique and different computational challenges.
The first stage of the machine learning pipeline, data processing, is becoming more complex
than ever. In fact, most data scientists will tell you they spend the vast majority of their
time doing feature engineering and data processing in the front stage of the machine learning pipeline.
What used to be processing hundreds of megabytes to gigabytes to terabytes of
data. Companies are now routinely processing tens if not hundreds of terabytes of data
and moving to petabytes of data. It is the reason why Spark is so popular. Spark is an
incredible computational platform. It turns an entire data center into a compute engine.
It partitions a very large dataset to be processed into small chunks, which is then distributed
across a bunch of servers in a data center, scheduled, then processed to completion. It was
the brainchild of Matei Zaharia at the Berkeley AMPLab and spun out and became Apache Spark.
It now has over a thousand companies contributing to it, nearly a million lines
of code. It is giganticly complex and incredibly powerful. Which is why 16,000+ companies around
the world use it for data processing today.
While the amount of data that they're processing is growing exponentially, it is now reaching
the limits of what Spark can do. Here's the reason why.
The CPU that it's being distributed across has a fundamental working set in the
order of megabytes. A CPU naturally likes to work in its cache and it caches typically
on the tens of megabytes. When the dataset is now
the hundreds of terabytes and into petabytes.
The overhead of coordinating the CPU servers is becoming the greatest bottleneck.
And, we're starting to see the limits now. What if instead of working on processors that
has tens of megabytes of working set, let's move towards
a processor that has tens of gigabytes of working set and if we could use multiple GPUs
to create large memories, then it is now possible for us to imagine scaling beyond that.
We started working on GPU acceleration of the data
processing stack several years ago and it's a giant body of work.
Ladies and gentlemen, today we're announcing that Spark 3.0, the next generation of Spark,
will be NVIDIA accelerated. This is a collaboration
between ourselves and a large community of researchers and developers in open source
all around the world and the results are really fantastic. It's possible because of several
groundbreaking achievements.
The first is the work that we did with Mellanox, NVIDIA called GPUDirect storage and the
acceleration of GPUDirect storage and UCX, this framework that makes possible the
management of IO and storage, and multi-node computing lightning fast.
Second is the scheduler of Spark. Scheduler of Spark now is aware of GPU and the GPU memory
so that it could partition work to the GPUs and schedule it in a distributed way and manage
the computation of this giant network of computers.
Third, a library called RAPIDS that has the ability to ingest data, create dataframes,
do feature engineering, do SQL queries and intercept
the calls of Spark to be accelerated by our GPU.
And then lastly, Spark has a Spark SQL accelerator they call Catalyst, and that has been optimized
for NVIDIA GPUs.
These elements make possible Spark 3.0.
Let me show you the potential acceleration that data scientists will be able to enjoy.
What you're looking at here is the benchmark of RAPIDS,
the foundation of Spark 3.0. This particular benchmark is TPCx-BB big data benchmark. This
particular data set is a scale factor of 10,000 which basically is 10 terabytes.
The state of the art is a Dell server, cost about a million dollars, and it has the ability
to deliver 17 gigabytes per second of data movement through
this benchmark. This particular benchmark is
hard to beat and the reason for that is because not only does it have to be fast, it also
has to be cost effective and the reason for that is
because price performance matters. The fastest in the
world today is the Dell server at a million dollars and 17 gigabytes per second.
With Spark 3.0 sitting on top RAPIDS. RAPIDS benchmarked
on TPCx-BB delivers 163 gigabytes per second for $2 million, 10 times the performance and
only twice the cost. If you were to look at this in
another way, suppose you were to create a data center that is able to achieve the same
performance as $2 million of DGXs accelerating RAPIDS of this benchmark TPCx-BB. It would
cost you $10 million and 150 kilowatts. Now, of course, data centers routinely process
a lot more than terabytes.
You're going to need data centers way larger than this in the future. As data continues
to grow exponentially, and so the ability to accelerate
Spark 3.0 with a library called RAPIDS is utterly
groundbreaking. The result is really spectacular. One fifth the cost, one third the power,
one fifth the cost and one third to power.
The more you buy, the more you save.
In fact, Databricks, which offers industrial strength Spark at a large scale as a service, is doing
fantastic every single day. A million virtual machines
are spun up to do data processing on Spark and
they're so delighted by the work and acceleration that they're going to go accelerate Databricks
with NVIDIA GPUs, they're a fantastic partner. I'm so happy with all the work that we've
done together.
Leading cloud service providers are offering Spark accelerated in their cloud or they're
accelerating their proprietary machine learning pipeline and data processing pipeline with
NVIDIA RAPIDS. Amazon Sagemaker. Azure Machine Learning, Databricks, Google Cloud AI,
Google Cloud Dataproc are now going to be accelerated with NVIDIA GPUs for data processing
and data analytics.
Spark acceleration is a great achievement. I'm so proud of the team. It's such a large
body of work and has taken us years and it requires
the collaboration with hundreds of collaborators and
open source built on several layers of foundational and fundamental new technology.
Now the part that is growing exponentially difficult. The first stage of machine learning is now
accelerated. Data scientists all over the world are going to be thrilled. Entire end-to-end,
from data processing to inference, we have three libraries. RAPIDS for data processing,
cuDNN, our core library for deep learning, AI. And then third TensorRT, our optimizing
compiler for these computational graphs that are created by the
training frameworks. The end-to-end acceleration is
now complete and we will continue to advance it over time.
But this represents the foundation of NVIDIA AI. I can't be more proud.
The team have done a great job. Thank you.