Welcome. My name is Werner Vogels, I'm the Chief Technology
Officer of Amazon.com. Right now, many scientists and researchers
and professionals in the high performance computing world are at Supercomputing 2010
in New Orleans. Since I can't be there in person, I wanted
to tell you a bit about the latest developments in HPC on Amazon Web Services.
The cloud is truly revolutionizing all business and industries, including HPC.
During my academic career I spent many years working on HPC technologies, such as user
level networking, large scale high performance interconnects, HPC software stacks, etc.
In those days, my main goal was to take the advances in building highly dedicated cluster
environments and turn them into commodity technologies for the enterprise.
Not just for HPC, but for mission critical enterprise systems such as OLTP as well.
Today, with the accessibility of cloud services from Amazon Web Services, we are making high
performance computing available as an on demand commodity for anyone to use.
It is no longer a technology for the elite few with large budgets.
After we launched Amazon Elastic Compute Cloud (EC2) in 2006 our customers immediately started
using it for high performance computing, because of its cost effective, on demand compute access.
Early users included Wall Street firms and hedge funds, academic researchers, pharmaceutical
companies, digital media producers, and others. To respond to this growing customer segment
and their specific needs, we launched cluster compute instances which are specifically engineered
to provide high performance compute and networking. Continuing our tradition of listening closely
to our customers, I'm excited to share with you a new HPC offering we released recently:
the cluster GPU instance. This new instance type extends our cluster
family by adding Nvidia Tesla cards to cluster compute instances.
We believe this will significantly lower the bar for customers who are interested in developing
GPU applications. At Amazon Web Services, we are continually
excited about the developments coming out of the HPC space, many coming from AWS customers
such as NASA Jet Propulsion Labs, Eli Lilly, Harvard Medical School, and Berkeley RAD Lab.
We are equally excited about enabling these developments for new services and features.
Stay tuned, though, it is still early in this technology revolution.
And now, let me hand it over to Deepak Singh for an in depth look at HPC on Amazon Web
Services. Hi, my name is Deepak Singh. I'm a senior
business development manager for Amazon EC2 and a cloud computing strategist at Amazon
Web Services. I work with the largest customers in the areas
of scientific computing, research computing, and large scale business analytics.
You just heard Dr. Werner Vogels, our CTO, talk to you about HPC and Amazon Web Services.
I wanted to spend the next few minutes talking to you about our HPC offerings and telling
you about our partners at Supercomputing 10. For many years, AWS customers have been using
Amazon EC2 to solve a variety of computationally challenging problems.
These have typically been embarrassingly parallel in nature, batch processing workflows, or
data intensive workloads using the Hadoop MapReduce framework, often using Amazon's
Elastic MapReduce. However, many of our customers also expressed
interest to run tightly coupled, I/O intensive workloads on Amazon EC2.
Given the distributed nature of the architecture, this was often challenging.
Earlier this year we launched the cluster compute instance.
The cluster compute instance combines the elasticity, scalability, and on demand nature
of EC2 with the high performance network that allows customers to solve challenging I/O
intensive workloads. What is the cluster compute instance?
The cluster compute instance—or cc1—has two Xeon X5570 processors, which gives you
a total of eight cores. You have hyperthreading, so you have access
to 16 compute threads. 23 gigs of 1333 GHz DDR3 memory.
Almost two terabytes of local disk space. And all of this on a 10 gig ethernet fabric.
We also introduced the concept of a placement group.
All instances launched within a single placement group have full bisection bandwidth between
all nodes in that group, essentially giving you a cluster.
You can connect multiple placement groups to form very large clusters, such as the 880
node cluster we used earlier this year for an HBL run, which gave us 41.82 teraFLOPS
of performance. For the last few years, GPUs have taken the
HPC world by storm. At Supercomputing 10, you will hear a lot
about GPUs and how they are the future of HPC.
Many of our customers have expressed the interest in running GPUs in the cloud, often due to
the challenges of running GPUs in your own facility.
They are blocked either by cost, by power requirements, or just the uncertainty of running
large scale clusters of GPUs when they are not sure if their applications are well suited
to the GPU architecture. I'm pleased to talk to you about our most
recent offering. The cluster GPU instance was launched last
week and adds to the cluster compute instance I talked about earlier.
In fact, it is exactly the same instance, on the same network fabric, except that on
each instance you now have access to two Nvidia Tesla M2050 GPUs.
Researchers doing molecular dynamics, rendering, CFD, and a variety of other computational
and technical tasks can take advantage of the cluster GPU instance and get performance
several times faster than they might get on CPUs.
You can learn more about the cluster GPU family by visiting Nvidia and asking them a little
bit more about their Tesla card as well as how you can provision GPUs at Amazon Web Services,
you might actually even get to see a demo. We have been working with a large number of
partners who have enabled our customers to get onboarded to the cloud much faster.
All of you have heard of MATLAB. MathWorks has been a long time partner of
Amazon Web Services and has been supporting cluster compute instances from the day they
were available. You can visit the MATLAB booth at SC10 and
get a demo on how you can use Amazon Web Services and Amazon EC2, including the cluster compute
family. You can also visit Cycle Computing, Clustercorp,
and Platform Computing to learn how you can provision clusters on Amazon Web Services
using the cluster compute instance. We look forward to your feedback on the kinds
of workloads you might want to run on the cluster family and other HPC needs that you
might have. We encourage you to visit aws.amazon.com/sc10
where you will find this video and learn more about our partners and other activities at
Supercomputing. In fact, one activity I would welcome all
of you to visit is a talk at the Intel theatre on the 16th of November at 1:30 PM central
time. The cluster compute instance is built on the
Intel platform. In that talk, you'll learn a little bit more
about the architecture of the cluster compute instance, our partners and customers like
NASA JPL who use the cluster compute family to solve complex image processing tasks.
You can also hear a live webcast at 11 AM central time on the 16th where I will be talking
to you in a lot more detail about the cluster compute family and HPC on Amazon Web Services.
We hope that you are able to use these instances. Give us feedback on what else you would like
to use. Thank you very much.