One of the questions I ask the students in the classes that I teach is,
"What are you going to do with 100 times more compute?"
And sometimes that's a really hard question for them.
There's a lot of head scratching both in terms of what can we do with a super computer that's 100 times more powerful
and what can you do with something on your desk or in your pocket?
-Where do you see us going in this direction? -Yeah, well I I have an insatiable appetite for FLOPs.
I would have no trouble using 100 or even 1,000 or even 10,000 times more compute.
A lot of what I do is designing computers,
and a lot of that involves prototyping and simulation new computer designs.
I'm always frustrated about how those simulations run.
So of I could run RTL simulations of a new computer 100 times faster,
it would enable me to be much more productive in trying out new ideas for computer design.
Same for circuit simulations.
I spend a lot of time waiting for circuit simulation to converge.
If I could run it 100 times faster, I could not just run one simulation but run whole parameter sweeps at once
and do optimizations the same time I'm simulating.
Another thing is also you look at the computers in your car.
I mean our Tegra processors are actually designed into lots of different automobiles,
including the Tesla Model S, the Motor Trend Car of the Year,
but also Audis and BMWs and all sorts of Fords have Tegras in them.
And the applications people are starting to use for these mobile processors in cars
involve having lots of computer vision to look at what people inside the car are doing,
look at what people outside of the car are doing.
And in many ways it makes your cars much safer by having the car aware of what's going on around it.
It can in many ways compensate for the driver not being completely alert
or perhaps texting or doing something they shouldn't be doing.
And in mobile devices I think there are a lot of compelling applications
in both computational photography and augmented reality.
If your mobile device is constantly aware of what's around you, it can be informing you.
Oh, I think you're hungry.
Here's a place that has gyros that I know you like
because I have your profile of your likes and dislikes. Maybe you should stop for lunch.
Or a block away is this guy who you really don't like.
Maybe you should turn right at this corner and avoid running into him.
In many ways, I think it sort of evolved to having your computing devices becoming your personal assistant.
I always liked Jeeves in the Iron Man movies.
I would like to have a device I can kind of talk to that is aware
of the environment around me and can be basically a brain amplifier for me.
It can sort of remember things that I forget
and tell me about things in my environment and basically assist me in going through my day,
both on professional and personal bases.
So one of the goals of the supercomputer industry is to get up to—the term they use is exascale
that they'd like to do 10 ^ 18 FLOPs per second.
Certainly, Nvidia is going to be interested in being in those computers. What are we going to use that for?
Well, I think first of all, there's nothing magical about an exascale.
It's like, you know, when we first made
petascale machines, which is just a few years ago,
it wasn't like breaking the sound barrier or anything really qualitatively changed,
but enabled better science and there's always—
You look at sort of the fidelity of simulations we're able to do today
to, say, simulate a more efficient engine for automobiles to improve gas mileage,
and we're making lots of approximations to fit them on the supercomputers we have today.
As we can get to higher fidelity by resolving grids finer
and modeling a bunch of effects like turbulence more directly
rather than using macro models to model them, we'll get more accurate simulations.
And that will enable a better understanding of combustion in some of the you biotech applications
of how proteins fold, various other climate—
-Climate modeling. -Sure.
Climate evolves.
Basically as we get better computing capacity—
and it's not you're reaching magic exascale and wonderful things happen,
but at every step along the way, we get better science,
we are able to design better products.
And computing is a big driver of both scientific understanding
and economic progress across across the board.
And I think it's very important that we maintain that steady march forward,
and exascale is just one milestone along that march.
And my understanding is that power is really an enormously crucial thing for them to get right
to be able to enable the exascale that we don't want machines
that are going to cost $2 million a month just to plug in. -Right.
It's really an economic argument.
I mean if you really wanted an exascale machine today, you could build one.
You just have to write a really big check and locate it right next to the nuclear power plant,
the entire output of which it will consume.
But I think if there was some application that was so compelling
they were willing to really write the multi-billion dollar check required to do that, you would do it.
I think that the real question of exascale is an economical exascale,
and because on total cost of ownership the power bill is a tremendous fraction.
So it's not actually an economical exascale machine
unless you can do it for reasonable power level,
and the number that's been thrown out is 20 megawatts.
So that's $20 million a year.
Yeah, $20 million a year power bill if you're paying roughly $10 a kilowatt hour.
In fact, the bill actually winds up usually being a little bit higher than that
because the cost of provisioning energy amortized over, say, a 30-year lifetime of the facility,
usually is about equal to the annual bill for the energy.
There is also something called the PUE,
which is basically the efficiency of providing the energy.
Even for a very good installation today maybe on the order of 1.1 to 1.2.
So you pay another, say, 20% to run the air conditioners and fans and things like that in the facility,
and basically energy you're consuming isn't being consumed by the computer.
But it's a big challenge for us to get from, say, Sandy Bridge today—
that's 1.5 nanojoules per instruction—to if you wanted to do exa instructions per second,
to do an exa FLOP you might have to do more than an exa instruction per second.
But even if you take that as a thing at 20 megawatts,
that's 20 picojoules per instruction.
And that's not just the processor; that's everything.
That's the memory system, that's the network, that's the io storage system.
It's the whole ball of wax that to do it.
So you mainly get 10 picojoules per instruction to actually use in the processor.
And so even in Nvidia it's not quite close enough to that.
Yeah, well compared to Sandy Bridge, that's a factor of 150 down,
and process isn't going to help you much.
So that's why conventional CPUs are not going to get there.
It's going to require a hybrid multi-core approach
with most of the work being done in a GPU like throughput processor to get there.
But even we have a ways to go.
We're probably close to over-magnitude, and we might get a factor of three from process.
We need to be very clever to come up with the other factor of 3 or 4 that we need.
-Titan does have CPUs in it, yes? -That's correct.
So is there a vision where that won't even be the case?
No I think there are always pieces of the code
where you have a critical path, you have a piece of single thread code that you need to run very quickly.
And so you always need a latency optimized processor around to do that,
but most of the work it's one of these things, it's kind of like a cache memory, where most of your acesses
are to this little memory that runs really fast but you still need the capacity
of the big memory sitting behind it, right?
And so it's, it's the same thing on throughput versus latency.
Most of your work is done in the throughput processors,
but when you do have a latency critical thing, you run it in on the latency optimized processors.
And so you wind up getting the critical path performance of the CPU
with the bulk of the energy consumption of the GPU.
-And the bulk of the FLOPs and Titan is certainly going to the GPU's. -Right.
The bulk of the FLOPs will be in the GPU's.