Hi and welcome to the Arm “What Is” program series. In each episode, we explore a tech topic
to give you insight and perspective into some of today’s hottest design trends. I’m Brian Fuller,
editor-in-chief at Arm and today we’re going to find out what is heterogeneous compute?
And to help us with that, I want to introduce Erik Jacobson, marketing director, architecture,
and technology, with Arm. When he’s not helping us understand what is heterogeneous compute,
Erik spends time learning new things from his two fabulous daughters. So, let's dive right in.
So Erik, what exactly is heterogeneous compute?
It's really about using additional processing elements
above and beyond the classic one or multiple core CPUs, you know? Whether it's GPUs, NPUs,
smart NIC, various combinations of accelerators. And then, tweaking that combination
or designing combination that is optimized for various specific use-case. And on top of that,
making it programable. So instead of having accelerators you can't actually get to,
making it as a developer really easy to use. So it is about taking that system level approach, having
these different types of processing elements in there and making them accessible. It's shifting
from just a CPU or heavy CPU centric to this much more heterogenous approach.
Yeah, so you mentioned that the CPU use to be the superstar of any system
and now we're in a period of diversification. What's driven that?
So, the easiest answer to that is Moore's Law, right? Moore's Law is coming to an end.
But actually, there are other market forces that work here.
A big one for instance is verticalization where companies are taking more and more responsibility
up and down the stack. And the reason they're doing that is to get more control.
So, if you're building your own SoC then you're clearly going to optimize for the
workloads that you need to run. And that's a very different world from looking around
from what's on the markets and just buying off-the-shelf SoCs. So, there's that market trend.
And there's another market trend, which is just some of these software stacks
have so much investment in them that actually the days of customizing software to run on hardware
are probably gone. Now it's about software driving new functionality and then you design the hardware
afterwards because the software has many years of investments in it. So to run an existing software
stack, you need to maximize the performance of the hardware to run that on. So, it's not
just the matter of technology or the technological reason, it's about the marketing forces as well.
Can you talk a little bit more about designers can mix and match
various compute engines to optimize their systems?
The main thing is to pick the best tool for the job right? And the job is the workload
and by the workload I mean the process and data. What the system can be doing, you know?
What's it going to do? What are you optimizing for? Is the classic power, performance, cost,
you know? You might be running similar types of workloads, but in different environments. The
designer has to know about the workload and the constrains. So, the days of we'll create hardware
and have lots of different software running on it , you know? What heterogeneous compute does
is saying well actually there are a lot of really material software stacks out there.
We need to run it with the best performance or lowest power or cheapest components in this
location, you know? And so then the designer maybe has to model that to understand running through
those models, etc. What the best trade-offs are and design their systems appropriately.
There are various, that we have various specifications, like AMBA and SystemReady program,
which ensure functional compliance – that it works. So SystemReady, for instance, is about
an operating system working, just working out of the box. But the hardware designer
then has to go and say well actually I'm going to change these elements in the hardware. It's
still going to work, but I'm going to squeeze out that extra whatever of the performance, you know?
Is heterogeneous compute being applied
in a few application spaces or is it more broadly deployed than that?
CPU and GPU combination has been around a long time,
really. And that's pretty much a heterogeneous compute method. Although maybe accessing the
GPU isn't quite as easy or rather a different form of programming of it.
I think there's still a lot of , you know, we haven't come to the end of CPU centric at all,
you know. But I think we will see more and more heterogenous. I think where we're seeing it is
where there are constraints or where there's quite a lot of money in it. So, for instance, if you are
running a large server farm let's say. Obviously, there's a lot of
budgets to be saved in power. Whether it's from air conditioning or just the actual base load,
you know? Other areas, existing CPU systems are perfectly fine, you know? And there's no reason
to change them. But cost, power, performance – those three are perfectly fine for those areas.
Well, thanks Erik. It's stunning to see how innovation at the processor level is
taking the electronics world to new heights. Now check out all our other "What is" episodes
here and be sure to subscribe to this channel because we'll be adding more
as the year progresses. Thanks for listening!