[MUSIC PLAYING]
Hello, and welcome to "Tech Decoded."
I'm Henry Gabb, editor of The Parallel Universe,
Intel's quarterly magazine for software innovation.
Today I'm here with Geoff Lowney a senior fellow
in the Intel Architecture, Graphics and Software Group.
Today, Geoff is here to talk about a new programming
language called Data Parallel C++.
Welcome, Geoff.
Glad to be here, Henry.
So, Geoff, Intel just released Data Parallel C++,
a new language for heterogeneous parallelism.
Why are we going with a new language?
Well, you have to first think that data parallelism, or data
parallel compute, is the dominant compute pattern
of now, and of the next maybe 10 years.
All of the new areas--
AI, machine learning, video processing-- these
are all data parallel computation.
And we need a language that will make it easy for a programmer
to express data parallelism.
So we're starting, though, with C++, and we're extending it.
So Data Parallel C++ programs are all C++ programs.
So in some sense, it's not a new language.
But we're adding these new ways to express parallelism.
And if you build the language with a Data Parallel C++
compiler, then the data parallel regions can be offloaded
to an accelerator and run in parallel.
Can you describe the parallel programming model?
So the basic data parallel paradigm
is that you have millions or billions of data items
in your data set.
And you want to do similar--
or maybe identical-- computations on each of them.
So this language, Data Parallel C++, is set up for that,
for bringing data in and doing similar computations across
very, very many pieces of data.
Does Data Parallel C++ require me to have separate hosting
kernel code?
Do I need to write and maintain a separate kernel for CPU, GPU,
FPGA, and whatever architecture is coming?
Well, it's sort of two issues here.
One, the nice thing about being based on C++ is that you can
structure your program the way you'd structure a normal C++
program.
You can put everything in one big file.
You can split it up.
But there's no requirements based on the language.
Now, targeting different accelerators,
our philosophy is that you're expressing a data parallel
algorithm.
The expression of the algorithm should
be roughly portable across the accelerators.
However, the different accelerators
have different degrees of data parallelism.
For example, a high-end Xeon CPU has, say,
28 cores with SIMD units.
Whereas a high-end GPU would have 100 or maybe
1,000 cores with SIMD units.
So when you're targeting a discrete GPU,
you need to express or you need to say
that you want more parallelism, more things running
in parallel, than you do on the CPU.
So you need to tune or parameterize the parallelism
differently.
But the basic expression of the kernel should be the same.
And I don't think you'll need end copies of the program.
So why didn't we just go with OpenCL Or even
CUDA for heterogeneous parallelism?
Well, OpenCL is great technology.
But if you've ever written an OpenCL program,
you know that it requires a pretty low level detail
programming, and it's not very productive.
So we wanted to bring the benefits of C++ productivity
to this area.
And in fact the Khronos committee--
that's the group that has developed OpenCL--
has made the same decision.
And they've introduced a language called SYCL,
which brings C++ to the OpenCL space.
And our language is based on SYCL
so that we are working, basically,
with the OpenCL committee on how to move forward.
Now, as for CUDA, CUDA is a proprietary language.
And it's really only focused on programming GPUs.
So it's not a good language for cross accelerator,
and it's not a good language for cross industry.
So it really wasn't something that we pursued.
OK, so Data Parallel C++ is built on C++ 11,
which is modern C++.
How much does Data Parallel C++ require using those modern
constructs?
Well, we do use some modern C++ for launching a kernel.
But that's only one part of your program,
and actually only a few lines of code.
So there is a small part of a program that will have to use
these newer features of C++.
But for the rest of the program, including the kernel or all
the host code you have, you can write it in C,
or traditional C++, or use modern features.
One of the nice features of what we're doing is that it lets you
use these high productivity features,
like template functions, in the data parallel kernel so that
you can write with C++ in the data parallel kernels if you
want to.
But if you're a C programmer, like maybe you are--
Mhm.
--you can stay with C.
OK.
So I don't really have to update my legacy C++ programs that
much?
You don't really need to change the language.
If you want the program, though, to run in parallel,
you have to rethink it and make sure
that the inner loops can be expressed in this data parallel
fashion.
And that's the challenge, actually,
of bringing any program to an accelerator
or to a parallel environment.
It's really not the language itself.
It's how you think through the problem and make it parallel.
And so, based on that, I can--
theoretically or in reality, can I combine Data Parallel C++,
OpenMP, Parallel Standard Template Library,
and maybe Threading Building Blocks,
all in the same source code?
Yes, you could use all of those technologies.
And both on host side, as we were discussing before,
Data Parallel C++ is really C++.
So we provide a standard C++ environment.
So all of the technologies that run in C++ will work there--
TBB, OpenMP, and so forth.
Now, on the accelerator side, we're
basically defining a standard for how to launch a kernel
and run it in parallel on an accelerator.
So Data Parallel C++ is our major way of doing that.
But for example, for Parallel STL,
we're defining a new execution policy
which will let Parallel STL algorithms run
on an accelerator.
And for OpenMP, at least for Intel,
we're taking our OpenMP offload and mapping it
through the same compiler and runtime.
So you can, on the accelerator, use all of these technologies
at the same time, as well.
So DP C++ is an open, evolving language.
What features do you see coming in the next few years?
Well, the first feature we're adding--
and we're doing this today--
is unified shared memory.
So a lot of modern accelerators, the host
will have its own memory, and the accelerator
will have its own memory.
And it can be a real struggle to manage that data.
So we want a common address space across those two
distinct memories, and the ability for you
to program with pointers and have a pointer to an address
mean the same thing, whether you're on the host
or on the accelerator.
So this means you can use pointer-based data structures
and move them to the data parallel kernels or not,
and it makes programming much simpler.
So that's the basic thing we're doing as a productivity
feature.
Now, looking forward, we're going
to evolve the language as hardware evolves.
Now, we're in an age where there's a lot of change now.
That's why we're doing OneAPI, actually.
So for example, neural net accelerators, the core
to a lot of AI, those are becoming much more
programmable.
And they're actually in hardware putting capabilities
of doing convolutions and other linear algebra expressions.
So in the language, we will add a new data type, a tensor,
which is really a high-dimensional array,
and provide high level operations
to operate in those tensors directly.
And then the compiler will map it to the hardware.
So that's one direction.
And then for spatial architectures,
like an FPGA or some other ideas that we've been thinking of,
you really want an idea of a channel or a pipe
so that you can write systolic operations, which
can communicate.
And so that's another direction that
will be going in the next year or two.
So with unified shared memory, I don't
have to worry about moving the data back and forth
between the host and the accelerator?
You don't need to if you don't want to.
So this now becomes a performance issue.
So you bring the program up.
Data can migrate.
But then the key data transfers--
and this is perhaps the most important performance issue--
you need to plan those out ahead.
But that's a problem whether you have this--
pointers or copying or whatever.
In parallel computing or high-performance computing,
making sure the data is ready when the computer's ready
is the hardest problem.
And you're going to have to address that anyway.
Because I can remember that data sloshing,
which is [INAUDIBLE] performance if I wasn't careful about when
and how I move data between the host--
Right.
--and the accelerator.
So DP C++ is an open, I guess, industry standard language.
What contributions would you like to see from industry?
So we're looking for industry to work with us,
and to be equal partners.
We're doing everything here in the open.
We're doing the specification in the open,
and we're doing our implementation in the open.
And we have a website where people can contribute.
We have one or two industry contributors already working
with us.
We're also working with the SYCL language committee, which
is part of the Khronos group, the group that brought OpenCL.
So we're working with them and with some other external people
to try to bring all these ideas together.
So going forward, as the project gains momentum,
I think we'll get more and more industry collaborators.
Do you think the data parallel features will ever become part
of standard C++, which would render DP C++ obsolete?
Well, let me answer that in two stages.
So I do believe that data parallelism will move into C++.
But C++ is a very big language and a relatively old language.
It was developed in the early '80s,
and has been extended since then.
It has a lot of different constituencies,
a lot of different groups that want features.
So it's hard for it to move very fast
because it needs to get consensus,
and everyone buys in.
So data parallelism will appear, but it will probably
take five to 10 years for the features we're showing now
to end up in the language.
So if you roll the clock ahead five or 10 years, yes,
we'll have today's features in the C++ standard.
But hardware will have evolved.
We'll need new language features to support that hardware.
So we'll be in the same position we are today.
We'll have some advanced features
that we want to get into the standard.
That's right, you and I, we live in a parallel computing world.
So we forget there are other people using C++ who have
different priorities.
That's certainly true.
OK, well, before I let you go, do
you have any quick tips for developers?
Well, this is a pretty exciting time
I think to be a developer in this data parallel space.
There's a lot of work in algorithms going on, and AI,
and so forth.
And the hardware is changing pretty rapidly.
And we have this programmer in the middle.
Now, you'd like to say that the magic compiler can do it all,
but compilers can only do relatively simple things.
And we still need the programmer to make
this mapping between the algorithm and the hardware.
So it's a pretty exciting challenge.
And I'm thinking with the language we're developing,
we'll make that easy, and it will let programmers shine.
And so I think it'll be fun, and happy hacking.
Thank you, Geoff, for sharing your insights today.
It's been a lot of fun, Henry.
Thank you.
I'm Henry Gabb with "Tech Decoded."
Please visit our website for more information about Data
Parallel C++ and OneAPI.