Welcome back
Choices are great, we all prefer not to be locked in into one option,
But too many choices are sometimes too much. In the near future, you will need to program
a multi-accelerator system A system that will have a CPU,
a GPU, an AI accelerator, maybe a graph accelerator and maybe more..
All in one system.. Let me show you how
to chose the right device for execution on a system with multiple accelerators..
The easiest way for me to work,
I have installed the Intel oneAPI Base toolkit, just search online for the base toolkit.
And I’ve added the Codeplay plugins for Nvidia GPUs, check my other video on that..
I’ll leave you all the links and code snippets for your use..
Initialize oneAPI,,, Source intel/oneapi/set-vars…
And now type sycl-ls.. And I can see all the devices in the system, very handy
As you can see I have here The FPGA emulator,
It basically runs on the CPU and comes with the base toolkit installation
An i9 CPU
A discrete GPU by Intel, actually you see the same GPU, you can see the number,, 56A0 twice
Because you can access it using the OpenCL backend, on top
Or with a level-zero backend
And a GPU by Nvidia with CUDA backend..
And the question is, how can you choose each of these devices for execution…
So there are multiple ways to do it, I’ll show you few options and you could choose
your preferred one.. But before we begin,
I’d like to show you this little utility code, I’ll leave you the link to all the code snippets
I’m using and this is running on VS-code, if you install the oneAPI base toolkit,
and the oneAPI add-ons, VS-code is configured to oneAPI automatically. Check out my video on that..
So I’m getting all the platforms in my system
(think about it as the driver…) For each of the platforms,
I’ll print the name of the platform And for each of the devices I’ll print
the name of the device and the vendor’ Let’s compile the code and run it.
And you can see all the platforms, all the device names and their vendor..
And of course, I can type here any attribute of each of the devices..
And this is the simplest code I could create just to choose a device and run something on it..
I’m using a default-device selector and assign it a Q
I have data, using malloc-shared so it could be seen by the CPU host and the device
I initialize the data And on the device, I’m just
multiply the data values. That’s all.
Now let’s compile the code.. And run it..
You can see that the default-selector chose my Intel GPU as the default accelerator
Now let’s run this program again.. This is the list of my devices..
If I just run the code, the default selector assign my Intel GPU..
One simple way to change it, is by running the same code, and just changing the default selector
So let’s assign the default SYCL_DEVICE_FILTER to be my first device.. And re-run
And yes, the accelerator was chosen as my FPGA emulation device..
And now let’s change the default to my second device..
And my CPU is used as the accelerator. And so on..
You can use this method to choose any device.. But If you want to run on the nVidia GPU,
you will need to re-compile the code according to Codeplay plugins instructions..
And this is the first way to choose a device
The 2nd way is to create a custom selector, I’ll leave you that code.
Here instead of the default selector, I’m using my-selector
And my-selector is looking for a device which is a GPU
and the vendor is nVidia,
Compile the code and run, and the device chosen is my nVidia GPU.
and you can just build any device you like,
you can prioritize devices, for example choose a GPU, is there are none, chose a CPU and so on…
And here is the last method to choose a device.. I’ll remove the default-selector
I have all the platforms and I’ll choose platform 0
This is again the list of my platforms. I chose platform 0 which is my FPGA emulator..
Compile and run and it’s running on my first platform, the FPGA emulator..
Let’s Change to platform 1, compile and run
And it’s running on my CPU Change to platform 4
Compile to nVidia and it’s running on the nvidia GPU…
Three methods to choose which, out of all the devices in your system is the device to
run the code on. Subscribe enjoy