Welcome back to the oneAPI channel.
We have talked a lot about heterogenous systems We have a CPU usually called the host
And an accelerator, or several of them.. a GPU, an FPGA, or other, or many of each..
in one system.
Let’s assume that every piece of code can run on the CPU, this is my baseline,
Now, we know that specific pieces of code can run faster or more efficiently on other
devices,
But, few questions I have in mind..
1) how to identify these pieces of code?
How to estimate that this function will run faster on a specific accelerator and by how
much
2.
Does it worth it to offload this piece of code to an accelerator,
Considering the overhead to transfer the data to and back from that accelerator.
And, I would like to know it all upfront, I do not want to purchase an expensive GPU
and only then understand that it’s not good enough
Well, Intel advisor is doing exactly that
In previous videos we have talked about the oneAPI components.
We mentioned the multi-device multi-vendor open spec libraries and tools
That can now work not only on CPU but on multiple devices and vendors
And are packed into toolkits,
The base toolkit for example.
Just google “Base toolkit”, or watch the video on the base toolkit.
Intel Advisor is a great tool, and part of the Base toolkit
With lots of options to analyze your code..
Check out the video on the roofline analysis for example..
Intel advisor could be installed stand alone or part of the base toolkit
And could be invoked stand alone or from your IDE..
The offload advisor, can analyze your code, It will run the baseline, on your current
hardware And using heuristics will estimate the performance
of each of the portions of the code on a different target devices
It understands the data traffic and transfer times, the memory capacity and more and factors
it all in.
So it can help you offload portions of your code from CPU to GPU
Or from one GPU to another one….
This for example is the summary dashboard
And here, for example , my code has 3 regions, or loops that could be offloaded to an accelerator.
You can see the 3 of them here bellow
These 3 loops include 75 % of the overall code
And they could be improved by 50%
And improve the overall application performance by 10 %
Lot’s more information, for example, you can see here, that the memory bandwidth is
what mainly limiting the performance
Let’s see a quick example, I’ll run it on my laptop which has a CPU and an integrated
GPU The only installation I have is the oneAPI
Base toolkit.
I just follow the get-started-guide I’ll leave you the link,
First stage is to get some code, it could be any C++ or SYCL code.
They recommend the vector-add sample code
Let me show you an elegant way to get the code.
Just open Visual-Studio, with no project, no code.
Extensions, Intel Browse oneAPI samples
You can find the vector-add here or under tools/advisor
And we have the code.
This is already SYCL code And it will be using the default selector,
so my integrated GPU I’ll change it to CPU_Selector
And I’ll add some instrumentational code, to measure the time for the whole program
and for the kernel
And running on my CPU, 60 sec, most of the time spent on the kernel
And running on the GPU, Significantly faster..
The exact numbers are less important, But I do have high hopes to accelerate that
code, with zero effort..
Just by offloading portions of it to my GPU.
Which anyway sits idle..
I’m on Windows, I’ll open a oneAPI command prompt, another
advantage of installing the base toolkit oneAPI initializes
And now, compile the application And execute it on a big buffer
And let’s open the advisor GUI Choose the offline modeling
I can choose many different Intel GPUs as my target platform
I’ll just choose Gen 9 GT2.
My baseline is my i7 CPU And the accelerator is Gen 9 GPU
And I have the results..
I can speed up by 51 % some portions of the code.
But I can accelerate only 1 % of the code.
So the overall improvement will be .7 %
This loop, for example, could be speed up by 67x
It is mainly bounded by the level 3 cache bandwidth
Let’s look at the accelerated regions, 3 potential regions
Let’s look at this one Open the code..
and yes..
This is the main kernel code… which makes a lot of sense to offload to my GPU..
Very useful !
Intel Offload advisor, Find code bottlenecks and functions to offload
to an accelerator A great tool on the way to heterogenous computing.
Subscribe, send me a question or feedback.