Welcome back to the oneAPI channel.
Optimizing code is not easy.
Optimizing code across a heterogeneous system with multiple devices is even more complicated.
Sometimes the problem is that in order to fine tune and optimize the code to my CPU
or GPU I need an intimate knowledge of the hardware,
the memory system and this is an overhead to add to my SW portability
Lucky for us, Intel advisor makes it all easier.
It can analyze your code, and present you with the dependency flow graphs
A great way to explore dependencies and understand the flow
It can help you identify the best candidates for parallelizing,
prototype threading and check, if there are data dependencies preventing parallelizing
Modern compilers knows how to vectorize, but sometimes they take the safer side
and sometimes with small re-write of the code they can do much more.
Intel advisor can help you Improve your application performance,
With actual recommendations for how to fix vectorization issues
In the next video I’ll show you the offload advisor
This is a powerful tool that can recommend which functions to offload from the CPU to
a GPU or other device and even project the performance improvement.
Here for example, I can choose the GPU to offload to
And even play with the HW parameters..
And I’m getting the full analysis, This code could run 580 times faster than
my CPU for example..
Look for that video
And last is the roofline analysis.
It’s pretty cool and I’d like to show you how it works.
your code has many functions, each has a different arithmetic intensity.
A function that add B and C into A, is less complex and takes less operations than calculating
the cosine of B and multiplying B by C and D.
This is measured by the number of operations per byte of data.
The X axis
Each of these functions has compute intensity, but how fast can they run?
And these could be measure by Flops per second, the Y axis
Now the arithmetic intensity is (theoretically at least) fixed, so cannot move right or left
But can run faster or slower, depending on the SW implementation or the hardware it uses
So can move up and down on the graph
Our HW has its limitation, for exempla this red line is the maximum Flops per second this
CPU can execute But sometimes the problem is not the compute
Our memory bandwidth can also limit us.
As you can see the other red line..
And these two form the roofline..
Which represents the maximum performance our machine can supply.
So theoretically each of these functions could be improved all the way up
But this function is hardly used, and improving it won’t help the overall program performance
much So the Advisor color it green.
And this function is already optimized so optimizing further is a big effort
So it is colored yellow
And this function is colored red… as the Advisor advises us to invest our time here.
The truth is that the representation of the HW is a little more complex
As you can see here a different roofline for vector operations and for scalar ones..
And also each of the memory hierarchies imposes a different limitation.
For example the level 1 cache is faster than the level 2
but Level 2 has a bigger capacity.
Intel advisor is not new, but now with oneAPI, it supports more devices, like Intel GPUs..
There are many ways to get and use Intel advisor..
You can get the advisor stand-alone, I’ll leave you the link
or you can get it as part of the oneAPI Base toolkit,
What I like in this option is that installing the base kit already set up your IDE
And all you have to do is run the advisor Choose the analysis you want, run, and explore
the results
And a very interesting option is to run it on the intel dev-cloud.
Where you can explore multiple CPU (or GPU) options…
So connect to the dev-cloud, ssh dev-cloud..
I’ll run a really nice lab..
I’ll leave you the link.. get the lab, you can clone it as well of course
Search for “devcloud get-started”..
I usually just open jupyter lab
SYCL performance portability And running the lab step by step
Basically running advisor to create rooflines for multiple devices,
In this case, multiple GPU options, and you can see my main function on these 3 HW options.
Pretty cool
Intel advisor, very powerful, and simple enough tool for analyzing your code performance.
In the next video, I’ll show you the offload advisor,
that can advice you on which portions of the code to offload to other device,
say GPU, and what will be the expected speedup..
See you there