You have a systems with 4 or more Intel Max GPUs, or 8 Gaudi 2 ​
Now with the new Intel developer cloud, it is easy to get your hands on such a complex systems​
Do you really have 4 GPUs? How are they connected?​
Do you know if they are all running Or maybe you are using only one GPU
And what is
the utilization of each of the devices ​ Is the bottleneck the memory or the compute ​
We need eyes, we need probes into the system…​ Let me show you the XPU-Manager
The XPU manager is currently mostly used for Intel GPUs but will be expanded to more devices.​
It is free and open source, it has many capabilities, let me just show you.
Search for the XPU manager​ It's right here​
The overview slides are very informative​ You can get the XPU-manager in a docker container​
I'll just go to git-hub​
You can see here the features, ​ And few screen shots​
Pay attentions that there are multiple possible solutions..​
The XPU manager is the full suite, you can also see here the XPU-SMI tools and others.​
I will focus on the XPU manager​ You can get the binaries here, in releases​
And here is the installation guide​ Basically you will need the GPU driver,
the level zero and others.​ The simplest way to go, is to just
get the oneAPI base toolkit, it has it all.​ Let's get the latest package from git-hub,
it is open source​ you can see the xpu-smi installation..​
I will use the xpu-manager for ubuntu 22​ Let's get the package​
And install it​ And we are all set
There is a ready good user guide, I'll leave you the link​
Let me just demonstrate few basics operations with the XPU manager..
I have a 4 GPU system from the Intel developer cloud​
You can see here the Xeon CPU and the 4 GPUs.​ Let's start with discovery..​
Yes, you can see my 4 GPUs, ​ Now let's run discovery for device 0 only​
You can find here any possible information on this device​
Whatever you need, the architecture, and more​ Now since we have few GPUs, let's see how are
they connected to each other​ So I'm asking for the topology​
There are several options​ Topology –m​
Will show you how each device is connected to the other​
I'm mostly interested in the telemetry​ For this we can use “dump”​
There are many parameters you could ask for, they are all numbered here​
IGPU utilization, power, Frequency ​ Statistics of the memory​
And each of the GPU sub-units..​ The compute engine, the render,
the media engines and so on…​ I will ask to dump these parameters..​
At interval of 5 seconds​ You can see the titles,
and that the 4 GPUs are idle….​ We get an update every 5 seconds..​
And here I am using 1 of the GPUs.. GPU-0​ And now I'm using 2 GPUs, GPU 0 and 1​
You can see it very clearly​ And now I'm using all the 4 GPUs,
they are all working in parallel​ I can also dump all the data into a file​
Here I'm tarting a process to dump the raw data for device zero​
I can continue to work as usual​ And the data will accumulate into this CVS file​
You can list all the processes that are running and collecting the statistics​
here I have only one..​ And I can stop it​
And all the data is saved to the log file.
So, eyes and ears into your system, the XPU manager​
User guide, other material and a contact email for question or any
other technical help you might need.​ Use it, subscribe, thank you