Welcome back​ ?Using PyTorch
How about getting more
performance ,using almost the same code ​ How about running on the latest intel GPUs
Let me tell you all about Intel extension for pytorch.
I’ll start with the bottom line​ I’m training Resnet50 ​
Left side,​ Pytorch, on Intel Max CPU​
In the middle, same CPU, same training, now using intel extension for pytorch​
Right side, intel extension for pytorch on the Intel Max GPU..​
​ I think that the value is pretty obvious..​
Better performance, better memory consumption and much more
You all want to write code in higher level, python like environment​
But you expect, (or at least hope) for the best performance. ​
In order to achieve this performance​ onn whatever hardware you are using
Intel is using oneAPI optimized libraries, like oneDNN the neural network library, the oneMKL
math kernel library and others..​ Check out my other videos
on each of these libraries,​ These libraries are multi device, multi vendor,
and multi-Instruction set, so you can get the HW acceleration of DL-Boost, AVX512, AMX and more
On top of them you can
find quite a few optimizations tools , ​ and improvements to the drivers and the run time
Intel is continuously contributing to the open source pytorch​
And introduces these improvements to the community whenever possible​
But some features and enhancements are not up-streamed yet​
You can only find these features in the intel extension for pyTorch.​
The changes required for your code in order to use the Intel extension
for PyTorch are minimal, as I’ll show in a minute
If you navigate to the Intel extension for pytorch page,​
I’ll leave you that link​ You’ll notice that there are
two versions, one for CPUs and one for Intel GPUs, or XPUs
Beside the fact that this is currently the only way to run on Intel latest GPUs, ​
You can see here a list of features to enhance performance​
And if you go to “Technical details” you can learn all
about the graph optimizations, memory management and more..​
Installation, make sure you have the right platform, ​
Check out my other video on the oneAPI base toolkit, a very useful toolkit to have​
Pay attention that Windows is not supported yet, but you can run on WSL.. It works for me​
You’ll need the extension version that correspond to the right Pytorch version​
This one line of installations just works.. ​ And if you are using Intel optimized python,
use this line.​
A good place to get started is the code examples​ You got here code for training or for inference​
And it’s really useful that the extension specific code is
highlighted very clearly, ​ Let’s us it for example..
I’m connected to the Intel Developers Cloud.​ Let’s run an example from scratch..
From the examples, training, I'll take a resnet-50 training example as is..​
I’ll start with pytorch, the regular stock pytorch​
So I will not import the extension​ And I’ll remove this line too..​
And I’ll replace all the XPU with CPU..​ So running pytorch on CPU ​
Let’ run it, ​ I’m using SPR, a very powerful CPU.​
Still this is a very heavy operation and it takes about 6 second per batch.​
Now just see how simple it is to move to the intel extension for pytorch​
I’ll just add this line.. Import the extension.​
And use it to optimize the model.​ 2 lines..​
I’m still running on CPU, but now using IPEX​
Model is loading and each batch now takes almost half.. About 3 seconds..​
And moving to the GPU,​
Change all the CPU to XPU ​ Or of course any other target device..​
And now using Intel Max 1100 GPU, running the intel extension for pytorch​
You can see that the run time is totally different..
This is a great return of investment.. Change 2-3 lines of the code and run much
faster on the same platform.​ There are many code examples
in the oneAPI code samples github​ Check out the video description for more link​
Intel extension for pytorch.​ Enjoy, subscribe, thank you.