Welcome back to the OpenVINO channel
In the previous video, we talked about the POT the post training optimization toolkit
We explained the expected performance boost, the full flow and where to find all the documentations
In this video, we ll go step by step over a full example..
We will install all the prerequisites Get a model and prepare it for inference,
Set everything up, the 3 configuration files, And check the baseline performance and accuracy
Then we will execute default quantization and compare the performance and accuracy
And do the same after accuracy aware quantization..
I am using a Xeon Gold server Lscpu
You can see that I'm using a Xeon gold machine that has a DL-boost supported,
So I have high expectations on the performance boost
For the installation, I am following the documentation in this link.
I ll do it step by step and share the instructions Look for it in the video description
First, I m making sure that I have everything I need for the model-optimizer
Model Optimizer, Install pre-requisits
Then I m installing the accuracy checker Just follow the README file in that directory
And the POT..
The post training optimization toolkit Setup install
And I m all set
I ll work under ~/POT directory
And I m using the model downloader to download a mobilenet-V2-pytorch model
The output is to my directory
Now I have to convert the pytorch model to ONNX
For that I need to install the pytorch requirements That are in the downloader directory
And now I need to know the specific parameters for this model..
The best way to find them is to look at the model yml file, located in the open-model-zoo
directory
And these are the parameters I need to use when converting from pytorch to ONNX
I will run the pytorch to onnx script, with all the parameters
And I have an ONNX model
Next stage is to convert the ONNX model to OpenVINO IR format..
Again I will look at the yml file for the right parameters
And I have the IR files,
Let s just rename them model.xml and .bin and place them in my directory
OK, this is the tricky part.
We have 3 configuration files to update..
The first one dataset_definitions.yml is more of a global file describing all the data-sets
I might have
You can find here the definition of all of them..
But the data-set for my mobilenet model is image-net
And I will set the pointers to the location of the data, the annotations file and pickle
Need to make sure to set the DEFINITIONS-FILE to the location of this file..
And now let copy the other 2 files to my directory
The first one is a json file, located under the POT,
I m doing classification, mobbilenet-V2..
The 2nd file is model.yml file that could be found under the accuracy checker configs
We have the 2 files, let s edit them.
In the json file..
I update the names and locations of the IR files.
Config is my yml file, Let s add few parameters, you can find the
list of all parameters in the documentation Target device is CPU for example
Let s edit the yml file
These are the instructions for the caffe Launcher we do not care about them..
We need the dl-sdk Launcher OpenVINO And again few parameters to add, the model
s location, the data location
And we are all set And now let s understand what is the baseline
I m running the accuracy checker with the yml file as a configuration file..
and it's running And my top1 accuracy is 69%
and my top 5 accuracy is 89.25%
Now let s run the benchmark-app And I m getting more than 4000 frames per
second, not bad
Looking at the json file, you can see that we are set to run default quantization
And I will run the POT The config file is my json file, I want to
see the accuracy results and enough information in the log
And it s running You can see that the accuracy I m getting
is 89 % a drop of less than 1 percent..
And let s benchmark the performance of the optimized model we got,
Under the default../optimized
And it is running, and pretty amazing I m getting almost 10,000 FPS
This is about 2.5 X the performance for less than 1% accuracy drop..
And last, let s run accuracy aware quantization
I will edit the json file, so we will perform accuracy aware quantization,
and I can set all kind of parameters here Let's run the POT again .
This takes a while and few iterations But you can see that the accuracy is even
better It's 89.12%
And let s run the benchmark and I'm getting more than 10,000 fps
very good
We have seen a full example of using the Post training optimization toolkit
You can download the files I used for this example in the video description
And check the full documentation in the link
Enjoy the performance boost Thank you