Welcome back to the OpenVINO Channel
On Video 40 we talked about running models in Integer 8 bits, INT8 format
Now I'd like to show you a real example, of a typical full inference flow..
Stage number 1 will be to get all the data we need.
The models, the data-sets, make sure the mode-optimizer is setup to work with tensor flow etc..
Stage number 2 We be what we call "transfer learning"
We will add new classes to the model and re-train the model to classify them
This is a very typical flow.
If you need to classify specific objects, you do not need to build a DL model from scratch,
You download a public one, add the classes you need, provide images of these classes
and just re-train the model to classify them.
We will take a mobilenet model and re-train it to classify new classes of flowers it was not originally trained for
You can follow the same procedure to add your specific classes..
Stage #3, We will use the model to run classification and check that it is really working.
Stage #4, we will calibrate the model to INT8
Stage #5, we wil try to benchmark the performance and see if there is a benefit when we move from FP32 to INT8
stage #1 first define the install directory
Initialize OpenVINO
go to the model-optimizer installation and make sure the prerequisites for tensor flow are met..
Now lets get the model and the setup for this example from git-hub
and let's download the images for the new classes
you can see that under flowers-photos I have 5 directories, Each with the name of the new class and holding images of this specific flower
stage number 2. I'm going to run tensor-flow to re-train the model with the new 5 classes
All the instructions are available to you in the transcript.. (at the end)
you can just copy paste them..
I'm re-training a mobile-net model but you can do it for other topologies as well
now I'm just re-naming the labels file..
and I have a new model, it's called "retrained_graph.pb"
I'm going to generate IR files,
I'm running the model-optimizer With the right scale values for this model,
I have explained in previous videos how to find these numbers..
and I got new IR files I can now use
stage number 3.
We have a new model, we converted it to IR, But can it really classify the new classes?
Let's run the classification sample
Pick a random rose image use the new model and see the results..
and with 99.5 percent this is a rose.
If you check the original model, you'll get zero
I am trying one example but you can run the validation app on the full data-set to make sure it's working,
well it does..
Check out Video 33 for the validation app instructions
Now we are at stage number 4
Let's run the calibration tool.. -t C.. this model type is classification
-d CPU I'm running on CPU.
my input data set is the flowers photos and my model is the FP32 IR file.
I am asking for a threshold of 5 %
meaning I can suffer a degradation of 5% in the accuracy..
the first stage will be to run with the FP32 model
There you can see that my reference accuracy is 85.9%,
I allow the tool to go all the way down to 81%.. 5% degradation
The tool is running, it takes time..
And you can see the accuracy degradation of each iteration..
and it reports which layers will be converted to INT8 and new IR files were generated
stage #5, you can see the new IR files that were generated by the calibration tool
They are called _i8
let's try to run again the classification sample with the new, calibrated IR
And yes, it's a rose...
And like before, you can use the validation utility to verify the full data-set
and even debug which images are now not classified and why..
But we care about performance, so let's run the benchmark app
First let's run with the FP32 model..
And I'm getting 825 FPS
INT8 is supported on specific machines only I'm checking if my machine is supported
lsCPU and I'm looking for AVX.. and my machine supported AVX2, so I can expect some performance boost
let's run the new calibrated Int8 model.
And I'm up from 800 to almost a 1000 FPS, not bad..
If you have a machine that supports AVX512 you can expect higher performance boost.
If you have a cascade-lake machine that supports VNNI, you will get even more..
So we demonstrated a full flow of downloading a public classification model,
Re-train it to classify new classes Generate IR files using the model-optimizer,
Calibrate the IR files using the Calibration tool to INT8 format
And run the benchmark utility to check the performance
Please use the official documentation and the forums to ask questions and share feedback
Don't forget to subscribe to our channel to get more videos like this one
Thank you