Welcome back ​
The AI Reference Kits from Intel In collaboration with Accenture..​
There are many of them, they are free and open source, ​
and they could be your first step to an AI base product.
Last videos we have overviewed the full flow of the visual inspection reference kit.​
The goal is to identify defected pills, at a product line.​
And in this video we’ll run the full flow together step by step
As I mentioned, this reference kit could be applied to many types of
products not only pills.. ​ And this is what I’ll
demonstrate in the next video.
So this is the full flow.​ You can find the link to the
git-hub page in this QR code or in this video description.
If you navigate to the git-hub page, you’ll find all the explanations you need.​
There are clear instructions for Windows, but I’ll run the flow on my Ubuntu machine​
Here is the link to the sample data-set used as reference.
You’ll have to download it yourself, ​ And the first thing to do is to clone
the repository, ​ And get into the
visual-quality-inspection directory​ And now we can create the conda environment ​
So first we are creating an environment for the stock, the regular pyTorch distribution.​
Later we’ll build an environment for the Intel PyTorch… ​
Under env/stock you can find the yaml file with the required version of each package​
And let’s create the environment. ​ It can take a while, but once it is there,​
Just conda activate, and you are in.​ And conda deactivate and you are out again..​
Pretty clean and easy to maintain multiple different environments like that​
So let’s create the other environment for the Intel extensions for pyTorch.. ​
We’ll call it AI-Kit environment and you can see the versions in the corresponding yaml file..​
And again getting into, or out of the environment is easy.​
And we are all set with the environment.
And now to the data-itself, ​ I’ve downloaded the pills data-set.​
Make sure you are in the top directory​ Let’s un-zip it​
The unzip will locate all the images under the directory “pill”​
Let’s Create directories under “data” for training and testing,
for good and bad examples​ Get into the pills directory​
and we copy the data-set to the right places under “data” for training.. ​
And for testing..
And now let’s train the model.​ There are several scripts that could help us.​
They are all under the top directory/ src..​ The first one is training.py….​
If you look at the main function.​ You can see that we are parsing the parameters..​
And here we are hard coding some of them like the batch size accuracy and number of epochs.​
My device will be CPU, ​
And here we are loading the data.​ And loading the custom VGG model.​
If you’d like to see the model definition itself, look under utils and open the model file..​
You can see that the model is the original VGG16, ​
But at line 39 we are freezing the first 23 layers. ​
Loading the weights, measuring the time it takes for training..​
And running the actual training process..​
Let’s run training..​ Before you do that, make sure you
are in the pytorch stock conda environment.​ And run the training.. Data-set was found..​
And training is done, accuracy is 91.3%​ And it took 70 seconds to train​
You can see the saved model, pill-intel-model.h5 and the .onnx file.​
​ And now, let’s run the same
training using IPEX, Intel extension for pytorch.​ Let’s get out of the stock environment, ​
get into IPEX environment,​ And re-run the training process..​
38 seconds almost half the time of the regular pytorch​
So let’s evaluate the model,​
We are running inference, first using pyTorch​ We’ll use the pytorch-evaluation script,
let’s look at it​ Under /src​
You can see that we are parsing the arguments.. ​ Setting the device​
Loading the model, and measuring the time for the function evaluate..​
Let’s make sure we are in the stock-pytorch conda environment​
And evaluate…​ Make sure to change to the
model name.. ​ And running..​
This can take some time to run all the test samples..​
And we got to accuracy of 92% test time was 101 seconds.​
Now let’s do the same with the Intel extensions to pytorch​
Get into the environment​ And run the evaluation again..​
The accuracy looks the same 92% but the run time is 3 times faster, only 30 seconds..​
Pretty cool. Considering the fact that it’s on the same HW, using the same code..​
So far, we have set the environment, arranged the data-set, trained using pytorch and IPEX​
And ran inference, evaluation of the performance of the model in pytorch and IPEX.​
You’ve seen how IPEX is faster, in my case 2 or 3 times than pytorch, for the same code.​
But all of these was done using FP32 weights..​ Now let’s take the model and
quantize it to Integer, 8 bits..​ We’ll do it using INC, Intel Neural Compressor
​So we’ll be using the neural compressor. That can do more than just quantization​
Make sure you are in the stock pytorch environment.​
Navigate to the intel-neural-compressor directory​
You can see here our scripts..​ Let’s look at the conversion script​
Parsing the arguments​ Loading the model ​
And here comes the quantization​ The code looks pretty simple.​
And now let’s just run it.​ Make sure to get the model name right.​
Data-set found, baseline is FP32​ And we are getting a report of how many
layers of each type were converted to INT8​ And how did the accuracy changed.​
And we got a quantized model at output directory.​ ​
Now let’s use the other script for inference..​ This script will run inference using the FP32
model and the INT8 model and compare the results..​
It’s using batch 1, you can of course change all the parameters..​
And it looks like the average inference using INT8 is 10 times
faster than the FP32, on my machine, on an i9 CPU​ with the accuracy still high at 93%, very nice.​
​So just a quick summary​ Training the model using pyTorch took me XX​
Training using Intel extensions for pytorch IPEX took..​
Inference using pytorch took 340 mS​ Inference using IPEX took about 100mS ​
And this is all FP32​ Inference using the
INT8 model took about 60 mS.​ Can it get better than that?​
We’ll it can, let’s take the model to OpenVINO.​ We will first just run inference using OpenVINO,
on the FP32 model.​ Then quantize the model
using OpenVINO quantization tools, and compare the inference time…​
​First thing is to set a new conda environment for OpenVINO.​
​And
activate that environment​ In the source directory you
can find the onnx trained model.​ And this is our starting point.​
Now we have to use the model-optimizer to convert it to the OpenVINO IR format.​
We are using the onnx file, and it takes a second
to convert​ ​
We got a bin file and an xml file, this is our model in IR format.​
Next step is to benchmark the performance..​ So we are using the benchmark-app, on this model,​
Pay attention that I am not providing any input data, and I’m running it
without any parameters, ​ And it’s just running​
The default is to run inference for a minute and report the average​
And we are getting an average latency of 54 milli second ​
And about 73 FPS, Inferences per second..​ Which means inference time of about 13 seconds.. ​
​ ​And now let’s use the OpenVINO quantization.​ Under env/OpenVINO-POT you can find
few configuration files..​ Let’s look at the _int8 yaml file,​
Make sure that the directories are correct, I had to change it..​
Basically you should have here links to all the
required files and you can play with the type of quantization and more.​
Let’s run the POT, post training optimization script​
Make sure you are in the top directory​ And it is running.​
This is the results directory​ Let’s look at the original neural network.​
It is the xml file​ Let’s look for FP32​
And all of the weights are FP32 on the INT8 model​
Now let’s look at the quantized network.​ You can see here several FP32 but also
Int8 layers..​ As there will be
a mixture of some layers that remains FP32 and some that were converted to Integer.​
And now let’s run the benchmark-app on the new model​
And we are getting 116 FPS.. Much better than the 73 FPS we got for the FP32 model.​
​And this is my summary on my specific i9 machine, ​
It will of course look totally different on your machine.​
Anyway, the speedup of IPEX, Intel extensions for Pytorch over the stock pytorch is pretty clear​
Quantization to INT8 benefit is really clear.​
And the huge speedup gained by using OpenVINO, and the quantized mode by OpenVINO is clear.​
So this is the full flow of the visual inspection reference kit.​
In the next video we’ll go a little deeper into the flow and talk about using your own
data-set for your specific problem.​ Subscribe, enjoy, See you there..