Welcome back to the OpenVINO Channel
During the full inference flow, we are manipulating the deep learning based models
We are processing the model with the model-optimizer,
changing the topology, maybe converting the data formats, scaling, normalizing..
We are using various library cells for the implementation which could be wrong.
And the Inference-Engine is also executing the model in a different way that Caffe or Tensorflow would do..
So after all of these customization, we would like to go back to our data and make sure
that we still have the accuracy level we trained this model for.
For this we have the validation application Look for the full documentation in the web.
It works for classification and detection models and it is quite simple..
I will start with a Classification example and move to a detection one..
Stage number one is to prepare the data for classification.
There are two options here
Option one is to place all the images in one folder, And to document the classes in a .txt file..
For example this image belong to class 489 This image to class 360 and so on..
Option number 2 is to have all the images organized in folders,
The folder name is the class name and it holds the samples for this class
Roses folder for example should hold images.. .bmp or jpeg or roses and so on..
Here under flowers_photos I will have a folder per class
and you can see that the "roses" folder, has images of roses and so on..
Running the validation app is easy You need to compile the sample, it's under inference-engine/samples.
If you are not sure how, please watch video number 5.
I will direct to the openvino installation directory
Initialize OpenVINO Go to the folder with my images
Run the validation app, -t my type is C.. Classification
-I the input will be my flowes folders -m my model is mobile-net here
-d device is CPU And --dump, I want to explore the results
in a file.. And it's running.. It's going over each
of the images, classify it and collect the statistics..
You can see that the top-1, meaning the times that I have classified correctly for my top probability is 85%
If you are ok with top-5, meaning your class
was one of the top 5 highest classes, we are at 100%...
Let's open the dump file.. This is the name of the image classified..
You can see here the top 1. probability, top 2 etc..
This column indicate if the classification was correct.. You can see 1 for most images,
meaning classified successfully For this image for example, classification
is ok, it was classified with probability of 94 % to be of class 4 which is Tulips..
This image was mis-classified.. The classifier thinks it's of class number 2..
Let's look for this image in the tulips folder..
And to be honest, I don't see here a tulip myself..
Going down to another class,
Daisy is class number 0 and so on..
So you can get the overall accuracy, and even debug which specific images were misclassify
Validating a detection model has few more steps..
Please check the documentation for the full explanations..
I'm going to show you an SSD 300 example.. First stage is to get the model..
Use the model-downloader for that.. , check out our video number 6
Check the yml file for all the parameters required for the model optimizer..
Run the model-optimizer, and we have IR files..
Now get the SSD data set.. You can download it from the public repository..
This could be the tricky part, to get the right data set and have the right legal rights to use it
you need to create a labels file,, you can just copy paste it from the documentation..
It's a simple text files with labels and their class number.
Once everything is set.. Run the validation app
-t OD type is Object detection -I input is the VOC-dev-kit
-ODa Object detection annotation -ODc Object detection classes..
-Odsubdir direct to the images.. -m our ssd model
And running on CPU ..
You can see the average precision per class. Class one was detected with 90% accuracy
And the average of all classes is 87%.. Not bad..
So we saw how to re-validate your model with the validation app Both for classification and detection models..
Subscribe to our channel to get more videos like this
Thank you.