Welcome back ​ ​The AI Reference Kits from
Intel In collaboration with Accenture..​ There are many of them, they are free
and open source, ​ and they could be
your first step to an AI based product.​
In these short videos, we’ll talk about the Visual Quality
Control Inspections for Life Sciences toolkit.​ In short, “the visual inspection” reference kit.​
This is an end to end example of performing quality control of a product line, ​
And I will leave you all the links and the instructions below..
The goal is to identify defected pills, you can see here examples of good pills
and of defected ones..​ Having humans visually
inspect each pill is expensive, ​ think about a 24/7 product line,
with millions of pills to sort at a rate of 2 pills per second for example.​
People need breaks, people make mistakes, ​ And Maybe this assembly line does not have
good working conditions, not safe, too hot, air is polluted, the machines do not care
So it makes a lot of sense to automate the process, and this
is exactly what this reference kit is doing.​ Once you build an application that can do that,
and detect the defected pills​ It should be easy to integrated
it into a working product line, This example show the implementation
of AI based detection of defects in Pills​ But you can easily try other types of
products and objects, ​ Bolts, capsules,
screws and other objects should work as well​ Pay attention that the data-set linked in the
toolkit is not free for commercial use.. So in this video I’ll overview the full
process in high level​ In the next video,
we’ll actually run the flow end to end​ And in the last video I’ll talk about using your
own data, to your own specific product line.​ Adjusting the flows and so on.
So it all starts with the Data.​ You can easily detect that this pill
is good and this one has a defect,​ But how can a computer detect it
For you, all these pills are the same, ​ Humans can still detect the pills if
they are rotated, scaled shifted, or of a little different color​
For a computer, if you change one pixel,, it’s a totally different image..​
What we need is a system that is invariant to these image
manipulations like rotating shifting and such.​ And this is exactly what our deep-learning
model is doing..​ It sounds like magic,
so let me explain it in a minute The Deep learning based model we are using
in this case is a variation of VGG16. ​ It is a convolution neural network,
most of the first layers are convolutions, ​ I has 16 layers, and it was originally trained
using a dataset (ImageNet) that has more than 14 million training
images across 1000 object classes. ​ Each of the first layers is using
mathematical operations to basically extract “features” of the image​
Features in an image are for example, lines, corners, ​
the specific regions in the image that helps us humans too, to understand what’s in that image.​
The deeper you go into the neural network you are getting more fine grain features..​
​And at the end. we can use a features-map to make decisions..​
For example, if feature 4 from the first layer and feature 8 from the 3rd layer exist, it might be a
round object with a defect.. For example.. Let’s look at the model we are using , ​
On the left side you can see the original VGG16​ If we zoom in​
You can see that the first layers are convolution.. And another convolution..​
And after few such layers, we have a large set of features that we can extract from each image..​
And we can use a features-map to make decisions.. Down here..​
On the right side, the first layers of our model are also convolutions, the same convolutions,​
But the final processing, the classification head is replaced to meet our specific needs,​
In this case classifying good or bad pills. ​ If you’d like more details,
You can read more in this article.. ​ Looking visually at the layers of the
neural network​ We can draw like
a heatmaps of the filters, of the layers​ What’s nice about that is that it is not only
performing classification, good or bad,​ But can also show you the location
of the defect, if found..​ You can see here for example
one pill I chose and how the feature map looks like after the 2nd layer.​
You can clearly see that it understands that the object is oval
Next layer a little more specific features, ​ You can see the curves
Next layer, smaller features, I can see here even points
And in the next layer, you can see even smaller, specific features
And here even smaller… So let’s try to get some
intuition by looking at a good pill below and a one with a defect on the top..​
Look at the red arrows.​ And you can see below, that we are
expecting a black image, but since we are getting a white dot, a high value for the top pill​
Not only that we know that is has a defect but we can even understand its location
And same goes for this defect, ​ Expected, below, dark heatmap​
A high value, a white dot instead for the top pill, with the location of the defect.. Nice.
So, I’d like to take a minute to overview the full story, the full pipeline as
brought in the reference kit.​ I’ll explain step by step and
actually run it in the next video.. First step is to arrange the data,
we divide it to training data and test data​ And ideally we have for each one of them good
pills and bad pills, placed in corresponding directories​
We do not label or mark the region of the defects, it will be done automatically...
Once the data is in place, we can train the model.​
As said the model is based on VGG16​ We are not training the full model,
we are freezing most of it, ​ All the feature extraction layers,
the first layers can be used as is. ​ With their original weights
so no need to re-train them.​ We will only train the head, the last layers..​
Since we are not training the full network, ​ This means that the training process can be
executed on a simple CPU in a reasonable time.​ Which is a great advantage..​
We will take all the images in the training directory,
pass them thru the model and adjust the weights.​ The result of this stage is a trained model, that
could classify the good pills from the bad ones.. The training is done using the pytorch framework.​
Intel is helping to improve pytorch, but also has an extensions for pytorch​
What we call IPEX.​ Basically enhancements to the stock pytorch.​
Please see my video on Intel extensions for pytorch, IPEX​
We will re-run the training and later the inference with
IPEX and you could see for yourselves​ the performance boost it can give you
over pytorch, using the same code..​ For me, on my i9 CPU, IPEX was almost
twice as faster as pyTorch Once we have a model we
would like to evaluate it,​ We have used the training
data-set for the training.​ And now we will try to classify
the test data-set,​ basically images that
were never introduced to the model​ We will check the accuracy, meaning the
percentage of correct classifications ​ and we will measure the performance,
how fast it can do it ​Everything we did so far,
the training and the inference ​ were done on a model which all the
operations are done using floating point numbers​ FP32, so 32 bit long, It allows for good accuracy
and a wide dynamic range of numbers​ But this level of accuracy is sometimes
not needed during the inference phase.​ Once the model is converged.
Many times, the weights does not need to be represented using floating point or 32 bits..​
and converting them to Integers, ​ INT8 for example will not degrade
the accuracy much​ The performance however,
could be dramatically improved.​ First, we’ll need only 8 bit Vs 32,
so much smaller weights, but also faster execution time and more.​
During this phase, we’ll quantize the model using INC – Intel Neural compressor. From FP32 to INT8​
And we’ll compare the results, the accuracy and the inference performance..
At that stage, ​ you have a working model,
and you can take the pytorch code to an application, and a product.​
But there are several advantages to run the inference using OpenVINO.​
OpenVINO is the deployment toolkit by Intel,​ I’ll show you how easy it is to take the model,
benchmark it and run it not only on a CPU but on a GPU, and many other devices,​
And with better performance, and lower memory consumption​
And this is all for now,​ On the next video,
we’ll run the reference kit step by step​ And in the video after, we’ll talk about
customizations and advanced stuff​ See you there​