Welcome back to the OpenVINO channel​
In previous videos we have seen how to construct Gstreamer pipelines using the Gstreamer elements
and the GVA –Gstreamer video analytic special elements​
But what if I need a customized operation somewhere along the pipe​
What if I need to build a special pipeline stage?​
For that we have the gvapython,
basically the ability to use a python script as one of the pipe elements.​
For this we need a little deeper understanding on the way the gstreamer pipeline is being managed..​
The GST-Buffer hold the frame’s information.​ Every pipe stage can read the gst-buffer
information as an input and may add information to the buffer as its output​
The DL-streamer have enriched the gst-buffer information types and ​
added few classes to the Gstreamer original meta-data…​
You can find here the full documentation of the new structures.​
ROI, tensors and video frame​
And this is the full documentations,​ VideoFrame holds the data for one
frame in the GST-Buffer​ It can hold many ROI. Each
can be the result of a detection for example ​ Each ROI may contain multiple tensors​
And the videoframe itself can have multiple tensors, output of inferences for example​
Going inside the video-frame class you can see the full description​
ROI for example will hold detection result
bounding boxes, confidence level, object ID and so on..​
These data-types inside the gst-buffer ​ are your hooks for the python script you
can add using gvaPython​ Let’s see an example​
So I’m looking at the gvapython/ ​ Face detection and classification sample​
And first let’s run the sample..​
You can see the face detection works,
the confidence level, the estimated age, this is the Female, this is a Male​
Looking at the sample​ We read the input video​
We are performing inference, face-detection and then post-processing using a python script​
And then we execute classification and post processing using another python script​
We will focus on that part​ And then present it to the screen​
Classification and Python-script-2​
Classification model is age-and –gender​
And the python script is age-genger-classification…​
Let’s look at the classification model​ You can see that it has 2 outputs..​
This one is called age_conv3​ And this one is prob​
Looking at the model description, this is the age and gender classification model..​
we can see the outputs..​
Let’s see the python script​
The function to run is process_frame, unless said otherwise​
And the return value is true if the frame is OK,
if the return value is false, this frame is dropped​
We are looking at the video-Frame​
For each of the regions in the frame, ​ For each of the tensors in the region​
We are not processing the detection results, so if it’s a detection tensor we don’t touch it,
only the classification​ If it’s a classification result,
we take the last layer name​
If this layer is age_conv3, than according to the description,
the data is the age​ We just need to multiply it by 100​
If it’s the layer prob,​ Than we have the probability
if it’s a male or a female​
​
And this is the processing that this script is doing,​
It adds to the label in the gst-buffer the gender, male or female and the age estimation​
This new capability basically means that you can not only use the existing
Gstreamer or GVA elements​ But build your own
custom processing using python ​ Which opens up many opportunities
and allow for a great flexibility in constructing more robust pipeline​
Thank you​