Welcome back to the OpenVINO channel
We have talked about the full Video-analytics pipeline in previous videos.
The first stage in the pipeline is to uncompressed or "decode" the input video stream.
Please check out video 54,
This is a very compute and memory demanding operation
especially if you are dealing with high definition videos in high frame rates
Or you are trying to process many video streams coming from multiple cameras or feeds concurrently
Intel Quick synch technology uses the fixed functions in the integrated GPU I call VDBOX and VEBOX
to accelerate video decode, encode and processing.
As I mentioned in previous videos, you will need to make sure the driver is installed.
And if you have OpenVINO working on the GPU, you are all set.
If not, check video 18
You will also have to make sure you have the right Gstreamer version and VAAPI.
we'll see that.
In this video I'm going to take a simple python sample code, and just change a line or 2 to move the decode from the CPU to the GPU.
I'll show it to you in 3 stages
1) We'll get the sample code, all the collaterals and we'll just operate it on the CPU
2) We'll run the same sample, but we will use Gstreamer to make sure the decode is executed on the GPU
3) We will run the same sample but use Media-SDK for the decode, again to execute on the GPU
part number 1. I'll build a new directory, Accelerate-Decode
I'll go to the model-downloader directory
And let's try to download mobilenet-ssd .. And place the model in my directory
And I'm getting the prototext and the caffemodell
Now we need to see of there are any special conversion parameters..
I'm looking for mobilenet-ssd and yes, we need to scale and apply mean to the values..
Let's run model optimizer for caffe My input file is the caffemodel
I want the output to be placed in my directory and I'll add the scaling and mean factors..
And I have IR files,..
An XML and a BIN file..
Le's go back to our directory Now I need to bring the python sample,
I'm using the object detection using SSD Asynch sample..
From inference-engine/demos/python-demos/
Now let's copy a video file..
As I explained in video 54. mp4 is a container,
Media-SDK need the native video stream in h.264
So we'll use ffmpeg to generate that video file..
This simple command line will take our video1.mp4 and convert it to video1.h264
That's all And we have the 2 files..
You can see that I can play both of them using VLC, they look the same..
So we are all set, We have the model
We have the sample code, and we have 2 input video files.
Let's open a metrics monitor to gauge the utilization of the GPU..
This utility is part of the media-sdk samples, Please take a look at video 45 for explanations.
I'll need sudo permissions Go to the samples executable directory
Set up all these environment parameters. You can copy and paste them from the transcript
And here are all the media-sdk samples, let's run the metrics monitor
You can see here, the utilization of the execution units
Here is the decode acceleration And the GPU is hardly working at 300 MHz
Now let's run the sample code,
Input is my mp4 video, model is mobilenet-ssd And it's working..
As you can see the decode is running on the CPU and the GPU decode is idle..
Now let's use Gstreamer for encoding..
I'll create a new version of the sample code..
And call it GST_object-detection..
Let's edit the code
On line 108 you can see the OpenCV videoCapture function. We will replace it with this line..
This is basically a Gstreamer pipeline..
Here is the video file to open, we want to decode it, convert it and so on..
This line is all it takes..
We do need to make sure that Gstreamer is set up properly
The result of vainfo should look like that Driver should be i965 , VAAPI version 1.5
Let's also check gst-inspect and the result should look like that
Now let's run the modified sample, with the same parameters..
You can see that the sample is working and the decode is running on the GPU
just like we wanted
If for some reason you do not have the same environment as I have
try to unset the following environment parameters..
next thing to do. Use media SDK
I'll create a new version of the sample code, I'll call is MSDK_obect-detection..etc..
Let's edit the sample..
On line 108
You can see the OpenCV video capture function
Let's change it to this..
I'm giving the path to the video..
Please note this is an h264 video MSDK cannot process containers like mp4..
Handling of the parameters is also a little different,
so we'll need to change these 2 lines to get the height and width from the frame
And that's all..
Let's make sure the environment setting for media-sdk is OK..
Vainfo should give you something like that Driver is iHD, VAAPI 1.4
And let's run it
And it is working And the decode is running on the GPU.
Just like we wanted Great
So, With just replacing 1 line of code you can move your decode from the CPU to GPU
Do it more efficiently, and hopefully be able to process more video streams in parallel
I have demonstrated here for OpenCV in Python, but you could do the same for C++
Feel free to ask questions in the forum, And check out the documents in the links..
Thank you