Welcome back to the OpenVINO channel
our application is a full pipeline..
we are streaming the video, decoding it , because we usually get it compressed from the camera RTSP feed or a file..
doing some format conversions and image processing, and then inference.,
maybe post processing the results or encoding the resulting video for display, storage or upstream
Maybe we even have many video streams running through the same pipeline..
This is of course a very simple kind of application,
and a real application could have loops and feedbacks across video streams or video frames,
If we have a heterogeneous system with a CPU, integrated GPU and accelerators like Movidius VPU or FPGA,
Theoretically each of these pipe stages could run on a different device.
We could run the decode on the GPU, the pre-processing on the CPU and the inference on the Movidius VPU and so on..
But how can I tell if this workload can really work on my system without actually coding it?
How can one predict what will be the expected frames per seconds
or, how many video streams I can really process in parallel?
For the inference part, we have talked about the benchmark utility..
you can use it to estimate the performance of your model on a specific system.,
Please refer to video 32.
But I want to build this full pipeline quickly
so I can run it with different combinations of pipe stages running on CPU or GPU or others
And potentially how can I run this pipeline on multiple systems to choose the right on e for me..
Now we can build and simulate such a pipeline with gstreamer.
GStreamer is a framework for creating streaming video applications.
It is free and open source and widely used,
Please find documentations and tutorials on this link
In this video I'll try to explain the concept, talk about installation, and show one simple demo.
On next video we'll talk about multiple streams and other samples..
Gstreamer can take elements, pipeline them to each other and construct a full workload.
Gstreamer elements could be, for example, open a file for video reading,
decode the video, convert the format and so on
We have basically added analytic elements to the pool of elements,
so now you can use inference operations in such pipelines..
If we go to the git hub page, you can see the description of the new elements
They all starts with GVA for "Gstreamer Video Analytics"
and they can do inference, detection, classification and more..
A typical workload that we can construct may look like this:
This is really one line that describes the full pipeline..
The source of this pipeline is a video file..
The exclamation marks indicate a new pipe stage
And the next stage is to decode the video
And then to convert the format so it will match the inference input format..
And now we are running inference, the model is face-detection and we can add here all the required parameters..
And now we want to draw the detection results on the original image
And we display it to the screen..
So you see how simple it is to describe and build a real video analytics pipeline
now let's do it together
All the instructions are in the git hub.
Let's take a look at the page, you can see here the HW and SW pre-requisits ..
Let's go to the "get started guide"..
Here just follow the instructions to get the data the DL models and the repository
You'll need to add your input video files..
Place them under gva/data/videos// Under data/models you can find all the models..
And now for the installation,
You can install everything on your local machine, or you can run it in a docker, which could be even more convenient
Once everything is installed, let's just run one sample,
I'll make sure OpenVINO is initialized
And there are some environment parameters to make sure are set.
Let's take a look at one simple sample..
Face detection and classification,
This is the input video file These are the DL models to use
The device is CPU, you can make it any device you'd like..
And here is the Gstreamer command line,
we read the video file, We decode it, convert the format And now detection, run on the CPU
And now take the result of the detection and perform classification..
And another classification
And now present the results. Gva-watermark is doing this.
Assuming I do not know what it does, I go back to the web page,
I look for the data-flow, description of the new elements.
And you can see that this element knows how to present the detection or classification results on the original image
And it's running..
So basically we saw how to install and run a sample.
This sample is a full application with 3 different models for detection and classification
And you can build it in one line
Now you can build applications using it or benchmark the performance on any target machine..
This is a powerful tool for creating pipelines, use it
Next video we'll see more examples and more options
See you there. Thank you