Welcome back to the OpenVINO channel
Last video we saw how to install and make the HDDL card work.
Just to mention, this is the Intel vision accelerator which has 8 Movidius myriad-X inside..
Let's see how to look inside this device and how to benchmark and operate it..
First let's make sure that the HDDL-Daemon is not working,
Please see video 35 for explanations.. And it is not..
Let's go to the HDDL install directory
We will edit the config/hddl_service.config file..
Look for device_snapshot_mode, you can set it to one of the 3 options mentioned here,
"none", "base" or "full" Let's make it "full"
Save the file And operate the hddl-daemon
You can see the 8 devices booting
Now let's go to the samples executable directory,
I describe how to compile the samples on video 5
Please note that now we have samples and demos On different directories..
I'm going to run the benchmark app on a just random model, -d HDDL
Please note that "a worker", a working thread, was initiated for each of the myriad-X devices..
Service is ready, And you can see the utilization of each of the 8 devices.. Currently zero
I start the benchmark
And the utilization is going up to 70 and to 100 % on the 8 devices
You can see here the thermal and other parameters, and it's done.
We are getting about 950 FPS and utilization is going down to zero again..
Now let's try to access each of the 8 M-X devices separately
First, let's make sure the HDDL-Daemon is not working..
Now from the HDDL Install directory Let's run the bin/bsl_reset
And we see that all the devices, the 8 M-X devices, are reset.
Now, when you run the benchmark-app -h you can see which devices are available in the system
And I have a CPU, GNA, GPU, HDDL but also the 8 M-X devices, each with his unique name..
So let's try to benchmark the same model only on 1 Myriad-X..
I will choose the first M-X as the device And now only 1 M-X out of the 8 of the HDDL is running
and I'm getting 110 FPS..
Now let's try to run in parallel on 3 of the devices..
Let's get the list of devices..
Now let's use the MULTI plugin..
Check video 53 for explanations
So MULTI :1st M-x, 2nd M-X, and so on
I'm using 3 M-x devices, so I expect 3 times the performance..
You can see that the 3 devices plugins are working and we are getting 356 FPS..
3 times the performance of one..
So this is great because the performance is scaling as you are using more M-X devices
And you can control how many devices could be used to which models..
If you are running few models in parallel and want to tightly control it..
So we saw how to view the HDDL utilization and device real time information
How to benchmark the performance of the HDDL and even each of the 8 M-X devices
The HDDL device, is a good fit if you are looking to get high inference rate on relatively smaller models, but not only.
So an NVR, or a hub serving dozens of cameras is a great example..
Subscribe to our channel to get more videos like this
Thank you