Welcome to the OpenVINO channel
On video 15, we talked about the Inference Engine plug-in architecture.
Basically, the IR files, The model that we are using,
Is optimized by the model-optimizer which is hardware-agnostic
The inference engine, is doing the device specific optimization,
by using the right library cells
and by implementing the inference using the specific plug-in for that specific device
on video 53 we introduced the "Mutli" plugin.
This plug-in could be used to run multiple inference requests on multiple devices in parallel,
So you could run your inference in parallel
on CPU and on our integrated GPU for example..
The devices are being utilized based on their availability,
Which basically means that we can now sense the devices..
And this is one important new capability, starting R2 2019..
We can sense which devices are connected and available to use and we get indications on
their real physical condition.
Basically, the IE has now eyes and ears to feel the system it's using and by that,
better utilize it..
Before that new API, you needed to create a plugin for each device you wanted to use,
Now we have a new class called Core, which works on top of the plugins and provides
the same functionality
It has all the methods as you can see here, same methods as you can find for the plugins..
But now you call them with the device name
So you could basically create a Core class object, without any specific device
And you specify a device only when you load network our actually use the device
So no need to explicitly register a plugin, it's all done by the CORE class internally
Registering and managing the plugins could be done using a configuration file or using
a C++ code
First interesting new method is "get available devices"
Which will return a vector of available devices in the system
This is an amazing capability that allows the IE to actually give you real time indication
of the available physical devices
The devices names here, could be used as device names for the API..
For example You can see here that we have 2 FPGA boards connected
And their device names are FPGA.0 and FPGA.1
"Get metric" will get you the full list of metrics this device has.
Than you can look for any specific metric to check specific information for this device
Could be the device name, supported formats or even current temperature
You can find the list of device metrics in the file ie_plugin_config.hpp
Check out more information in the below link
When you run the benchmark app And please watch video 32 if you're not sure
how to operate it, You will get the list of all available devices,
For example you can see here, that I have available in my system.
CPU, GPU and HDDL
If I plug in a Myriad NCS to my USB port..
You can see that I also have a MYRIAD device
And now, I'll plug in another Neural Compute Stick in..
And you can see that I have 2 myriad devices available..
The first one is called myriad.2.1.-ma2450 and you can see the name of the second as well
An interesting sample is the "hello-query" which basically sense the available devices
and print out all their parameters.
Let's first look at the code Under inference-engine/samples/hello_query
You can see in the main function that I'm assigning a "Core" object
Look for all the available devices For each available device, looking for the
available matrices
For each of these parameters, I print them
When I run the sample, you can see all the parameters for my CPU
It can support FP32, INT8 and Binary models..
My integrated GPU my HDDL, And the Myriad-2 that is connected to the system
So we have seen the new API introduced at mid 2019
It will allow you to have a closer and better use of the physical system
Please check the full documentation in that link
Feel free to submit a question in the forum and of course, subscribe to our channel to
get more videos like this one
Thank you