Hi, I'm Priyanka Bagade.
In this video series I introduce how
to implement custom layers for your custom model
with the Intel distribution of OpenVINO Toolkit.
In this video, I show you how to implement custom layers
at the model optimizer level.
We are considering a very simple custom
layer called hyperbolic cosine.
The formula shown here represents
the hyperbolic cosine mathematically.
We have created a simple TensorFlow model with a custom
hyperbolic cosine layer.
Even though the weights are random and untrained
for this model, I want to use it for a demonstration purpose
to show the custom layer implementation for the Intel
distribution of OpenVINO Toolkit.
First, let's run this TensorFlow model
through the model optimizer using this command.
The Marcosh is the full name of the final output layer.
Here you can see that the mo flagged an error,
showing that there is a custom layer which cannot be converted
through the mo.
This is telling us that we need to implement this custom
operation at the mo and at the IE level
in order to get it working with the internal distribution
of OpenVINO Toolkit.
Let's run the extension generator tool
to generate the template files to be added at the mo
and at the IE.
Using this command, check the available arguments
for the script.
For the mo, we have two options, ops, and the model framework
dependent extractor.
For the inference engine, we can select the extension option
based on our hardware of choice.
For this example, we are using a TensorFlow model
and generating the custom layer for CPU and GPU.
These are the arguments to select.
Let's run the extension generator command,
and answer the model optimizer extension generator questions.
First, name the cosh.
When asked if you want to automatically parse
all the parameters from the model file, answer no.
Then it will ask you to enter all parameters.
Since there are no parameters for our custom operation,
just quit out of the question.
Select no for no change in any answer.
For this next question, keep the layer names
same as the operation name by selecting yes.
Select no for the next question, as the cosh operation
does not change shape.
And finally, we don't want to change any answers.
So select no.
The templates are successfully generated and stored
in the output folder.
It has two sub directories for storing extension template
files for mo and IE.
In this video, we will update the files for the mo.
Using this command, open the operation extension source
file from the mo folder.
The class is defined with the unique name, cosh op.
And it is inherited from the base operation op class.
The class variable op is set to the cosh,
which is the name of the layer operation.
The infer function is defined to provide more information
on a layer, specifically returning
the shape of the layer output for each node.
The helper function, copy shape infer node
is used to define the output sheep,
making layer output shape the same as the input.
The copy shape infer function is used,
because for the hyperbolic cosine custom operation,
the input shape is the same as the output shape.
Next, using this command, open the extractor extension source
file.
The class is defined with the unique name, coshfrontextractor
that is inherited from the base extractor, frontextractorOP
class.
The class variable OP is set to the name of the layer
operation, or cosh.
Then the enabled flag is set to true
to include the layer during mo processing.
The extract function is overridden to allow
modifications while extracting parameters from layers
within the input model.
The layer parameters are extracted from the input model
and stored in para.
This is where the layer's parameters in para
may be retrieved and used as needed.
The attributes for the specific node are updated.
This is where you can modify or create attributes
in ATTRS before updating the node with the results
and enabled class variable is returned.
With the extensions now complete,
we can use the mo to convert and optimize example TensorFlow
model into IR files.
Using this command with the mo, create
the IR files for the TensorFlow model
with the hyperbolic cosine layer.
Note that the extension spot has been added in this command.
Let's run the command.
Now, you can see the model, IR, is generated successfully.
In the next two videos, I show you
how to implement the custom layer for CPU
and integrated GPU at the inference engine level.
You can find the sample code and the tutorial
for implementing custom layers in the links provided.
Thanks for watching.
And I will see you next time.
[INTEL LOGO JINGLE]