In this video, I'll be introducing a new Simulink
subsystem, called the Neighborhood Processing
Subsystem, that will provide an easy and fast way
to design algorithms that operate on a small neighborhood
of a larger matrix.
This design pattern is often referred to
as the neighborhood pattern.
And it is commonly used in computer vision and image
processing, deep learning, signal processing, and other AI
applications.
In simpler terms, if the matrix of numbers
that you see on the screen is the input to the subsystem,
the Neighborhood Processing Subsystem
will perform an operation on a smaller section, indicated
on the screen using the smaller matrix in yellow,
of that larger input matrix.
The operation is usually mathematical.
Arithmetic operations such as summing the matrix elements
or performing a form of filtering are common examples.
The neighborhood then slides along the original matrix
to perform the same operation on another section.
This was, in simple terms, the concept of the Neighborhood
Processing Subsystem.
To dive more into details, I'm going to use some demo models.
The Neighborhood Processing Subsystem, or NPSS for short,
brings the following features into Simulink.
The ability to alter algorithms on neighborhoods
of n-dimensional signals, an easy way to pad the matrix
boundaries, high performance simulation,
and the ability to generate production C/C++ code from
your Simulink model.
Here's how it works.
The following model performs a basic edge detection algorithm
applied on a 3 by 3 sliding neighborhood
of the original image.
As you can see on the screen, the output image
on the right-hand side of the screen
represents the edges of the peppers
that are shown in the original image on the left-hand side.
In between those two images, you will find the NPSS.
Notice how the signal dimensions on the input and output signals
are 384 by 512, which represent the full matrix of pixels
of this black and white image.
Now, let's look under the subsystem
to find out exactly how this algorithm is implemented.
Inside the NPSS, we multiply the 3 by 3 neighborhood of pixels
by a matrix gain that represents a filter or a kernel.
Then we sum the neighborhood elements.
The sum of elements corresponds to the output pixel
of this neighborhood algorithm.
It is placed at the central location of the neighborhood
matrix in the output image.
We place a saturation block to ensure that the output is
between 0 and 255.
And notice how the signal dimensions inside the subsystem
indicate a size of 3 by 3 compared to the larger signal
dimensions we saw when we were outside the subsystem.
That is one of the many ways you can configure the neighborhood
block.
To change the parameters of the subsystem,
double-click on the configuration
block you see over here.
You can specify which input signals to the subsystem
are subject to be transformed into neighborhoods
using the following checkbox.
Multiple inputs and multiple outputs
are supported for the NPSS.
To specify the neighborhood size,
use the following parameter.
To indicate the size of the matrix,
use the MATLAB language vector format.
The output subsection parameter allows
you to specify the section of the neighborhood operation
that will be computed as the subsystem output.
The parameter of same means neighborhoods
whose central pixel falls within the input matrix will be used.
Hence, the input and output sizes will be the same.
Full means that neighborhoods will
be used if they contain at least one pixel from the input
matrix.
The parameter valid means only neighborhoods
that can be constructed without padding will be used.
For neighborhoods at the matrix boundary,
you can specify the padding method
using the following parameter.
You can either pad the neighborhood with constants
and set the constant values over here, replicate the bordering
pixels, use symmetry to pad with the mirror image of the pixel,
or use a circular padding method.
More information on padding and other configuration settings
of the subsystem can be found in documentation links
on The MathWorks website.
The edge detection Simulink model
is a simple example of how you can use the NPSS for image
processing applications.
A more complex application of the NPSS is optical flow.
Optical flow is using computer vision
to measure the apparent motion of objects in the frame.
The following Simulink model is used
to compute the optical flow of a moving
object using several neighborhood processing
subsystems.
The NPSS was used for algorithms that
involve the computation of pixel brightness
gradients along x and y and the computation of the window
function and solution of the Lucas-Kanade method.
As I run the model, you can see that the rhinos, as well
as the trees in the background, are the moving
objects in this video.
Yellow markers are placed around the moving objects
to demonstrate the optical flow.
The optical flow model is a shift example
with MATLAB R2022b.
Feel free to go ahead and use this model along
with some of our other shift examples
to explore the different capabilities
of the neighborhood processing subsystem.