this is the second part of our series on targeting vision applications to FPGA or ASIC hardware the
previous video outlines some high-level considerations for adapting an algorithm
for Hardware implementation this video will overview a recommended workflow from a matlab
algorithm to an FPGA implementation algorithm work typically starts in MATLAB and MATLAB is
great at manipulating large data sets such as image matrices exploring mathematical algorithms
and visualizing data on the other hand Simulink is great for adding the implementation detail
that you need to describe how your algorithm will work in streaming hardware you can still include
MATLAB code within Simulink but Simulink lets you express and visualize parallel architectures
it has a built in sense of timing and can show you how data types and sizes propagate
through calculations which are all key aspects of creating a good Hardware implementation for your
algorithm so our recommended workflow based on our experience across thousands of customer projects
is use your MATLAB algorithm is a golden reference to verify that your Hardware implementation gets
the right answers move to Simulink to build a hardware implementation for your algorithm
convert to fix point and Simulink where you can visualize the word lengths propagate through the
design use simulates visualization to optimize your hardware implementation before you generate
HDL code and verify that each refinement functions properly using the Lane detection example here's
the matlab algorithm it's based on the automated driving system toolbox example the test bench
reads in a video file and sends a frame at a time to the algorithm which returns parabolic
Lane boundaries at the same time it stores the frames to a workspace variable that will be used
to drive the hardware implementation the test bench displays the input video and a version
with the lane overlay outputs from the algorithm it also stores the parabolic equation parameters
for each frame here in this structure which gets used later to check the results of the hardware
implementation now I could have pretty easily replicated this test environment in Simulink
but since I've already done this work in MATLAB I can use that and call Simulink using the sim
command passing data back and forth via work space variables over in Simulink the input to
the hardware model is this video from workspace block which uses that workspace variable and the
sample time parameter in the model the first thing that needs to be done is to convert the incoming
frames to a stream of pixels this gateway block is from vision HDL toolbox and with Simulink you
can see the changes in the data types from the 480 by 640 frame to one pixel at a time the difference
in colors is showing sample rate differences the output needs to be sampled every pixel whereas the
input needs to be sampled once every 420 thousand pixels which represents one frame with horizontal
and vertical blanking included in this block you can specify a preset or custom frame size it will
automatically convert the frame to a stream of pixels at the faster sample rate and will also
add in horizontal and vertical blanking intervals to give time to the processing to catch up and it
generates a control signal bus for managing the stream with signals to specify the beginning and
end of rows and frames along with a valid signal also notice in this diagram the clock signal there
is no clock in the Simulink design but there is on an FPGA the default mapping is one unit of
same a link sample time maps to one clock period on the FPGA which first say at 200 megahertz
frequency would be a 5 nanosecond clock period before moving into the hardware implementation
details in Simulink i want to step back and show the rest of the simulation environment
the sample by sample results get buffered here to return to the test bench this isn't generating a
video frame out just laying boundary coordinates it's likely this buffer gets implemented on the
software side and the final product but I use it here to reduce and condition the data passed back
to the test bench back in the MATLAB test bench I just get the Lane boundary coordinates from the
workspace variable and compare the distance from the center to the lane boundaries to the offset
parameter from the reference parabolic equation for this application I'm okay with a 20 percent
tolerance due to the fixed point quantization and the algorithms curve fitting estimates now
I can just run right from MATLAB simulating 30 frames in MATLAB then in Simulink which takes
about nine minutes on my laptop so I'll just cut to the results every time you add new details to
your implementation model you should verify it either against the previously verified model or
against the golden reference which is what I did here if I create a new model I can just pass that
model to the sim command but it depends on your specific designs characteristics and requirements
part three of this series covers adding Hardware implementation details to the design