Testing an autonomous vehicle in a virtual environment
is a very widely used low-cost and effective method
for developing and testing automated driving algorithms.
In this video, we will learn how to integrate MATLAB
and Simulink with Roadrunner for developing or training
your automated driving algorithms
in a detailed environment.
I will be demonstrating a workflow
to make use of the photo-realistic see-nothing
features of Roadrunner to create a scene within minutes,
and use MATLAB and Simulink to design our training
algorithms using Unreal Engine.
Integrating MATLAB and Simulink with Roadrunner
has four major components.
First, we start with building a custom scene in RoadRunner.
Next, we export the 3D portion of the scene as a Filmbox file
to Unreal Engine and export the road semantics as an OpenDRIVE
file to MATLAB and Simulink.
Here, the Unreal Engine will act as a code simulation platform
to visualize the scene, as well as simulate the vehicles.
We will then import the OpenDRIVE file
into MATLAB and Simulink using Automated Driving Toolbox's
Driving Scenario Designer app and customize the scene
by adding vehicles to it.
And finally, we will be perform the code simulation
with Unreal and Simulink with the scenarios and the vehicles.
So let's get into it.
We start with building a scene in RoadRunner.
You can use different assets, stencils, props,
and different features within Roadrunner
to customize the scene.
A key point to remember during this step
is to have as many road centers as possible.
This ensures that the co fitting between the two roads,
highlighted by the red dot, matches
the road generated when importing the OpenDRIVE file
in Driving Scenario Designer.
Once you're happy with the scene you have created,
we move to step two, where we export the generated scene
into a Filmbox file, to input the file into an Unreal Engine,
and an OpenDRIVE file, to input the file
into MATLAB and Simulink.
On the MATLAB and Simulink site, install and configure
the automated driving tool box interface for Unreal Engine 4.
To copy the project to a local folder
and to plug into Unreal Editor, use
the following one-time steps code to configure and install
the files.
Make sure to specify where the Unreal Engine installation
folder is and the destination folder to install
example files that are shipped with the support package.
Upon successful installation of the plugin,
you will receive the following prompt in MATLAB.
To import the scene in Unreal Editor,
the Roadrunner plugin must be installed.
Start by downloading the latest Roadrunner Unreal Engine plugin
from exporting to Unreal page, and extract the plugin
into a local directory.
Next, creating an Unreal Engine C++ 4 project.
Currently, Unreal Engine 4.23.1 is supported
with R2020a and R2020b.
Now, close the project and copy the Roadrunner Importer folder
and Roadrunner Materials folder into the Plugins folder
under the Project Directory.
If a Plugins folder does not exist in a Project Directory,
create one.
Now, right click on the project and select Gendered Visual
Studio project files to generate binary files for the project.
After this step, open MATLAB and run the Sim 3D editor
to open the project with both MATLAB and Roadrunner plugins.
You will need to provide the path and the Unreal project
title to the same 3D editor.
Click yes to build the plugins.
The script then opens the Unreal editor.
In Unreal, navigate to the plugins folder in the Edit tab
to make sure it's enabled.
If it's not enabled by default, select the checkbox
and hit the Restart Now option to enable the plugins.
Once the two plugins are installed,
import the OpenDRIVE file we had earlier created
into Driving Scenario Designer.
Add a vehicle and specify waypoints to customize
your open-loop scenario.
Feel free to specify additional vehicles or waypoints.
After customizing the scenario, explored the scenario
into Simulink model.
You will need to save the scenario before exporting.
Once the scenario block has been exported,
to control the Ego Vehicle, add a Cuboid to 3d Simulation block
and Simulation 3d vehicle with Ground following block.
And to control actors, additionally specify a Vehicle
to World coordinate block.
You will also have to enable the output Ego Vehicle Pose
option within Scenario Reader to get
the Ego Vehicle coordinates.
Finally, use a Sim configuration block
to specify the source of the scene.
Set the scene source to Unreal Editor
and specify the project part.
The scene view shows us a chase cam perspective
of the vehicle selected.
Also, make sure to set the Solver to Variable Step Solver.
The plugins and the platform are now set.
To import the scene in Unreal, drag
and drop the Unreal Filmbox scene
created in Roadrunner in Unreal Editor.
In the following prompt, set Normal Input Method
to Import Normals, and disable Remove Degenerates.
Hit Import.
This will begin the importing the scene.
And depending on number of assets added,
the importing step may take some time.
The Roadrunner scene is now imported into Unreal Editor.
The props, assets, used in the scene
can be viewed in the Unreal Editor.
Before we begin the code simulation,
repairing the blueprint is required.
You can do so by navigating to Blueprints
and setting the Repairment Blueprint
to Sim 3D level script actor.
Now hit play on the Simulink side.
The dynastic view will prompt the user
to select Play in Unreal Editor to view the scene.
Hit play in Unreal Editor.
And this will begin the code simulation.
The car then follows a set of waypoints
we had defined earlier in the Driving Scenario Designer app
within this open-loop simulation.
You can specify multiple actors, vehicles, and the waypoints
to further customize your scene.
Following similar steps, you can create
a closed-loop automated driving algorithm
that can use sensory information from detections
made in Unreal Engine.
In this demonstration, the vehicle
travels within a marked lane and maintains set velocity,
or safe distance to a preceding vehicle, the same lane
using region processing, sensor fusion, decision logic,
and control components.
This example uses a combination of vehicle dynamics blocks
and model predictive control and Automated Driving Toolbox.
To learn more about Roadrunner, Simulink, and Unreal,
follow the link in the description.
Thank you for watching.