Researchers from MIT and Stanford University have devised a new machine-learning approach
that could be used to control a robot, such as a drone or autonomous vehicle, more effectively
and efficiently in dynamic environments where conditions can change rapidly.
This technique could help an autonomous vehicle learn to compensate for slippery road conditions
to avoid going into a skid, allow a robotic free-flyer to tow different objects in space,
or enable a drone to closely follow a downhill skier despite being buffeted by strong winds.
The researchers’ approach incorporates certain structure from control theory into the process
for learning a model in such a way that leads to an effective method of controlling complex
dynamics, such as those caused by impacts of wind on the trajectory of a flying vehicle.
One way to think about this structure is as a hint that can help guide how to control
a system.
Using this structure in a learned model, the researchers’ technique immediately extracts
an effective controller from the model, as opposed to other machine-learning methods
that require a controller to be derived or learned separately with additional steps.
With this structure, their approach is also able to learn an effective controller using
fewer data than other approaches.
This could help their learning-based control system achieve better performance faster in
rapidly changing environments.
Determining the best way to control a robot to accomplish a given task can be a difficult
problem, even when researchers know how to model everything about the system.
A controller is the logic that enables a drone to follow a desired trajectory, for example.
This controller would tell the drone how to adjust its rotor forces to compensate for
the effect of winds that can knock it off a stable path to reach its goal.
This drone is a dynamical system — a physical system that evolves over time.
In this case, its position and velocity change as it flies through the environment.
If such a system is simple enough, engineers can derive a controller by hand.
Modeling a system by hand intrinsically captures a certain structure based on the physics of
the system.
For instance, if a robot were modeled manually using differential equations, these would
capture the relationship between velocity, acceleration, and force.
Acceleration is the rate of change in velocity over time, which is determined by the mass
of and forces applied to the robot.
But often the system is too complex to be exactly modeled by hand.
Aerodynamic effects, like the way swirling wind pushes a flying vehicle, are notoriously
difficult to derive manually.
Researchers would instead take measurements of the drone’s position, velocity, and rotor
speeds over time, and use machine learning to fit a model of this dynamical system to
the data.
But these approaches typically don’t learn a control-based structure.
This structure is useful in determining how to best set the rotor speeds to direct the
motion of the drone over time.
Once they have modeled the dynamical system, many existing approaches also use data to
learn a separate controller for the system.
The team from MIT and Stanford developed a technique that uses machine learning to learn
the dynamics model, but in such a way that the model has some prescribed structure that
is useful for controlling the system.
With this structure, they can extract a controller directly from the dynamics model, rather than
using data to learn an entirely separate model for the controller.
When they tested this approach, their controller closely followed desired trajectories, outpacing
all the baseline methods.
The controller extracted from their learned model nearly matched the performance of a
ground-truth controller, which is built using the exact dynamics of the system.
The researchers also found that their method was data-efficient, which means it achieved
high performance even with few data.
For instance, it could effectively model a highly dynamic rotor-driven vehicle using
only 100 data points.
Methods that used multiple learned components saw their performance drop much faster with
smaller datasets.
This efficiency could make their technique especially useful in situations where a drone
or robot needs to learn quickly in rapidly changing conditions.
Plus, their approach is general and could be applied to many types of dynamical systems,
from robotic arms to free-flying spacecraft operating in low-gravity environments.
In the future, the researchers are interested in developing models that are more physically
interpretable, and that would be able to identify very specific information about a dynamical
system.
This could lead to better-performing controllers.