Researchers from the University of California, Berkeley have developed a new ‘deep learning’
technique that enables robots to learn motor tasks through trial and error using a process
that more closely approximates the way humans learn, marking a major milestone in the field
of artificial intelligence.
They demonstrated their technique, a type of reinforcement learning, by having a robot
complete various tasks — putting a clothes hanger on a rack, assembling a toy plane,
screwing a cap on a water bottle, and more — without pre-programmed details about its
surroundings.
The challenge of putting robots into real-life settings, like homes or offices, is that those
environments are constantly changing.
The robot must be able to perceive and adapt to its surroundings
There are various Conventional approaches for helping a robot make its way through a
3D world.
One approach is pre-programming the robot to handle the vast range of possible scenarios,
and another approach is creating simulated environments within which the robot operates.
But these approaches are impractical.
So now the UC Berkeley researchers turned to a new branch of artificial intelligence
known as deep learning, which is loosely inspired by the neural circuitry of the human brain
when it perceives and interacts with the world.
In the world of artificial intelligence, deep learning programs create “neural nets”
in which layers of artificial neurons process overlapping raw sensory data, whether it be
sound waves or image pixels.
This helps the robot recognize patterns and categories among the data it is receiving.
People who use Siri on their iPhones, Google’s speech-to-text program or Google Street View
might already have benefited from the significant advances deep learning has provided in speech
and vision recognition.
In the experiments, the UC Berkeley researchers worked with a Willow Garage Personal Robot
2 (PR2), which they nicknamed BRETT, B.R.E.T.T ie Berkeley Robot for the Elimination of Tedious
Tasks.
They presented BRETT with a series of motor tasks, such as placing blocks into matching
openings or stacking Lego blocks.
The algorithm controlling BRETT’s learning included a reward function that provided a
score based upon how well the robot was doing with the task.
BRETT takes in the scene, including the position of its own arms and hands, as viewed by the
camera.
The algorithm provides real-time feedback via the score based upon the robot’s movements.
Movements that bring the robot closer to completing the task will score higher than those that
do not.
The score feeds back through the neural net, so the robot can learn which movements are
better for the task at hand.
This end-to-end training process underlies the robot’s ability to learn on its own.
As the PR2 moves its joints and manipulates objects, the algorithm calculates good values
for the 92,000 parameters of the neural net it needs to learn.
With this approach, when given the relevant coordinates for the beginning and end of the
task, the PR2 could master a typical assignment in about 10 minutes.
When the robot is not given the location for the objects in the scene and needs to learn
vision and control together, the learning process takes about three hours.