In recent years, robots have gained artificial vision, touch, and even smell.
Researchers have been giving robots human-like perception.
In a new paper, MIT research team is pushing the technology a step further.
They say that they are trying to give robots superhuman perception.
The researchers have developed a robot that uses radio waves, which can pass through walls,
to sense occluded objects.
The robot, called RF-Grasp, combines this powerful sensing with more traditional computer
vision to locate and grasp items that might otherwise be blocked from view.
The advance could one day streamline e-commerce fulfillment in warehouses or help a machine
pluck a screwdriver from a jumbled toolkit.
The research will be presented in May at the IEEE International Conference on Robotics
and Automation.
As e-commerce continues to grow, warehouse work is still usually the domain of humans,
not robots, despite sometimes-dangerous working conditions.
That’s in part because robots struggle to locate and grasp objects in such a crowded
environment.
Perception and picking are two roadblocks in the industry today.
Using optical vision alone, robots can’t perceive the presence of an item packed away
in a box or hidden behind another object on the shelf — visible light waves, of course,
don’t pass through walls.
But radio waves can.
For decades, radio frequency (RF) identification has been used to track everything from library
books to pets.
RF identification systems have two main components: a reader and a tag.
The tag is a tiny computer chip that gets attached to — or, in the case of pets, implanted
in — the item to be tracked.
The reader then emits an RF signal, which gets modulated by the tag and reflected back
to the reader.
The reflected signal provides information about the location and identity of the tagged
item.
The technology has gained popularity in retail supply chains — Japan aims to use RF tracking
for nearly all retail purchases in a matter of years.
The researchers realized this profusion of RF could be a boon for robots, giving them
another mode of perception.
RF Grasp uses both a camera and an RF reader to find and grab tagged objects, even when
they’re fully blocked from the camera’s view.
It consists of a robotic arm attached to a grasping hand.
The camera sits on the robot’s wrist.
The RF reader stands independent of the robot and relays tracking information to the robot’s
control algorithm.
So, the robot is constantly collecting both RF tracking data and a visual picture of its
surroundings.
Integrating these two data streams into the robot’s decision making was one of the biggest
challenges the researchers faced.
The robot has to decide, at each point in time, which of these streams is more important
to think about.
The researcher says that it’s not just eye-hand coordination, it’s RF-eye-hand coordination.
So, the problem gets very complicated.
The robot initiates the seek-and-pluck process by pinging the target object’s RF tag for
a sense of its whereabouts.
It starts by using RF to focus the attention of vision.
Then use vision to navigate fine maneuvers.
The sequence is akin to hearing a siren from behind, then turning to look and get a clearer
picture of the siren’s source.
With its two complementary senses, RF Grasp zeroes in on the target object.
As it gets closer and even starts manipulating the item, vision, which provides much finer
detail than RF, dominates the robot’s decision making.
RF Grasp proved its efficiency in a battery of tests.
Compared to a similar robot equipped with only a camera, RF Grasp was able to pinpoint
and grab its target object with about half as much total movement.
Plus, RF Grasp displayed the unique ability to “declutter” its environment — removing
packing materials and other obstacles in its way in order to access the target.
This demonstrates RF Grasp’s “unfair advantage” over robots without penetrative RF sensing.
RF Grasp could one day perform fulfilment in packed e-commerce warehouses.
Its RF sensing could even instantly verify an item’s identity without the need to manipulate
the item, expose its barcode, then scan it.
The researchers also envision potential home applications for the robot, like locating
the right Allen wrench to assemble Ikea chair.
Or we could imagine the robot finding lost items.