Carnegie Mellon (CMU) roboticists have developed a new technique that allows robots to grasp transparent and reflective objects.
Robots utilize depth cameras which shine infrared light on an object to determine its shape. This works well for identifying opaque objects.
But when it comes to transparent and reflective objects, the infrared light passes right through them and scatters off reflective surfaces.
Aptly called a ‘robot’s nightmare’, depth cameras cannot calculate an accurate shape and this results in largely flat or hole-riddled shapes for transparent and reflective objects.
Now a CMU team led by assistant professor David Held has developed a new technique that they say could help robots grasp non-opaque objects.
The technique doesn’t require fancy sensors, exhaustive training or human guidance, but relies primarily on a commercial RGB-D camera that’s capable of both color images (RGB) and depth images (D).
The new color camera system recognizes shapes based on color. It has been trained to imitate the depth camera system and implicitly infer shape to grasp objects.
Once trained, the color camera system was applied to transparent and shiny objects and based on those images – along with whatever scant information a depth camera could provide, the system was able to grasp these challenging objects with a high degree of success.
“We do sometimes miss,” acknowledges Held, “but for the most part it did a pretty good job, much better than any previous system for grasping transparent or reflective objects.”
Though the system can’t pick up transparent or reflective objects as efficiently as opaque objects, it is far more successful than depth camera systems alone, notes robotics PhD student Thomas Weng.
“Our system not only can pick up individual transparent and reflective objects, but it can also grasp such objects in cluttered piles,” adds Weng.
The multimodal transfer learning used to train the system was also so effective that the color system proved almost as good as the depth camera system at picking up opaque objects.
Image and content: Carnegie Mellon University