Learning Continuous Grasp Affordances by Sensorimotor Exploration

We develop means of learning and representing object grasp affordances probabilistically. By grasp affordance, we refer to an entity that is able to assess whether a given relative object-gripper configuration will yield a stable grasp. These affordances are represented with grasp densities, continuous probability density functions defined on the space of 3D positions and orientations. Grasp densities are registered with a visual model of the object they characterize. They are exploited by aligning them to a target object using visual pose estimation. Grasp densities are refined through experience: A robot “plays” with an object by executing grasps drawn randomly for the object’s grasp density. The robot then uses the outcomes of these grasps to build a richer density through an importance sampling mechanism. Initial grasp densities, called hypothesis densities, are bootstrapped from grasps collected using a motion capture system, or from grasps generated from the visual model of the object. Refined densities, called empirical densities, represent affordances that have been confirmed through physical experience. The applicability of our method is demonstrated by producing empirical densities for two object with a real robot and its 3-finger hand. Hypothesis densities are created from visual cues and human demonstration.

[1]  Justus H. Piater,et al.  A Probabilistic Framework for 3D Visual Object Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Joachim Hertzberg,et al.  Towards Affordance-based Robot Control , 2008 .

[3]  Danica Kragic,et al.  Birth of the Object: Detection of Objectness and Extraction of Object Shape through Object-Action complexes , 2008, Int. J. Humanoid Robotics.

[4]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[5]  Guillaume Bouchard,et al.  Hierarchical part-based visual object categorization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[6]  Danica Kragic,et al.  Early reactive grasping with second order 3D feature relations , 2007 .

[7]  Michael Isard,et al.  Nonparametric belief propagation , 2010, Commun. ACM.

[8]  Markus Lappe,et al.  Biologically Motivated Multi-modal Processing of Visual Primitives , 2003 .

[9]  Justus Piater,et al.  Hierarchical Integration of Local 3D Features for Probabilistic Pose Recovery , 2007 .

[10]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[11]  Markus Vincze,et al.  Efficient 3D Object Detection by Fitting Superquadrics to Range Image Data for Robot's Object Manipulation , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[12]  Danica Kragic,et al.  Minimum volume bounding box decomposition for shape approximation in robot grasping , 2008, 2008 IEEE International Conference on Robotics and Automation.

[13]  Alexander Stoytchev,et al.  Learning the Affordances of Tools Using a Behavior-Grounded Approach , 2006, Towards Affordance-Based Robot Control.

[14]  Daniel P. Huttenlocher,et al.  Efficient matching of pictorial structures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  Oliver Kroemer,et al.  Active learning using mean shift optimization for robot grasping , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  James J. Kuffner,et al.  Effective sampling and distance metrics for 3D rigid body path planning , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[17]  Henrik I. Christensen,et al.  Automatic grasp planning using shape primitives , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[18]  Olivier Sigaud,et al.  From Motor Learning to Interaction Learning in Robots , 2010, From Motor Learning to Interaction Learning in Robots.

[19]  S. R. Jammalamadaka,et al.  Directional Statistics, I , 2011 .

[20]  Manuel Lopes,et al.  Learning grasping affordances from local visual descriptors , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[21]  E. Reed.  The Ecological Approach to Visual Perception , 1989 .

[22]  Danica Kragic,et al.  Real-time tracking meets online grasp planning , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[23]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[24]  Neil J. Gordon,et al.  Editors: Sequential Monte Carlo Methods in Practice , 2001 .

[25]  José Santos-Victor,et al.  Abstraction Levels for Robotic Imitation: Overview and Computational Approaches , 2010, From Motor Learning to Interaction Learning in Robots.

[26]  A. Stoytchev.  Toward Learning the Binding Affordances of Objects : A Behavior-Grounded Approach , 2022 .

[27]  A. Fagg,et al.  Learning Grasp Affordances Through Human Demonstration , 2008 .

[28]  N. Kruger,et al.  Learning object-specific grasp affordance densities , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[29]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[30]  Eiichi Yoshida,et al.  Human-Robot Cooperation Based on Interaction Learning , 2010, From Motor Learning to Interaction Learning in Robots.

[31]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[32]  Nicolas Pugeault,et al.  Early cognitive vision: feedback mechanisms for the disambiguation of early visual representation , 2008 .