Generalization of human grasping for multi-fingered robot hands

Multi-fingered robot grasping is a challenging problem that is difficult to tackle using hand-coded programs. In this paper we present an imitation learning approach for learning and generalizing grasping skills based on human demonstrations. To this end, we split the task of synthesizing a grasping motion into three parts: (1) learning efficient grasp representations from human demonstrations, (2) warping contact points onto new objects, and (3) optimizing and executing the reach-and-grasp movements. We learn low-dimensional latent grasp spaces for different grasp types, which form the basis for a novel extension to dynamic motor primitives. These latent-space dynamic motor primitives are used to synthesize entire reach-and-grasp movements. We evaluated our method on a real humanoid robot. The results of the experiment demonstrate the robustness and versatility of our approach.

[1]  E. Bizzi,et al.  Human arm trajectory formation. , 1982, Brain : a journal of neurology.

[2]  M. Jeannerod The timing of natural prehension movements. , 1984, Journal of motor behavior.

[3]  M. Arbib Coordinated control programs for movements of the hand , 1985 .

[4]  M. Schieber How might the motor cortex individuate movements? , 1990, Trends in Neurosciences.

[5]  M. Jeannerod,et al.  Grasping Objects: The Hand as a Pattern Recognition Device , 1996 .

[6]  J. F. Soechting,et al.  Postural Hand Synergies for Tool Use , 1998, The Journal of Neuroscience.

[7]  Robert J. Vanderbei,et al.  Linear Programming: Foundations and Extensions , 1998, Kluwer international series in operations research and management service.

[8]  J. F. Soechting,et al.  Gradual molding of the hand to object contours. , 1998, Journal of neurophysiology.

[9]  R. Lemon Neural control of dexterity: what has been achieved? , 1999, Experimental Brain Research.

[10]  Carolyn R. Mason,et al.  Hand synergies during reach-to-grasp. , 2001, Journal of neurophysiology.

[11]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[12]  K. Dautenhahn,et al.  Imitation in Animals and Artifacts , 2002 .

[13]  Carolyn R. Mason,et al.  Monkey hand postural synergies during reach-to-grasp in the absence of vision of the hand and object. , 2004, Journal of neurophysiology.

[14]  Jun Morimoto,et al.  Learning from demonstration and adaptation of biped locomotion , 2004, Robotics Auton. Syst..

[15]  S. Chieffi,et al.  Coordination between the transport and the grasp components during prehension movements , 2004, Experimental Brain Research.

[16]  Raúl Suárez Feijóo,et al.  Grasp quality measures , 2006 .

[17]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[18]  Heni Ben Amor,et al.  Grasp synthesis from low‐dimensional probabilistic grasp models , 2008, Comput. Animat. Virtual Worlds.

[19]  Betty J. Mohler,et al.  Learning perceptual coupling for motor primitives , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Florian Schmidt,et al.  Rollin' Justin - Mobile platform with variable base , 2009, 2009 IEEE International Conference on Robotics and Automation.

[21]  D. Nowak,et al.  Sensorimotor Control of Grasping: Physiology and Pathophysiology , 2009 .

[22]  Danica Kragic,et al.  Demonstration-based learning and control for automatic grasping , 2009, Intell. Serv. Robotics.

[23]  Matei T. Ciocarlie,et al.  Hand Posture Subspaces for Dexterous Robotic Grasping , 2009, Int. J. Robotics Res..

[24]  Ulrich Hillenbrand Non-parametric 3D Shape Warping , 2010, 2010 20th International Conference on Pattern Recognition.

[25]  Marc Toussaint,et al.  A Bayesian View on Motor Control and Planning , 2010, From Motor Learning to Interaction Learning in Robots.

[26]  Guido Heumer Simulation, Erfassung und Analyse direkter Objektmanipulationen in virtuellen Umgebungen , 2010 .

[27]  Heni Ben Amor,et al.  Imitation Learning of Motor Skills for Synthetic Humanoids , 2010, Ausgezeichnete Informatikdissertationen.

[28]  Alexander Fuchs,et al.  An experimental study of four variants of pose clustering from dense range data , 2011, Comput. Vis. Image Underst..

[29]  Oliver Kroemer,et al.  Learning robot grasping from 3-D images with Markov Random Fields , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  N. Hatsopoulos,et al.  Encoding of Coordinated Reach and Grasp Trajectories in Primary Motor Cortex , 2012, The Journal of Neuroscience.

[31]  Máximo A. Roa,et al.  Transferring functional grasps through contact warping and local replanning , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.