Learning robot in-hand manipulation with tactile features

Dexterous manipulation enables repositioning of objects and tools within a robot's hand. When applying dexterous manipulation to unknown objects, exact object models are not available. Instead of relying on models, compliance and tactile feedback can be exploited to adapt to unknown objects. However, compliant hands and tactile sensors add complexity and are themselves difficult to model. Hence, we propose acquiring in-hand manipulation skills through reinforcement learning, which does not require analytic dynamics or kinematics models. In this paper, we show that this approach successfully acquires a tactile manipulation skill using a passively compliant hand. Additionally, we show that the learned tactile skill generalizes to novel objects.

[1]  Antonio Bicchi,et al.  Dexterous manipulation through rolling , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[2]  Kazuo Tanie,et al.  Tactile sensor based manipulation of an unknown object by a multifingered hand with rolling contact , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[3]  Jeffrey C. Trinkle,et al.  Dextrous manipulation by rolling and finger gaiting , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[4]  Kamal K. Gupta,et al.  Planning quasi-static fingertip manipulations for reconfiguring objects , 1999, IEEE Trans. Robotics Autom..

[5]  Hong Zhang,et al.  Control of contact via tactile sensing , 2000, IEEE Trans. Robotics Autom..

[6]  Emil M. Petriu,et al.  Intelligent haptic sensor system for robotic manipulation , 2004, IMTC 2004.

[7]  Ana-Maria Cretu,et al.  Intelligent haptic sensor system for robotic manipulation , 2004, IEEE Transactions on Instrumentation and Measurement.

[8]  Suguru Arimoto,et al.  Control of an object with parallel surfaces by a pair of finger robots without object sensing , 2005, IEEE Transactions on Robotics.

[9]  Stefan Schaal,et al.  Reinforcement learning by reward-weighted regression for operational space control , 2007, ICML '07.

[10]  Masatsugu Iribe,et al.  Adaptive grasping by multi fingered hand with tactile sensor based on robust force and position control , 2008, 2008 IEEE International Conference on Robotics and Automation.

[11]  Zhiwei Luo,et al.  On iterative learning control for simultaneous force/position trajectory tracking by using a 5 D.O.F. robotic thumb under non-holonomic rolling constraints , 2008, 2008 IEEE International Conference on Robotics and Automation.

[12]  Oliver Brock,et al.  Learning to Manipulate Articulated Objects in Unstructured Environments Using a Grounded Relational Representation , 2008, Robotics: Science and Systems.

[13]  Matei T. Ciocarlie,et al.  Contact-reactive grasping of objects with partial shape information , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Suguru Arimoto,et al.  Dynamic object manipulation using a virtual frame by a triple soft-fingered robotic hand , 2010, 2010 IEEE International Conference on Robotics and Automation.

[15]  Oliver Kroemer,et al.  Combining active learning and reactive control for robot grasping , 2010, Robotics Auton. Syst..

[16]  Yasemin Altun,et al.  Relative Entropy Policy Search , 2010 .

[17]  Aaron M. Dollar,et al.  On dexterity and dexterous manipulation , 2011, 2011 15th International Conference on Advanced Robotics (ICAR).

[18]  Carl E. Rasmussen,et al.  Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning , 2011, Robotics: Science and Systems.

[19]  Sachin Chitta,et al.  Human-Inspired Robotic Grasp Control With Tactile Sensing , 2011, IEEE Transactions on Robotics.

[20]  Stefan Schaal,et al.  Skill learning and task outcome prediction for manipulation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[21]  Jan Peters,et al.  Reinforcement Learning to Adjust Robot Movements to New Situations , 2010, IJCAI.

[22]  Stefan Schaal,et al.  Learning force control policies for compliant manipulation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Jimmy A. Jørgensen,et al.  Assessing Grasp Stability Based on Learning and Haptic Data , 2011, IEEE Transactions on Robotics.

[24]  Stefan Schaal,et al.  Towards Associative Skill Memories , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[25]  Ville Kyrki,et al.  Probabilistic sensor-based grasping , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Zoran Popovic,et al.  Contact-invariant optimization for hand manipulation , 2012, SCA '12.

[27]  Máximo A. Roa,et al.  Planning in-hand object manipulation with multifingered hands considering task constraints , 2013, 2013 IEEE International Conference on Robotics and Automation.

[28]  Helge J. Ritter,et al.  Rotary object dexterous manipulation in hand: a feedback-based method , 2013, Int. J. Mechatronics Autom..

[29]  Zoe Doulgeri,et al.  On rolling contact motion by robotic fingers via prescribed performance control , 2013, 2013 IEEE International Conference on Robotics and Automation.

[30]  Martin A. Riedmiller,et al.  Acquiring visual servoing reaching and grasping skills using neural reinforcement learning , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[31]  Oliver Brock,et al.  A Novel Type of Compliant, Underactuated Robotic Hand for Dexterous Grasping , 2014, Robotics: Science and Systems.

[32]  Robert D. Howe,et al.  Limits to compliance and the role of tactile sensing in grasping , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Oliver Kroemer,et al.  Learning robot tactile sensing for object manipulation , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  C. Karen Liu,et al.  Dexterous manipulation using both palm and fingers , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Peter K. Allen,et al.  Stable grasping under pose uncertainty using tactile feedback , 2014, Auton. Robots.

[36]  Aude Billard,et al.  Learning object-level impedance control for robust grasping and dexterous manipulation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[37]  Danica Kragic,et al.  Learning of grasp adaptation through experience and tactile sensing , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Jan Peters,et al.  Learning of Non-Parametric Control Policies with High-Dimensional State Features , 2015, AISTATS.

[39]  Sergey Levine,et al.  Learning Visual Feature Spaces for Robotic Manipulation with Deep Spatial Autoencoders , 2015, ArXiv.

[40]  Martin A. Riedmiller,et al.  Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images , 2015, NIPS.

[41]  Nolan Wagener,et al.  Learning contact-rich manipulation skills with guided policy search , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[42]  Oliver Kroemer,et al.  Towards learning hierarchical skills for multi-phase manipulation tasks , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[43]  Jan Peters,et al.  Stable reinforcement learning with autoencoders for tactile and visual data , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).