A new data source for inverse dynamics learning

Modern robotics is gravitating toward increasingly collaborative human robot interaction. Tools such as acceleration policies can naturally support the realization of reactive, adaptive, and compliant robots. These tools require us to model the system dynamics accurately — a difficult task. The fundamental problem remains that simulation and reality diverge-we do not know how to accurately change a robot's state. Thus, recent research on improving inverse dynamics models has been focused on making use of machine learning techniques. Traditional learning techniques train on the actual realized accelerations, instead of the policy's desired accelerations, which is an indirect data source. Here we show how an additional training signal — measured at the desired accelerations — can be derived from a feedback control signal. This effectively creates a second data source for learning inverse dynamics models. Furthermore, we show how both the traditional and this new data source, can be used to train task-specific models of the inverse dynamics, when used independently or combined. We analyze the use of both data sources in simulation and demonstrate its effectiveness on a real-world robotic platform. We show that our system incrementally improves the learned inverse dynamics model, and when using both data sources combined converges more consistently and faster.

[1]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Lorenzo Jamone,et al.  Incremental learning of context-dependent dynamic internal models for robot control , 2014, 2014 IEEE International Symposium on Intelligent Control (ISIC).

[3]  S. Shankar Sastry,et al.  Adaptive Control of Mechanical Manipulators , 1987, Proceedings. 1986 IEEE International Conference on Robotics and Automation.

[4]  Giorgio Metta,et al.  Real-time model learning using Incremental Sparse Spectrum Gaussian Process Regression. , 2013, Neural networks : the official journal of the International Neural Network Society.

[5]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[6]  Giorgio Metta,et al.  Incremental semiparametric inverse dynamics learning , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Stefan Schaal,et al.  Incremental Local Gaussian Regression , 2014, NIPS.

[8]  Jan Peters,et al.  Model learning for robot control: a survey , 2011, Cognitive Processing.

[9]  Andrej Gams,et al.  Online learning of task-specific dynamics for periodic tasks , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Dana Kulic,et al.  On-Line Dynamic Model Learning for Manipulator Control , 2012, SyRoCo.

[11]  Marc Toussaint,et al.  Learning discontinuities with products-of-sigmoids for switching between local models , 2005, ICML.

[12]  Robert F. Stengel,et al.  Optimal Control and Estimation , 1994 .

[13]  Duy Nguyen-Tuong,et al.  Local Gaussian Process Regression for Real Time Online Model Learning , 2008, NIPS.

[14]  C. Atkeson,et al.  Estimation of inertial parameters of rigid body links of manipulators , 1985, 1985 24th IEEE Conference on Decision and Control.

[15]  Jan Peters,et al.  Using model knowledge for learning inverse dynamics , 2010, 2010 IEEE International Conference on Robotics and Automation.

[16]  Marc Toussaint,et al.  Learning Multiple Models of Non-linear Dynamics for Control Under Varying Contexts , 2006, ICANN.

[17]  Stefan Schaal,et al.  Towards robust online inverse dynamics learning , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Olivier Sigaud,et al.  On-line regression algorithms for learning mechanical models of robots: A survey , 2011, Robotics Auton. Syst..

[19]  John Hollerbach,et al.  Rigid body load identification for manipulators , 1985, 1985 24th IEEE Conference on Decision and Control.

[20]  Michael N. Mistry,et al.  Computed torque control with variable gains through Gaussian process regression , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[21]  Stefan Schaal,et al.  DOOMED: Direct Online Optimization of Modeling Errors in Dynamics , 2016, Big Data.

[22]  Jun Nakanishi,et al.  Feedback error learning and nonlinear adaptive control , 2004, Neural Networks.

[23]  Stefan Schaal,et al.  Locally Weighted Projection Regression: Incremental Real Time Learning in High Dimensional Space , 2000, ICML.

[24]  Wojciech Zaremba,et al.  Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model , 2016, ArXiv.

[25]  Jan Peters,et al.  Learning inverse dynamics models with contacts , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).