A Differentiable Newton-Euler Algorithm for Real-World Robotics

Obtaining dynamics models is essential for robotics to achieve accurate model-based controllers and simulators for planning. The dynamics models are typically obtained using model specification of the manufacturer or simple numerical methods such as linear regression. However, this approach does not guarantee physically plausible parameters and can only be applied to kinematic chains consisting of rigid bodies. In this article, we describe a differentiable simulator that can be used to identify the system parameters of real-world mechanical systems with complex friction models, holonomic as well as non-holonomic constraints. To guarantee physically consistent parameters, we utilize virtual parameters and gradient-based optimization. The described Differentiable Newton-Euler Algorithm (DiffNEA) can be applied to a class of dynamical systems and guarantees physically plausible predictions. The extensive experimental evaluation shows, that the proposed model learning approach learns accurate dynamics models of systems with complex friction and non-holonomic constraints. Especially in the offline reinforcement learning experiments, the identified DiffNEA models excel. For the challenging ball in a cup task, these models solve the task using model-based offline reinforcement learning on the physical system. The black-box baselines fail on this task in simulation and on the physical system despite using more data for learning the model.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Joshua B. Tenenbaum,et al.  End-to-End Differentiable Physics for Learning and Control , 2018, NeurIPS.

[3]  Jan Peters,et al.  A Survey on Policy Search for Robotics , 2013, Found. Trends Robotics.

[4]  Jan Peters,et al.  Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning , 2019, ICLR.

[5]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[6]  Yuval Tassa,et al.  Learning Dynamics Models for Model Predictive Agents , 2021, ArXiv.

[7]  Adrien Escande,et al.  Identification of fully physical consistent inertial parameters using optimization on manifolds , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  M. Spong,et al.  Robot Modeling and Control , 2005 .

[9]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[10]  Frédo Durand,et al.  DiffTaichi: Differentiable Programming for Physical Simulation , 2020, ICLR.

[11]  Andrea L. Thomaz,et al.  TuneNet: One-Shot Residual Tuning for System Identification and Sim-to-Real Robot Task Transfer , 2019, CoRL.

[12]  Martin A. Riedmiller,et al.  Batch Reinforcement Learning , 2012, Reinforcement Learning.

[13]  Sergey Levine,et al.  Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models , 2018, NeurIPS.

[14]  Mohammad Norouzi,et al.  Dream to Control: Learning Behaviors by Latent Imagination , 2019, ICLR.

[15]  Austin Wang,et al.  Encoding Physical Constraints in Differentiable Newton-Euler Algorithm , 2020, L4DC.

[16]  Marc Peter Deisenroth,et al.  Variational Integrator Networks for Physically Structured Embeddings , 2020, AISTATS.

[17]  Jan Peters,et al.  High Acceleration Reinforcement Learning for Real-World Juggling with Binary Rewards , 2020, CoRL.

[18]  Jan Peters,et al.  Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learning , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Roy Featherstone,et al.  Rigid Body Dynamics Algorithms , 2007 .

[20]  Dieter Fox,et al.  BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators , 2019, Robotics: Science and Systems.

[21]  Kim D. Listmann,et al.  Deep Lagrangian Networks for end-to-end learning of energy-based control for under-actuated systems , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Takeo Kanade,et al.  Parameter identification of robot dynamics , 1985, 1985 24th IEEE Conference on Decision and Control.

[23]  Gaurav S. Sukhatme,et al.  Interactive Differentiable Simulation , 2019, ArXiv.

[24]  Jean-Jacques E. Slotine,et al.  Linear Matrix Inequalities for Physically Consistent Inertial Parameter Identification: A Statistical Perspective on the Mass Distribution , 2017, IEEE Robotics and Automation Letters.

[25]  Raia Hadsell,et al.  Graph networks as learnable physics engines for inference and control , 2018, ICML.

[26]  Jan Peters,et al.  Using model knowledge for learning inverse dynamics , 2010, 2010 IEEE International Conference on Robotics and Automation.

[27]  Jan Peters,et al.  A Differentiable Newton Euler Algorithm for Multi-body Model Learning , 2020, ArXiv.

[28]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[29]  Amitabha Mukerjee,et al.  Self-calibration in robot manipulators , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[30]  Vicente Mata,et al.  Dynamic parameter identification in industrial robots considering physical feasibility , 2005, Adv. Robotics.

[31]  Karl Johan Åström,et al.  BOOK REVIEW SYSTEM IDENTIFICATION , 1994, Econometric Theory.

[32]  Bernhard Thomaszewski,et al.  ADD , 2020, ACM Trans. Graph..

[33]  Mykel J. Kochenderfer,et al.  A General Framework for Structured Learning of Mechanical Systems , 2019, ArXiv.

[34]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[35]  Christopher G. Atkeson,et al.  Estimation of Inertial Parameters of Manipulator Loads and Links , 1986 .

[36]  Miles Cranmer,et al.  Lagrangian Neural Networks , 2020, ICLR 2020.

[37]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Arne Wahrburg,et al.  Motor-Current-Based Estimation of Cartesian Contact Forces and Torques for Robotic Manipulators and Its Application to Force Control , 2018, IEEE Transactions on Automation Science and Engineering.

[39]  Jun Nakanishi,et al.  A Bayesian Approach to Nonlinear Parameter Identification for Rigid Body Dynamics , 2006, Robotics: Science and Systems.

[40]  Jure Leskovec,et al.  Learning to Simulate Complex Physics with Graph Networks , 2020, ICML.

[41]  K. Jarrod Millman,et al.  Array programming with NumPy , 2020, Nat..

[42]  C. Karen Liu,et al.  Fast and Feature-Complete Differentiable Physics for Articulated Rigid Bodies with Contact , 2021, ArXiv.

[43]  Gentiane Venture,et al.  Identification of consistent standard dynamic parameters of industrial robots , 2013, 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[44]  Sergey Levine,et al.  SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[45]  Sebastian Trimpe,et al.  Structured learning of rigid‐body dynamics: A survey and unified view from a robotics perspective , 2020, GAMM-Mitteilungen.

[46]  Jan Peters,et al.  Using probabilistic movement primitives in robotics , 2017, Autonomous Robots.

[47]  Joonho Lee,et al.  Learning agile and dynamic motor skills for legged robots , 2019, Science Robotics.

[48]  Carlos Canudas de Wit,et al.  Friction Models and Friction Compensation , 1998, Eur. J. Control.

[49]  A. Albu-Schäffer Regelung von Robotern mit elastischen Gelenken am Beispiel der DLR-Leichtbauarme , 2002 .

[50]  Jonas Degrave,et al.  A DIFFERENTIABLE PHYSICS ENGINE FOR DEEP LEARNING IN ROBOTICS , 2016, Front. Neurorobot..

[51]  Yashraj S. Narang,et al.  DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cutting , 2021, Robotics: Science and Systems.

[52]  Gaurav S. Sukhatme,et al.  Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gap , 2020, ArXiv.

[53]  Eduardo D. Sontag,et al.  Neural Networks for Control , 1993 .

[54]  Junggon Kim Lie Group Formulation of Articulated Rigid Body Dynamics , 2012 .

[55]  Gentiane Venture,et al.  Real-time implementation of physically consistent identification of human body segments , 2011, 2011 IEEE International Conference on Robotics and Automation.

[56]  Lionel Ott,et al.  DISCO: Double Likelihood-free Inference Stochastic Control , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[57]  J. Kocijan,et al.  Gaussian process model based predictive control , 2004, Proceedings of the 2004 American Control Conference.

[58]  Amit Chakraborty,et al.  Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control , 2020, ICLR.

[59]  Duy Nguyen-Tuong,et al.  Computed torque control with nonparametric regression models , 2008, 2008 American Control Conference.

[60]  Yasemin Altun,et al.  Relative Entropy Policy Search , 2010 .

[61]  Stefan Schaal,et al.  Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning , 2002, Applied Intelligence.

[62]  Rui Pedro Duarte Cortesão,et al.  Physical feasibility of robot base inertial parameter identification: A linear matrix inequality approach , 2014, Int. J. Robotics Res..

[63]  Jason Yosinski,et al.  Hamiltonian Neural Networks , 2019, NeurIPS.

[64]  Jan Peters,et al.  Model Learning with Local Gaussian Process Regression , 2009, Adv. Robotics.

[65]  Andrew W. Moore,et al.  Locally Weighted Learning for Control , 1997, Artificial Intelligence Review.

[66]  Boris Belousov,et al.  Neural Posterior Domain Randomization , 2021, CoRL.

[67]  S. Levine,et al.  Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems , 2020, ArXiv.

[68]  Maxime Gautier,et al.  Identification of Robots Dynamics , 1986 .

[69]  M. Indri,et al.  Friction Compensation in Robotics: an Overview , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[70]  Jan Peters,et al.  Model learning for robot control: a survey , 2011, Cognitive Processing.

[71]  Koji Yoshida,et al.  Verification of the Positive Definiteness of the Inertial Matrix of Manipulators Using Base Inertial Parameters , 2000, Int. J. Robotics Res..

[72]  Gentiane Venture,et al.  Identification of standard dynamic parameters of robots with positive definite inertia matrix , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.