ImitationFlow: Learning Deep Stable Stochastic Dynamic Systems by Normalizing Flows

We introduce ImitationFlow, a novel Deep generative model that allows learning complex globally stable, stochastic, nonlinear dynamics. Our approach extends the Normalizing Flows framework to learn stable Stochastic Differential Equations. We prove the Lyapunov stability for a class of Stochastic Differential Equations and we propose a learning algorithm to learn them from a set of demonstrated trajectories. Our model extends the set of stable dynamical systems that can be represented by state-of-the-art approaches, eliminates the Gaussian assumption on the demonstrations, and outperforms the previous algorithms in terms of representation accuracy. We show the effectiveness of our method with both standard datasets and a real robot experiment.

[1]  Stefan Schaal,et al.  Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning , 2002, Applied Intelligence.

[2]  Andrew W. Moore,et al.  Locally Weighted Learning , 1997, Artificial Intelligence Review.

[3]  S. Schaal.  Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics , 2006 .

[4]  Aude Billard,et al.  Dynamical System Modulation for Robot Learning via Kinesthetic Demonstrations , 2008, IEEE Transactions on Robotics.

[5]  Nicola Bruti-Liberati.  Numerical Solution of Stochastic Differential Equations with Jumps in Finance , 2010 .

[6]  Aude Billard,et al.  Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models , 2011, IEEE Transactions on Robotics.

[7]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[8]  Klaus Neumann,et al.  Neurally imprinted stable vector fields , 2013, ESANN.

[9]  Klaus Neumann,et al.  Neural learning of stable dynamical systems based on data-driven Lyapunov candidates , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[11]  Aude Billard,et al.  Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions , 2014, Robotics Auton. Syst..

[12]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[13]  Klaus Neumann,et al.  Learning robot motions with stable dynamical systems under diffeomorphic transformations , 2015, Robotics Auton. Syst..

[14]  Ashwin P. Dani,et al.  Learning Contracting Nonlinear Dynamics From Human Demonstration for Robot Motion Planning , 2015, HRI 2015.

[15]  Nicolas Perrin,et al.  Fast diffeomorphic matching to learn globally asymptotically stable nonlinear dynamical systems , 2016, Syst. Control. Lett..

[16]  Jessica Fuerst,et al.  Stochastic Differential Equations And Applications , 2016 .

[17]  Samy Bengio,et al.  Density estimation using Real NVP , 2016, ICLR.

[18]  Uri Shalit,et al.  Structured Inference Networks for Nonlinear State Space Models , 2016, AAAI.

[19]  Iain Murray,et al.  Masked Autoregressive Flow for Density Estimation , 2017, NIPS.

[20]  Sandra Hirche,et al.  Learning Stable Stochastic Nonlinear Dynamical Systems , 2017, ICML.

[21]  Stefan Schaal,et al.  Learning from Demonstration , 1996, NIPS.

[22]  Ashwin P. Dani,et al.  Learning Partially Contracting Dynamical Systems from Demonstrations , 2017, CoRL.

[23]  Prafulla Dhariwal,et al.  Glow: Generative Flow with Invertible 1x1 Convolutions , 2018, NeurIPS.

[24]  Vikas Sindhwani,et al.  Learning Contracting Vector Fields For Stable Imitation Learning , 2018, ArXiv.

[25]  R. Haftka,et al.  Similarity measures for identifying material parameters from hysteresis loops using inverse analysis , 2018, International Journal of Material Forming.

[26]  J. Zico Kolter,et al.  Learning Stable Deep Dynamics Models , 2020, NeurIPS.

[27]  Darwin G. Caldwell,et al.  Kernelized movement primitives , 2017, Int. J. Robotics Res..

[28]  H. Razaghi.  Filtering Normalizing Flows , 2019 .

[29]  S. Levine,et al.  VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation , 2019, ICLR.