On-line learning and modulation of periodic movements with nonlinear dynamical systems

The paper presents a two-layered system for (1) learning and encoding a periodic signal without any knowledge on its frequency and waveform, and (2) modulating the learned periodic trajectory in response to external events. The system is used to learn periodic tasks on a humanoid HOAP-2 robot. The first layer of the system is a dynamical system responsible for extracting the fundamental frequency of the input signal, based on adaptive frequency oscillators. The second layer is a dynamical system responsible for learning of the waveform based on a built-in learning algorithm. By combining the two dynamical systems into one system we can rapidly teach new trajectories to robots without any knowledge of the frequency of the demonstration signal. The system extracts and learns only one period of the demonstration signal. Furthermore, the trajectories are robust to perturbations and can be modulated to cope with a dynamic environment. The system is computationally inexpensive, works on-line for any periodic signal, requires no additional signal processing to determine the frequency of the input signal and can be applied in parallel to multiple dimensions. Additionally, it can adapt to changes in frequency and shape, e.g. to non-stationary signals, such as hand-generated signals and human demonstrations.

[1]  Aude Billard,et al.  Dynamical System Modulation for Robot Learning via Kinesthetic Demonstrations , 2008, IEEE Transactions on Robotics.

[2]  Jun Nishii,et al.  A learning model for oscillatory networks , 1998, Neural Networks.

[3]  Andrej Gams,et al.  Imitating human acceleration of a gyroscopic device , 2007, Robotica.

[4]  Aude Billard,et al.  A Simple, Adaptive Locomotion Toy-System , 2004 .

[5]  Auke Jan Ijspeert,et al.  AmphiBot I: an amphibious snake-like robot , 2005, Robotics Auton. Syst..

[6]  Daniel Bullock,et al.  Chapter 11 Vite and Flete: Neural Modules for Trajectory Formation and Postural Control , 1989 .

[7]  M. Matarić Behavior-based robotics as a tool for synthesis of artificial behavior and analysis of natural behavior , 1998, Trends in Cognitive Sciences.

[8]  Mitsuo Kawato,et al.  TRAJECTORY FORMATION IN ARM MOVEMENTS: MINIMIZATION PRINCIPLES AND PROCEDURES , 1996 .

[9]  Lennart Ljung,et al.  Theory and Practice of Recursive Identification , 1983 .

[10]  Christopher G. Atkeson,et al.  Constructive Incremental Learning from Only Local Information , 1998, Neural Computation.

[11]  Ferdinando A. Mussa-Ivaldi,et al.  Nonlinear force fields: a distributed system of control primitives for representing and learning movements , 1997, Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation'.

[12]  Stefan Schaal,et al.  Dynamics systems vs. optimal control--a unifying view. , 2007, Progress in brain research.

[13]  Gordon Cheng,et al.  Synthesizing goal-directed actions from a library of example movements , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[14]  Yoshiyuki Tanaka,et al.  Bio-mimetic trajectory generation of robots via artificial potential field with time base generator , 2002, IEEE Trans. Syst. Man Cybern. Part C.

[15]  Thomas Hofmann,et al.  Learning Nonparametric Models for Probabilistic Imitation , 2007 .

[16]  Martin A. Giese,et al.  Trajectory synthesis by hierarchical spatio-temporal correspondence: comparison of different methods , 2005, APGV '05.

[17]  Maja J. Mataric,et al.  Automated Derivation of Primitives for Movement Classification , 2000, Auton. Robots.

[18]  Yasuharu Koike,et al.  PII: S0893-6080(96)00043-3 , 1997 .

[19]  Jun Tani,et al.  Adaptive Motor Primitive and Sequence Formation in a Hierarchical Recurrent Neural Network , 2004 .

[20]  Perry Y. Li,et al.  Passive velocity field control of mechanical manipulators , 1995, IEEE Trans. Robotics Autom..

[21]  Ludovic Righetti,et al.  Engineering entrainment and adaptation in limit cycle systems , 2006, Biological Cybernetics.

[22]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[23]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[24]  Maja J. Mataric,et al.  Exemplar-based primitives for humanoid movement classification and control , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[25]  Yoshihiko Nakamura,et al.  Polynomial design of the nonlinear dynamics for the brain-like information processing of whole body motion , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[26]  Neville Hogan,et al.  Avoiding spurious submovement decompositions: a globally optimal algorithm , 2003, Biological Cybernetics.

[27]  Malur K. Sundareshan,et al.  Trajectory generation and modulation using dynamic neural networks , 2003, IEEE Trans. Neural Networks.

[28]  Jun Nakanishi,et al.  Learning Attractor Landscapes for Learning Motor Primitives , 2002, NIPS.

[29]  Daniel M. Wolpert,et al.  Signal-dependent noise determines motor planning , 1998, Nature.

[30]  John Hallam,et al.  Evolving Swimming Controllers for a Simulated Lamprey with Inspiration from Neurobiology , 1999, Adapt. Behav..

[31]  A. Ijspeert,et al.  Dynamic hebbian learning in adaptive frequency oscillators , 2006 .

[32]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[33]  Aude Billard,et al.  Reinforcement learning for imitating constrained reaching movements , 2007, Adv. Robotics.

[34]  Yoshihiko Nakamura,et al.  Embodied Symbol Emergence Based on Mimesis Theory , 2004, Int. J. Robotics Res..

[35]  Sethu Vijayakumar,et al.  Adaptive Optimal Control for Redundantly Actuated Arms , 2008, SAB.

[36]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[37]  Jun Tani,et al.  Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[38]  Jun Nakanishi,et al.  Learning rhythmic movements by demonstration using nonlinear oscillators , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[40]  Ludovic Righetti,et al.  Programmable central pattern generators: an application to biped locomotion control , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[41]  Auke Jan Ijspeert,et al.  Central pattern generators for locomotion control in animals and robots: A review , 2008, Neural Networks.

[42]  Yann LeCun,et al.  Reverse TDNN: An Architecture For Trajectory Generation , 1991, NIPS.

[43]  Howard N. Zelaznik,et al.  Advances in Motor Learning and Control , 1996 .