Online Learning with Stochastic Recurrent Neural Networks using Intrinsic Motivation Signals

Continuous online adaptation is an essential ability for the vision of fully autonomous and lifelong-learning robots. Robots need to be able to adapt to changing environments and constraints while this adaption should be performed without interrupting the robot’s motion. In this paper, we introduce a framework for probabilistic online motion planning and learning based on a bio-inspired stochastic recurrent neural network. Furthermore, we show that the model can adapt online and sample-efficiently using intrinsic motivation signals and a mental replay strategy. This fast adaptation behavior allows the robot to learn from only a small number of physical interactions and is a promising feature for reusing the model in different environments. We evaluate the online planning with a realistic dynamic simulation of the KUKA LWR robotic arm. The efficient online adaptation is shown in simulation by learning an unknown workspace constraint using mental replay and cognitive dissonance as intrinsic motivation signal.

[1]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[2]  Masaki Ogino,et al.  Cognitive Developmental Robotics: A Survey , 2009, IEEE Transactions on Autonomous Mental Development.

[3]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[4]  Jürgen Schmidhuber,et al.  Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) , 2010, IEEE Transactions on Autonomous Mental Development.

[5]  T. Martin McGinnity,et al.  Novelty Detection as an Intrinsic Motivation for Cumulative Learning Robots , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[6]  Brad E. Pfeiffer,et al.  Hippocampal place cell sequences depict future paths to remembered goals , 2013, Nature.

[7]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[8]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[9]  J. Kagan.  Motives and development. , 1972, Journal of personality and social psychology.

[10]  Jan Peters,et al.  Deep spiking networks for model-based planning in humanoids , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[11]  Sebastian Thrun,et al.  Lifelong robot learning , 1993, Robotics Auton. Syst..

[12]  Stephen Hart,et al.  Learning Generalizable Control Programs , 2011, IEEE Transactions on Autonomous Mental Development.

[13]  Marco Mirolli,et al.  Intrinsically Motivated Learning Systems: An Overview , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[14]  Giulio Sandini,et al.  Developmental robotics: a survey , 2003, Connect. Sci..

[15]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[16]  Jürgen Schmidhuber,et al.  Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts , 2006, Connect. Sci..

[17]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[18]  Juyang Weng,et al.  Developmental Robotics: Theory and Experiments , 2004, Int. J. Humanoid Robotics.

[19]  Jan Peters,et al.  Recurrent Spiking Networks Solve Planning Tasks , 2016, Scientific Reports.

[20]  L. Festinger.  Cognitive dissonance. , 1962, Scientific American.

[21]  David J. Foster,et al.  Reverse replay of behavioural sequences in hippocampal place cells during the awake state , 2006, Nature.

[22]  Pierre-Yves Oudeyer,et al.  Intrinsic motivation, curiosity, and learning: Theory and applications in educational technologies. , 2016, Progress in brain research.