Adaptation and Robust Learning of Probabilistic Movement Primitives

Probabilistic representations of movement primitives open important new possibilities for machine learning in robotics. These representations are able to capture the variability of the demonstrations from a teacher as a probability distribution over trajectories, providing a sensible region of exploration and the ability to adapt to changes in the robot environment. However, to be able to capture variability and correlations between different joints, a probabilistic movement primitive requires the estimation of a larger number of parameters compared to their deterministic counterparts, which focus on modeling only the mean behavior. In this article, we make use of prior distributions over the parameters of a probabilistic movement primitive to make robust estimates of the parameters with few training instances. In addition, we introduce general purpose operators to adapt movement primitives in joint and task space. The proposed training method and adaptation operators are tested in a coffee preparation and in robot table tennis task. In the coffee preparation task we evaluate the generalization performance to changes in the location of the coffee grinder and brewing chamber in a target area, achieving the desired behavior after only two demonstrations. In the table tennis task we evaluate the hit and return rates, outperforming previous approaches while using fewer task specific heuristics.

[1]  Jan Peters,et al.  Using Bayesian Dynamical Systems for Motion Template Libraries , 2008, NIPS.

[2]  Neil D. Lawrence,et al.  Latent Force Models , 2009, AISTATS.

[3]  Jun Morimoto,et al.  Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives , 2010, IEEE Transactions on Robotics.

[4]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[5]  Jun Morimoto,et al.  Learning from demonstration and adaptation of biped locomotion , 2004, Robotics Auton. Syst..

[6]  Darwin G. Caldwell,et al.  A task-parameterized probabilistic model with minimal intervention control , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Jan Peters,et al.  Model-free Probabilistic Movement Primitives for physical interaction , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Jan Peters,et al.  A biomimetic approach to robot table tennis , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Jan Peters,et al.  Learning Motor Skills - From Algorithms to Robot Experiments , 2013, Springer Tracts in Advanced Robotics.

[10]  Christoph H. Lampert,et al.  Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components , 2012, Journal of Real-Time Image Processing.

[11]  Henk Nijmeijer,et al.  Robot Programming by Demonstration , 2010, SIMPAR.

[12]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[13]  Oliver Kroemer,et al.  Learning to select and generalize striking movements in robot table tennis , 2012, AAAI Fall Symposium: Robots Learning Interactively from Human Teachers.

[14]  Sandra Hirche,et al.  Risk-Sensitive Optimal Feedback Control for Haptic Assistance , 2012, 2012 IEEE International Conference on Robotics and Automation.

[15]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[16]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[17]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  David A. Belsley,et al.  Regression Analysis and its Application: A Data-Oriented Approach.@@@Applied Linear Regression.@@@Regression Diagnostics: Identifying Influential Data and Sources of Collinearity , 1981 .

[19]  Dagmar Sternad,et al.  Motor learning: changes in the structure of variability in a redundant task. , 2009, Advances in experimental medicine and biology.

[20]  G. Oriolo,et al.  Robotics: Modelling, Planning and Control , 2008 .

[21]  Jan Peters,et al.  Online optimal trajectory generation for robot table tennis , 2018, Robotics Auton. Syst..

[22]  Bernhard Schölkopf,et al.  Using probabilistic movement primitives for striking movements , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[23]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[24]  W. W. Muir,et al.  Regression Diagnostics: Identifying Influential Data and Sources of Collinearity , 1980 .

[25]  Darwin G. Caldwell,et al.  Robot motor skill coordination with EM-based Reinforcement Learning , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.