Dimensionality reduction for probabilistic movement primitives

Humans as well as humanoid robots can use a large number of degrees of freedom to solve very complex motor tasks. The high-dimensionality of these motor tasks adds difficulties to the control problem and machine learning algorithms. However, it is well known that the intrinsic dimensionality of many human movements is small in comparison to the number of employed DoFs, and hence, the movements can be represented by a small number of synergies encoding the couplings between DoFs. In this paper, we want to apply Dimensionality Reduction (DR) to a recent movement representation used in robotics, called Probabilistic Movement Primitives (ProMP). While ProMP have been shown to have many benefits, they suffer with the high-dimensionality of a robotic system as the number of parameters of a ProMP scales quadratically with the dimensionality. We use probablistic dimensionality reduction techniques based on expectation maximization to extract the unknown synergies from a given set of demonstrations. The ProMP representation is now estimated in the low-dimensional space of the synergies. We show that our dimensionality reduction is more efficient both for encoding a trajectory from data and for applying Reinforcement Learning with Relative Entropy Policy Search (REPS).

[1]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[2]  N. A. Bernshteĭn The co-ordination and regulation of movements , 1967 .

[3]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[4]  Emilio Bizzi,et al.  Shared and specific muscle synergies in natural motor behaviors. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Alessandro Lazaric,et al.  Bayesian Multi-Task Reinforcement Learning , 2010, ICML.

[6]  Yasemin Altun,et al.  Relative Entropy Policy Search , 2010 .

[7]  Zhi-Hong Mao,et al.  Dimensionality Reduction in Control and Coordination of the Human Hand , 2008, IEEE Transactions on Biomedical Engineering.

[8]  Oliver Kroemer,et al.  Generalization of human grasping for multi-fingered robot hands , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Luís Paulo Reis,et al.  Omnidirectional Walking and Active Balance for Soccer Humanoid Robot , 2013, EPIA.

[10]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[11]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[12]  Jan Peters,et al.  A Survey on Policy Search for Robotics , 2013, Found. Trends Robotics.

[13]  Carme Torras,et al.  Dimensionality reduction and motion coordination in learning trajectories with Dynamic Movement Primitives , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Jan Peters,et al.  Latent space policy search for robotics , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.