The Natural Gradient as a control signal for a humanoid robot

This paper presents Natural Gradient Control (NGC), a control algorithm that efficiently estimates and applies the natural gradient for high-degree of freedom robotic control. In contrast to the standard task Jacobian, the natural gradient follows the direction of steepest descent with respect to a parameterized model with extra degrees of freedom injected. This procedure enables NGC to maneuver smoothly in regions where the task Jacobian is ill-conditioned or singular. NGC efficiently estimates the natural gradient using only forward kinematics evaluations. This sampling-based algorithm prevents the need for gradient calculations and therefore allows great flexibility in the cost functions. Experiments show NGC can even use statistics of rendered images as part of the cost function, which would be impossible with traditional inverse kinematics approaches. The advantages of NGC are shown on the full 41-degree upper body of an iCub humanoid, in simulation and on a real robot, and compared to a Jacobian-based controller. Experiments show that the natural gradient is robust and avoids common pitfalls such as local minima and slow convergence, which often affects the application of Jacobian-based methods. Demonstrations on the iCub show that NGC is a practical method that can be used for complex movements.

[1]  Giulio Sandini,et al.  The iCub humanoid robot: an open platform for research in embodied cognition , 2008, PerMIS.

[2]  Andrew W. Moore,et al.  Locally Weighted Learning for Control , 1997, Artificial Intelligence Review.

[3]  Jürgen Leitner,et al.  Curiosity driven reinforcement learning for motion planning on humanoids , 2014, Front. Neurorobot..

[4]  Jürgen Leitner,et al.  Task-relevant roadmaps: A framework for humanoid motion planning , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Jürgen Leitner,et al.  The Modular Behavioral Environment for Humanoids and other Robots (MoBeE) , 2012, ICINCO.

[6]  Jürgen Leitner,et al.  Reactive reaching and grasping on a humanoid: Towards closing the action-perception loop on the iCub , 2014, 2014 11th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[7]  Nicolas Courty,et al.  Inverse Kinematics Using Sequential Monte Carlo Methods , 2008, AMDO.

[8]  Tom Schaul,et al.  Natural Evolution Strategies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[9]  Ioannis Iossifidis,et al.  Autonomous reaching and obstacle avoidance with the anthropomorphic arm of a robotic assistant using the attractor dynamics approach , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[10]  Ronan Boulic,et al.  An inverse kinematics architecture enforcing an arbitrary number of strict priority levels , 2004, The Visual Computer.

[11]  J. Angeles On the Numerical Solution of the Inverse Kinematic Problem , 1985 .

[12]  Jürgen Schmidhuber,et al.  Rapid Humanoid Motion Learning through Coordinated, Parallel Evolution , 2014, SAB.

[13]  Max Suell Dutra,et al.  New technique for inverse kinematics problem using Simulated Annealing , 2008 .

[14]  Nikolaos G. Tsagarakis,et al.  iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research , 2007, Adv. Robotics.

[15]  Chris Hecker,et al.  Real-time motion retargeting to highly varied user-created morphologies , 2008, SIGGRAPH 2008.

[16]  Jun Nakanishi,et al.  Learning Attractor Landscapes for Learning Motor Primitives , 2002, NIPS.

[17]  R. Paul,et al.  Kinematic control equations for simple manipulators , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[18]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[19]  Tom Schaul,et al.  Exponential natural evolution strategies , 2010, GECCO '10.

[20]  Jürgen Schmidhuber,et al.  Explore to see, learn to perceive, get the actions for free: SKILLABILITY , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).