A Robustness Analysis of Inverse Optimal Control of Bipedal Walking

Cost functions have the potential to provide compact and understandable generalizations of motion. The goal of inverse optimal control (IOC) is to analyze an observed behavior which is assumed to be optimal with respect to an unknown cost function, and infer this cost function. Here we develop a method for characterizing cost functions of legged locomotion, with the goal of representing complex humanoid behavior with simple models. To test this methodology we simulate walking gaits of a simple 5 link planar walking model which optimize known cost functions, and assess the ability of our IOC method to recover them. In particular, the IOC method uses an iterative trajectory optimization process to infer cost function weightings consistent with those used to generate a single demonstrated optimal trial. We also explore sensitivity of the IOC to sensor noise in the observed trajectory, imperfect knowledge of the model or task, as well as uncertainty in the components of the cost function used. With appropriate modeling, these methods may help infer cost functions from human data, yielding a compact and generalizable representation of human-like motion for use in humanoid robot controllers, as well as providing a new tool for experimentally exploring human preferences.

[1]  A. Kuo,et al.  Human walking isn't all hard work: evidence of soft tissue contributions to energy dissipation and return , 2010, Journal of Experimental Biology.

[2]  T. McMahon,et al.  Ballistic walking. , 1980, Journal of biomechanics.

[3]  C. Karen Liu,et al.  Learning physics-based motion style with nonlinear inverse optimization , 2005, ACM Trans. Graph..

[4]  Katja D. Mombaur,et al.  Humanoid gait generation in complex environments based on template models and optimality principles learned from human beings , 2018, Int. J. Robotics Res..

[5]  Russ Tedrake,et al.  Direct Trajectory Optimization of Rigid Body Dynamical Systems through Contact , 2012, WAFR.

[6]  Matthew Kelly,et al.  An Introduction to Trajectory Optimization: How to Do Your Own Direct Collocation , 2017, SIAM Rev..

[7]  H. Ralston,et al.  Optimization of energy expenditure during level walking , 2004, European Journal of Applied Physiology and Occupational Physiology.

[8]  A. Hof The 'extrapolated center of mass' concept suggests a simple control of balance in walking. , 2008, Human movement science.

[9]  Dmitry Berenson,et al.  Goal Set Inverse Optimal Control and Iterative Replanning for Predicting Human Reaching Motions in Shared Workspaces , 2016, IEEE Transactions on Robotics.

[10]  Alexander Herzog,et al.  Balancing experiments on a torque-controlled humanoid with hierarchical inverse dynamics , 2013, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  R. Kram,et al.  Mechanical and metabolic determinants of the preferred step width in human walking , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[12]  A. Kuo,et al.  Mechanical Work as an Indirect Measure of Subjective Costs Influencing Human Movement , 2012, PloS one.

[13]  N. Hogan An organizing principle for a class of voluntary movements , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  Sergey Levine,et al.  Continuous Inverse Optimal Control with Locally Optimal Examples , 2012, ICML.

[15]  Twan Koolen,et al.  Capturability-based analysis and control of legged locomotion, Part 1: Theory and application to three simple gait models , 2011, Int. J. Robotics Res..

[16]  Jerry E. Pratt,et al.  Learning Capture Points for humanoid push recovery , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[17]  Twan Koolen,et al.  Capturability-based analysis and control of legged locomotion, Part 2: Application to M2V2, a lower-body humanoid , 2012, Int. J. Robotics Res..

[18]  Sergey Levine,et al.  Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization , 2016, ICML.

[19]  Steven H Collins,et al.  Dynamic arm swinging in human walking , 2009, Proceedings of the Royal Society B: Biological Sciences.

[20]  Jean-Paul Laumond,et al.  From human to humanoid locomotion—an inverse optimal control approach , 2010, Auton. Robots.

[21]  Alexander Herzog,et al.  Structured contact force optimization for kino-dynamic motion generation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Konrad Paul Kording,et al.  An Examination of the Generalizability of Motor Costs , 2013, PloS one.

[23]  Daniel Vélez Día,et al.  Biomechanics and Motor Control of Human Movement , 2013 .

[24]  D. De Clercq,et al.  A Simple Exoskeleton That Assists Plantarflexion Can Reduce the Metabolic Cost of Human Walking , 2013, PloS one.

[25]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[26]  Katja D. Mombaur,et al.  Inverse optimal control based identification of optimality criteria in whole-body human walking on level ground , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[27]  Vladlen Koltun,et al.  Optimizing locomotion controllers using biologically-based actuators and objectives , 2012, ACM Trans. Graph..

[28]  M. Daley,et al.  Compass gait mechanics account for top walking speeds in ducks and humans , 2008, Journal of Experimental Biology.

[29]  Marc Toussaint,et al.  Direct Loss Minimization Inverse Optimal Control , 2015, Robotics: Science and Systems.

[30]  Emanuel Todorov,et al.  Inverse Optimal Control with Linearly-Solvable MDPs , 2010, ICML.