Robust Physics‐based Motion Retargeting with Realistic Body Shapes

Motion capture is often retargeted to new, and sometimes drastically different, characters. When the characters take on realistic human shapes, however, we become more sensitive to the motion looking right. This means adapting it to be consistent with the physical constraints imposed by different body shapes. We show how to take realistic 3D human shapes, approximate them using a simplified representation, and animate them so that they move realistically using physically‐based retargeting. We develop a novel spacetime optimization approach that learns and robustly adapts physical controllers to new bodies and constraints. The approach automatically adapts the motion of the mocap subject to the body shape of a target subject. This motion respects the physical properties of the new body and every body shape results in a different and appropriate movement. This makes it easy to create a varied set of motions from a single mocap sequence by simply varying the characters. In an interactive environment, successful retargeting requires adapting the motion to unexpected external forces. We achieve robustness to such forces using a novel LQR‐tree formulation. We show that the simulated motions look appropriate to each character's anatomy and their actions are robust to perturbations.

[1]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2015, ACM Trans. Graph..

[2]  Zoran Popovic,et al.  Physically based motion transformation , 1999, SIGGRAPH.

[3]  Michael Gleicher,et al.  Retargetting motion to new characters , 1998, SIGGRAPH.

[4]  Stefan Schaal,et al.  Full dynamics LQR control of a humanoid robot: An experimental study on balancing and squatting , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[5]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH '05.

[6]  Victor B. Zordan,et al.  Mapping optical motion capture data to skeletal motion using a physical model , 2003, SCA '03.

[7]  Jaakko Lehtinen,et al.  Online motion synthesis using sequential Monte Carlo , 2014, ACM Trans. Graph..

[8]  C. K. Liu,et al.  Learning physics-based motion style with nonlinear inverse optimization , 2005, SIGGRAPH 2005.

[9]  Sehoon Ha,et al.  Iterative Training of Dynamic Skills Inspired by Human Coaching Techniques , 2014, ACM Trans. Graph..

[10]  Ronan Boulic,et al.  Egocentric Mapping of Body Surface Constraints , 2018, IEEE Transactions on Visualization and Computer Graphics.

[11]  Marco da Silva,et al.  Interactive simulation of stylized human locomotion , 2008, ACM Trans. Graph..

[12]  Aaron Hertzmann,et al.  Trajectory Optimization for Full-Body Movements with Complex Contacts , 2013, IEEE Transactions on Visualization and Computer Graphics.

[13]  Andrea Tagliasacchi,et al.  Sphere-meshes for real-time hand modeling and tracking , 2016, ACM Trans. Graph..

[14]  Frédo Durand,et al.  Linear Bellman combination for control of character animation , 2009, ACM Trans. Graph..

[15]  Ziv Bar-Joseph,et al.  Modeling spatial and temporal variation in motion data , 2009, ACM Trans. Graph..

[16]  Yuval Tassa,et al.  Synthesis and stabilization of complex behaviors through online trajectory optimization , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Sungeun Kim,et al.  Data-driven biped control , 2010, ACM Trans. Graph..

[18]  Tamy Boubekeur,et al.  Sphere-Meshes , 2013, ACM Trans. Graph..

[19]  Michael J. Black,et al.  MoSh: motion and shape capture from sparse markers , 2014, ACM Trans. Graph..

[20]  Lucas Kovar,et al.  Footskate cleanup for motion capture editing , 2002, SCA '02.

[21]  Zoran Popović,et al.  Contact-aware nonlinear control of dynamic characters , 2009, SIGGRAPH 2009.

[22]  Libin Liu,et al.  Learning reduced-order feedback policies for motion skills , 2015, Symposium on Computer Animation.

[23]  Jun-yong Noh,et al.  Data‐guided Model Predictive Control Based on Smoothed Contact Dynamics , 2016, Comput. Graph. Forum.

[24]  Eugene Fiume,et al.  Domain of attraction expansion for physics-based character control , 2017, TOGS.

[25]  Charlie C. L. Wang,et al.  Ellipsoid-tree construction for solid objects , 2007, Symposium on Solid and Physical Modeling.

[26]  C. Karen Liu,et al.  Synthesis of Responsive Motion Using a Dynamic Model , 2010, Comput. Graph. Forum.

[27]  Tianjia Shao,et al.  Sampling-based contact-rich motion control , 2010, SIGGRAPH 2010.

[28]  Jessica K. Hodgins,et al.  Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces , 2004, SIGGRAPH 2004.

[29]  Brian Wyvill,et al.  Robust iso-surface tracking for interactive character skinning , 2014, ACM Trans. Graph..

[30]  Zoran Popovic,et al.  Composite control of physically simulated characters , 2011, TOGS.

[31]  Nancy Argüelles,et al.  Author ' s , 2008 .

[32]  Michael J. Black,et al.  Dyna: a model of dynamic human shape in motion , 2015, ACM Trans. Graph..

[33]  Daniel Thalmann,et al.  Using an Intermediate Skeleton and Inverse Kinematics for Motion Retargeting , 2000, Comput. Graph. Forum.

[34]  Ming C. Lin,et al.  Free-flowing granular materials with two-way solid coupling , 2010, SIGGRAPH 2010.

[35]  Libin Liu,et al.  Guided Learning of Control Graphs for Physics-Based Characters , 2016, ACM Trans. Graph..

[36]  Daniel Cohen-Or,et al.  Generalized cylinder decomposition , 2015, ACM Trans. Graph..

[37]  C. Karen Liu,et al.  Online control of simulated humanoids using particle belief propagation , 2015, ACM Trans. Graph..

[38]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[39]  Zoran Popovic,et al.  Discovery of complex behaviors through contact-invariant optimization , 2012, ACM Trans. Graph..

[40]  C. Karen Liu,et al.  Learning physics-based motion style with nonlinear inverse optimization , 2005, ACM Trans. Graph..

[41]  Jitendra Malik,et al.  Recurrent Network Models for Human Dynamics , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[42]  Nadia Magnenat-Thalmann,et al.  Motion adaptation based on character shape , 2008 .

[43]  Jessica K. Hodgins,et al.  Adapting simulated behaviors for new characters , 1997, SIGGRAPH.

[44]  Jovan Popovic,et al.  Adaptation of performed ballistic motion , 2005, TOGS.

[45]  Antti Oulasvirta,et al.  Fast and robust hand tracking using detection-guided optimization , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Baining Guo,et al.  Improving Sampling‐based Motion Control , 2015, Comput. Graph. Forum.

[47]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[48]  Andrew P. Witkin,et al.  Spacetime constraints , 1988, SIGGRAPH.

[49]  Taku Komura,et al.  Spatial relationship preserving character motion adaptation , 2010, SIGGRAPH 2010.

[50]  Russ Tedrake,et al.  LQR-trees: Feedback motion planning on sparse randomized trees , 2009, Robotics: Science and Systems.