Learning Articulated Structure and Motion

Humans demonstrate a remarkable ability to parse complicated motion sequences into their constituent structures and motions. We investigate this problem, attempting to learn the structure of one or more articulated objects, given a time series of two-dimensional feature positions. We model the observed sequence in terms of “stick figure” objects, under the assumption that the relative joint angles between sticks can change over time, but their lengths and connectivities are fixed. The problem is formulated as a single probabilistic model that includes multiple sub-components: associating the features with particular sticks, determining the proper number of sticks, and finding which sticks are physically joined. We test the algorithm on challenging datasets of 2D projections of optical human motion capture and feature trajectories from real videos.

[1]  Richard S. Zemel,et al.  Learning Articulated Skeletons from Motion , 2007, ICCV 2007.

[2]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[3]  Geoffrey E. Hinton,et al.  Parameter estimation for linear dynamical systems , 1996 .

[4]  Andrew Zisserman,et al.  Multiple View Geometry , 2009, Encyclopedia of Biometrics.

[5]  T. Kanade,et al.  A multi-body factorization method for motion analysis , 1995, ICCV 1995.

[6]  Y. Weiss,et al.  Multibody factorization with uncertainty and missing data using the EM algorithm , 2004, CVPR 2004.

[7]  Yang Song,et al.  Learning probabilistic structure for human motion detection , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[8]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[9]  D Thalmann,et al.  Using skeleton-based tracking to increase the reliability of optical motion capture. , 2001, Human movement science.

[10]  Richard S. Zemel,et al.  Unsupervised Learning of Skeletons from Motion , 2008, ECCV.

[11]  Richard S. Zemel,et al.  Learning Parts-Based Representations of Data , 2006, J. Mach. Learn. Res..

[12]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[13]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[14]  C. W. Gear,et al.  Multibody Grouping from Motion Images , 1998, International Journal of Computer Vision.

[15]  Ian D. Reid,et al.  Articulated structure from motion by factorization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[18]  Yair Weiss,et al.  Multibody factorization with uncertainty and missing data using the EM algorithm , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[19]  Gene H. Golub,et al.  Matrix computations , 1983 .

[20]  D. Thalmann,et al.  Local and Global Skeleton Fitting Techniques for Optical Motion Capture , Modeling and Motion Capture Techniques for Virtual Environments , 1998 .

[21]  Marc Pollefeys,et al.  A Factorization-Based Approach to Articulated Motion Recovery , 2005, CVPR.

[22]  David A. Ross.  Learning Probabilistic Models for Visual Motion , 2008 .

[23]  Vítězslav Beran,et al.  Robust motion segmentation for on-line application , 2012 .

[24]  Yair Weiss,et al.  Segmentation using eigenvectors: a unifying view , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[25]  Marc Pollefeys,et al.  Automatic Kinematic Chain Building from Feature Trajectories of Articulated Objects , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[26]  Pascal Fua,et al.  Local and Global Skeleton Fitting Techniques for Optical Motion Capture , 1998, CAPTECH.

[27]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[28]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[29]  Marc Pollefeys,et al.  Articulated Motion Segmentation Using RANSAC with Priors , 2006, WDV.

[30]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[31]  Marc Pollefeys,et al.  A Factorization-Based Approach for Articulated Nonrigid Shape, Motion and Kinematic Chain Recovery From Video , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Trevor Darrell,et al.  Recovering Articulated Model Topology from Observed Rigid Motion , 2002, NIPS.

[34]  Cristian Sminchisescu,et al.  Estimating Articulated Human Motion with Covariance Scaled Sampling , 2003, Int. J. Robotics Res..

[35]  Yang Song,et al.  Unsupervised Learning of Human Motion , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Pushmeet Kohli,et al.  PoseCut: Simultaneous Segmentation and 3D Pose Estimation of Humans Using Dynamic Graph-Cuts , 2006, ECCV.

[37]  Thomas Viklands.  Algorithms for the Weighted Orthogonal Procrustes Problem and other Least Squares Problems , 2006 .

[38]  P. Anandan,et al.  Factorization with Uncertainty , 2000, International Journal of Computer Vision.

[39]  Phil F. Culverhouse,et al.  Robust Motion Segmentation by Spectral Clustering , 2003, BMVC.

[40]  Yair Weiss,et al.  Factorization with Uncertainty and Missing Data: Exploiting Temporal Coherence , 2003, NIPS.

[41]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[42]  David A. Forsyth,et al.  Skeletal parameter estimation from optical motion capture data , 2004, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[43]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .