3D articulated models and multi-view tracking with silhouettes

We propose a method to estimate the motion of a person filmed by two or more fixed cameras. The novelty of our technique is its ability to cope with fast movements, self-occlusions and noisy images. Our algorithms are based on the latest works on calibration and image segmentation developed in our lab. We compare the projections of a 3D model of a person on the images to the detected silhouettes of the person, and by creating forces that will move the 3D model towards the final estimation of the real pose. We developed a fast algorithm that computes the motion of the articulated 3D model. We show that our results are good, even if the cameras are not synchronized.

[1]  Ronald L. Huston,et al.  Dynamics of Multibody Systems , 1988 .

[2]  W. Rulka,et al.  Aspects of Efficient and Reliable Multibody System Simulation , 1990 .

[3]  Thomas S. Huang,et al.  Vision based hand modeling and tracking for virtual teleconferencing and telecollaboration , 1995, Proceedings of IEEE International Conference on Computer Vision.

[4]  Ioannis A. Kakadiaris,et al.  3D human body model acquisition from multiple views , 1995, Proceedings of IEEE International Conference on Computer Vision.

[5]  Takeo Kanade,et al.  Model-based tracking of self-occluding articulated objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[6]  Jean-Philippe Thirion,et al.  Non-rigid matching using demons , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Ioannis A. Kakadiaris,et al.  Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Quentin Delamarre.  Modélisation de la main pour sa localisation dans une séquence d'images , 1996 .

[9]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Olivier D. Faugeras,et al.  Finding pose of hand in video images: a stereo-based approach , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[11]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[12]  Rachid Deriche,et al.  A PDE-based level-set approach for detection and tracking of moving objects , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[13]  Rachid Deriche,et al.  Unifying boundary and region-based information for geodesic active tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[14]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[15]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[16]  Ioannis A. Kakadiaris,et al.  Model-Based Estimation of 3D Human Motion , 2000, IEEE Trans. Pattern Anal. Mach. Intell..