Recognizing Human Motion Using Parameterized Models of Optical Flow

The tracking and recognition of human motion is a challenging problem with diverse applications in virtual reality, medicine, teleoperations, animation, and human-computer interaction to name a few. The study of human motion has a long history with the use of images for analyzing animate motion beginning with the improvements in photography and the development of motion-pictures in the late nineteenth century. Scientists and artists such as Marey [12] and Muybridge [26] were early explorers of human and animal motion in images and image sequences. Today, commercial motion-capture systems can be used to accurately record the 3D movements of an instrumented person, but the motion analysis and motion recognition of an arbitrary person in a video sequence remains an unsolved problem. In this chapter we describe the representation and recognition of human motion using parameterized models of optical flow. A person’s limbs, face, and facial features are represented as patches whose motion In a image sequence can be modeled by low-order polynomials. A robust optical flow estimation technique is used to recover the motion of these patches and the recovered motion parameters provide a rich, yet concise, description of the human motion which can be used to recognize human activities, gestures, and facial expressions.

[1]  Charles R. Dyer,et al.  Cyclic motion detection using spatiotemporal surfaces and curves , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[2]  Alex Pentland,et al.  A vision system for observing and extracting facial action parameters , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Alex Pentland,et al.  Visually Controlled Graphics , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  David C. Hogg,et al.  Learning Flexible Models from Image Sequences , 1994, ECCV.

[5]  Sebastian Toelg,et al.  Towards an Example-Based Image Compression Architecture for Video-Conferencing , 1994 .

[6]  Pertti Roivainen,et al.  3-D Motion Estimation in Model-Based Facial Image Coding , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Larry S. Davis,et al.  Recognition of head gestures using hidden Markov models , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[8]  Irfan Essa,et al.  Tracking facial motion , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[9]  Michael Isard,et al.  3D position, attitude and shape input using video tracking of hands and lips , 1994, SIGGRAPH.

[10]  Pietro Perona,et al.  Monocular tracking of the human arm in 3D , 1995, Proceedings of IEEE International Conference on Computer Vision.

[11]  Nan Li,et al.  Lipreading Using Eigensequences , 1995 .

[12]  Alex Pentland,et al.  Recovery of Nonrigid Motion and Structure , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Andrew Blake,et al.  Surface Orientation and Time to Contact from Image Divergence and Deformation , 1992, ECCV.

[14]  Kenji Mase,et al.  Recognition of Facial Expression from Optical Flow , 1991 .

[15]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Takeo Kanade,et al.  Model-based tracking of self-occluding articulated objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[17]  M. Rosenblum,et al.  Human emotion recognition from motion using a radial basis function network architecture , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[18]  J. Little,et al.  Describing motion for recognition , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[19]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  C. Jennison,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[21]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[22]  Larry S. Davis,et al.  Computing spatio-temporal representations of human faces , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[24]  K. Rohr.  Towards model-based recognition of human movements in image sequences , 1994 .

[25]  Andrea J. van Doorn,et al.  Invariant Properties of the Motion Parallax Field due to the Movement of Rigid Bodies Relative to an Observer , 1975 .

[26]  Steven M. Seitz,et al.  Affine invariant detection of periodic motion , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Michael J. Black,et al.  Cardboard people: a parameterized model of articulated image motion , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[28]  Eadweard Muybridge,et al.  The Human Figure in Motion , 1955 .

[29]  MotionSteven M. Seitzseitz,et al.  A ne Invariant Detection of Periodic Motion , 1994 .

[30]  Mubarak Shah,et al.  Matching motion trajectories using scale-space , 1993, Pattern Recognit..

[31]  Harpreet S. Sawhney,et al.  3D geometry from planar parallax , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Demetri Terzopoulos,et al.  Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[34]  Marta Braun,et al.  Picturing Time: The Work of Etienne-Jules Marey (1830-1904) , 1995 .

[35]  Aaron F. Bobick,et al.  Recognition of human body motion using phase space constraints , 1995, Proceedings of IEEE International Conference on Computer Vision.

[36]  J. Aggarwal,et al.  Lower limb kinematics of human walking with the medial axis transformation , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[37]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[38]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[39]  J. Eklundh.  First international conference on computer vision: 8–11 June 1987, London-U.K. , 1988 .