A framework for modeling appearance change in image sequences

Image "appearance" may change over time due to a variety of causes such as: 1) object or camera motion; 2) generic photometric events including variations in illumination (e.g. shadows) and specular reflections; and 3) "iconic changes" which are specific to the objects being viewed and include complex occlusion events and changes in the material properties of the objects. We propose a general framework for representing and recovering these "appearance changes" in an image sequence as a "mixture" of different causes. The approach generalizes previous work on optical flow to provide a richer description of image events and more reliable estimates of image motion.

[1]  John Law,et al.  Robust Statistics—The Approach Based on Influence Functions , 1986 .

[2]  E. Reed.  The Ecological Approach to Visual Perception , 1989 .

[3]  D. N. Geary.  Mixture Models: Inference and Applications to Clustering , 1989 .

[4]  Naoki Mukawa.  Estimation of shape, reflection coefficients and illuminant direction from image sequences , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[5]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Shahriar Negahdaripour,et al.  A generalized brightness change model for computing optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[7]  Peter W. Hallinan,et al.  A deformable model for the recognition of human faces under arbitrary illumination , 1995 .

[8]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[9]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.

[10]  Tony Ezzat,et al.  Facial analysis and synthesis using image-based models , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[11]  Alex Pentland,et al.  Generalized Image Matching: Statistical Learning of Physically-Based Deformations , 1996, ECCV.

[12]  Tomaso Poggio,et al.  Image Representations for Visual Learning , 1996, Science.

[13]  Edward H. Adelson,et al.  A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Gregory D. Hager,et al.  Real-time tracking of image regions with changes in geometry and illumination , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[16]  David J. Fleet,et al.  Learning parameterized models of image motion , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  David J. Fleet,et al.  Modelling ap-pearance change in image sequences , 1997 .

[18]  Yair Weiss,et al.  Smoothness in layers: Motion segmentation using nonparametric mixture estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Tomaso A. Poggio,et al.  A bootstrapping algorithm for learning linear models of object classes , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.