Robustly Estimating Changes in Image Appearance

We propose a generalized model of image “appearance change” in which brightness variation over time is represented as a probabilistic mixture of different causes. We define four generative models of appearance change due to (1) object or camera motion; (2) illumination phenomena; (3) specular reflections; and (4) “iconic changes” which are specific to the objects being viewed. These iconic changes include complex occlusion events and changes in the material properties of the objects. We develop a robust statistical framework for recovering these appearance changes in image sequences. This approach generalizes previous work on optical flow to provide a richer description of image events and more reliable estimates of image motion in the presense of shadows and specular reflections.

[1]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[2]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[3]  John Law,et al.  Robust Statistics—The Approach Based on Influence Functions , 1986 .

[4]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[5]  Jeremy MG Taylor,et al.  Robust Statistical Modeling Using the t Distribution , 1989 .

[6]  E. Reed The Ecological Approach to Visual Perception , 1989 .

[7]  D. N. Geary Mixture Models: Inference and Applications to Clustering , 1989 .

[8]  Tomaso A. Poggio,et al.  Motion Field and Optical Flow: Qualitative Properties , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  David W. Murray,et al.  Experiments in the machine interpretation of visual motion , 1990 .

[10]  Naoki Mukawa Estimation of shape, reflection coefficients and illuminant direction from image sequences , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[11]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  David J. Fleet,et al.  Stability of phase information , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[13]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[14]  Michal Irani,et al.  Detecting and Tracking Multiple Moving Objects Using Temporal Integration , 1992, ECCV.

[15]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[17]  J. Kittler,et al.  Robust motion analysis , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Peter W. Hallinan,et al.  A deformable model for the recognition of human faces under arbitrary illumination , 1995 .

[19]  Alex Pentland,et al.  Cooperative Robust Estimation Using Layers of Support , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.

[21]  Harpreet S. Sawhney,et al.  Model-based 2D&3D dominant motion estimation for mosaicing and video representation , 1995, Proceedings of IEEE International Conference on Computer Vision.

[22]  Michael J. Black,et al.  Estimating Optical Flow in Segmented Images Using Variable-Order Parametric Models With Local Deformations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Tony Ezzat,et al.  Facial analysis and synthesis using image-based models , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[24]  Harpreet S. Sawhney,et al.  Compact Representations of Videos Through Dominant and Multiple Motion Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Michael J. Black,et al.  Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Alex Pentland,et al.  Generalized Image Matching: Statistical Learning of Physically-Based Deformations , 1996, ECCV.

[27]  Tomaso Poggio,et al.  Image Representations for Visual Learning , 1996, Science.

[28]  Thomas Vetter,et al.  Learning novel views to a single face image , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[29]  Edward H. Adelson,et al.  A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Gregory D. Hager,et al.  Real-time tracking of image regions with changes in geometry and illumination , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[32]  David Beymer,et al.  Feature correspondence by interleaving shape and texture computations , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  David J. Fleet,et al.  Learning parameterized models of image motion , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Yair Weiss,et al.  Smoothness in layers: Motion segmentation using nonparametric mixture estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Tomaso A. Poggio,et al.  A bootstrapping algorithm for learning linear models of object classes , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  David J. Fleet,et al.  A framework for modeling appearance change in image sequences , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[37]  David J. Fleet,et al.  Motion feature detection using steerable flow fields , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[38]  Shahriar Negahdaripour,et al.  Revised Definition of Optical Flow: Integration of Radiometric and Geometric Cues for Dynamic Scene Analysis , 1998, IEEE Trans. Pattern Anal. Mach. Intell..