Combining Intensity and Motion for Incremental Segmentation and Tracking Over Long Image Sequences

This paper presents a method for incrementally segmenting images over time using both intensity and motion information. This is done by formulating a model of physically significant image resgions using local constraints on intensity and motion and then finding the optimal segmentation over time using an incremental stochastic minimization technique. The result is a robust and dynamic segmentation of the scene over a sequence of images. The approach has a number of benefits. First, discontinuities are extracted and tracked simultaneously. Second, a segmentation is always available and it improves over time. Finally, by combining motion and intensity, the structural properties of discontinuities can be recovered; that is, discontinuities can be classified as surface markings or actual surface boundaries.

[1]  W. B. Thompson,et al.  Combining motion and contrast for segmentation , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[4]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Patrick Bouthemy,et al.  Multimodal motion estimation and segmentation using Markov random fields , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[6]  Patrick Bouthemy,et al.  Detection and Tracking of Moving Objects Based on a Statistical Regularization Method in Space and Time , 1990, European Conference on Computer Vision.

[7]  Rachid Deriche,et al.  Recovering 3D motion and structure from stereo and 2D token tracking cooperation , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[8]  Donald Geman,et al.  Boundary Detection by Constrained Optimization , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Patrick Bouthemy,et al.  Multiframe-based identification of mobile components of a scene with a moving camera , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Michael J. Black,et al.  Robust dynamic motion estimation over time , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.