Image change detection algorithms: a systematic survey

Detecting regions of change in multiple images of the same scene taken at different times is of widespread interest due to a large number of applications in diverse disciplines, including remote sensing, surveillance, medical diagnosis and treatment, civil infrastructure, and underwater sensing. This paper presents a systematic survey of the common processing steps and core decision rules in modern change detection algorithms, including significance and hypothesis testing, predictive models, the shading model, and background modeling. We also discuss important preprocessing methods, approaches to enforcing the consistency of the change mask, and principles for evaluating and comparing the performance of change detection algorithms. It is hoped that our classification of algorithms into a relatively small number of categories will provide useful guidance to the algorithm designer.

[1]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  L. Davis,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002, Proc. IEEE.

[3]  Neil A. Thacker,et al.  Non-parametric image subtraction using grey level scattergrams , 2002, Image Vis. Comput..

[4]  Paul L. Rosin,et al.  Evaluation of global image thresholding for change detection , 2003, Pattern Recognit. Lett..

[5]  Rudolf Mester,et al.  Detection of moving objects using a robust displacement estimation including a statistical error analysis , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[6]  Pascal Fua,et al.  Self-Consistency and MDL: A Paradigm for Evaluating Point-Correspondence Algorithms, and Its Application to Detecting Changes in Surface Elevation , 2004, International Journal of Computer Vision.

[7]  Hervé Delingette,et al.  Automatic Detection and Segmentation of Evolving Processes in 3D Medical Images: Application to Multiple Sclerosis , 1999, IPMI.

[8]  G. Hay,et al.  A Multiscale Object-Specific Approach to Digital Change Detection , 2003 .

[9]  Luigi di Stefano,et al.  A change-detection algorithm based on structure and colour , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[10]  S. Kay Fundamentals of statistical signal processing: estimation theory , 1993 .

[11]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[12]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[13]  Til Aach,et al.  Bayesian illumination-invariant motion detection , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[14]  Fionn Murtagh,et al.  Digital change detection with the aid of multiresolution wavelet analysis , 2001 .

[15]  Charles V. Stewart,et al.  A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[17]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.

[18]  Siamak Khorram,et al.  An introduction to using generalized linear models to enhance satellite-based change detection , 1997, IGARSS'97. 1997 IEEE International Geoscience and Remote Sensing Symposium Proceedings. Remote Sensing - A Scientific Vision for Sustainable Development.

[19]  Lorenzo Bruzzone,et al.  An adaptive semiparametric and context-based approach to unsupervised change detection in multitemporal remote-sensing images , 2002, IEEE Trans. Image Process..

[20]  Til Aach,et al.  Illumination-invariant change detection , 2000, 4th IEEE Southwest Symposium on Image Analysis and Interpretation.

[21]  Michael Isard,et al.  Active Contours , 2000, Springer London.

[22]  Daniel P. Lopresti,et al.  Why table ground-truthing is hard , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[23]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[24]  Eric P. Crist,et al.  A Physically-Based Transformation of Thematic Mapper Data---The TM Tasseled Cap , 1984, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Chin-Seng Chua,et al.  Statistical background modeling for non-stationary camera , 2003, Pattern Recognit. Lett..

[26]  Stephen J. Aldington,et al.  The accurate assessment of changes in retinal vessel diameter using multiple frame electrocardiograph synchronised fundus photography. , 1996, Current eye research.

[27]  Lorenzo Bruzzone,et al.  An iterative technique for the detection of land-cover transitions in multitemporal remote-sensing images , 1997, IEEE Trans. Geosci. Remote. Sens..

[28]  Shahriar Negahdaripour,et al.  Revised Definition of Optical Flow: Integration of Radiometric and Geometric Cues for Dynamic Scene Analysis , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[30]  Ramakant Nevatia,et al.  Detecting changes in aerial views of man-made structures , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[31]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Robert L. Lillestrand,et al.  Techniques ror Change Detection , 1972, IEEE Transactions on Computers.

[33]  Andrew Blake,et al.  A Probabilistic Background Model for Tracking , 2000, ECCV.

[34]  Ridha Touzi,et al.  A review of speckle filtering in the context of estimation theory , 2002, IEEE Trans. Geosci. Remote. Sens..

[35]  Morton J. Canty,et al.  Unsupervised change detection techniques using multispectral satellite images , 1999, IEEE 1999 International Geoscience and Remote Sensing Symposium. IGARSS'99 (Cat. No.99CH36293).

[36]  Yawgeng A. Chau,et al.  Optimum multisensor data fusion for image change detection , 1995, IEEE Trans. Syst. Man Cybern..

[37]  Keinosuke Fukunaga,et al.  The Reduced Parzen Classifier , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Paul L. Rosin Thresholding for change detection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[39]  Joachim M. Buhmann,et al.  Topology free hidden Markov models: application to background modeling , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[40]  Chia-Ling Tsai,et al.  The dual-bootstrap iterative closest point algorithm with application to retinal image registration , 2003, IEEE Transactions on Medical Imaging.

[41]  Aaron F. Bobick,et al.  Fast Lighting Independent Background Subtraction , 2004, International Journal of Computer Vision.

[42]  Takeo Kanade,et al.  Advances in Cooperative Multi-Sensor Video Surveillance , 1999 .

[43]  Hong Heather Yu,et al.  A Hierarchical Multiresolution Video Shot Transition Detection Scheme , 1999, Comput. Vis. Image Underst..

[44]  J. Thirion,et al.  Deformation analysis to detect and quantify active lesions in three-dimensional medical image sequences , 1999, IEEE Transactions on Medical Imaging.

[45]  Peter W. Eklund,et al.  VALUES FOR THE FUZZY -MEANS CLASSIFIER IN CHANGE DETECTION FOR REMOTE SENSING , 2001 .

[46]  H. Niemann,et al.  Adaptive change detection for real-time surveillance applications , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[47]  Hans-Hellmut Nagel,et al.  New likelihood test methods for change detection in image sequences , 1984, Comput. Vis. Graph. Image Process..

[48]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  T. Ebrahimi,et al.  Change detection and background extraction by linear algebra , 2001, Proc. IEEE.

[50]  Pramod K. Varshney,et al.  An image change detection algorithm based on Markov random field models , 2002, IEEE Trans. Geosci. Remote. Sens..

[51]  Naoki Mukawa,et al.  Detecting changes of buildings from aerial images using shadow and shading model , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[52]  Robert THOMA,et al.  Motion compensating interpolation considering covered and uncovered background , 1989, Signal Process. Image Commun..

[53]  A. S. Elfishawy,et al.  Adaptive algorithms for change detection in image sequence , 1991, Signal Process..

[54]  Emanuele Trucco,et al.  Computer and Robot Vision , 1995 .

[55]  Liyuan Li,et al.  Integrating intensity and texture differences for robust change detection , 2002, IEEE Trans. Image Process..

[56]  Longin Jan Latecki,et al.  Detection of changes in surveillance videos , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[57]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[58]  Massimo Filippi,et al.  Image registration and subtraction to detect active T2 lesions in MS: an interobserver study , 2002, Journal of Neurology.

[59]  H. Vincent Poor,et al.  An Introduction to Signal Detection and Estimation , 1994, Springer Texts in Electrical Engineering.

[60]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[61]  Peng Gong,et al.  Change Detection Using Principal Component Analysis and Fuzzy Set Theory , 1993 .

[62]  Lorenzo Bruzzone,et al.  An adaptive approach to reducing registration noise effects in unsupervised change detection , 2003, IEEE Trans. Geosci. Remote. Sens..

[63]  Fredrik Gustafsson,et al.  Adaptive filtering and change detection , 2000 .

[64]  Hanumant Singh,et al.  Toward large-area mosaicing for underwater scientific applications , 2003 .

[65]  Tong Zhang,et al.  Volume and Surface Area Distributions of Cracks in Concrete , 2001, IWVF.

[66]  William M. Wells,et al.  Statistical Approaches to Feature-Based Object Recognition , 2004, International Journal of Computer Vision.

[67]  Steven E. Franklin,et al.  Forest Change Detection , 2001 .

[68]  Shaun Quegan,et al.  The principles of polarimetric filtering , 1997, IGARSS'97. 1997 IEEE International Geoscience and Remote Sensing Symposium Proceedings. Remote Sensing - A Scientific Vision for Sustainable Development.

[69]  Chris Clifton Change Detection in Overhead Imagery Using Neural Networks , 2004, Applied Intelligence.

[70]  Bui Tuong Phong Illumination for computer generated pictures , 1975, Commun. ACM.

[71]  John R. Jensen,et al.  Introductory Digital Image Processing: A Remote Sensing Perspective , 1986 .

[72]  Hiroshi Hanaizumi,et al.  A change detection method for remotely sensed multispectral and multitemporal images using 3-D segmentation , 2001, IEEE Trans. Geosci. Remote. Sens..

[73]  C. Woodcock,et al.  An assessment of several linear change detection techniques for mapping forest mortality using multitemporal landsat TM data , 1996 .

[74]  H. Vincent Poor,et al.  An introduction to signal detection and estimation (2nd ed.) , 1994 .

[75]  James V. Krogmeier,et al.  MODEL-BASED VEHICLE TRACKING FROM IMAGE SEQUENCES WITH AN APPLICATION TO ROAD SURVEILLANCE , 1996 .

[76]  Touradj Ebrahimi,et al.  Video object extraction based on adaptive background and statistical change detection , 2000, IS&T/SPIE Electronic Imaging.

[77]  H. Hirosawa,et al.  Suppression of speckle in synthetic aperture radar images using wavelet , 1998 .

[78]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[79]  Jakob J. van Zyl,et al.  Change detection techniques for ERS-1 SAR data , 1993, IEEE Trans. Geosci. Remote. Sens..

[80]  Shyang Chang,et al.  Statistical change detection with moments under time-varying illumination , 1998, IEEE Trans. Image Process..

[81]  Ramakant Nevatia,et al.  Detecting changes in aerial views of man-made structures , 2000, Image Vis. Comput..

[82]  Siamak Khorram,et al.  The effects of image misregistration on the accuracy of remotely sensed change detection , 1998, IEEE Trans. Geosci. Remote. Sens..

[83]  Til Aach,et al.  Bayesian algorithms for adaptive change detection in image sequences using Markov random fields , 1995, Signal Process. Image Commun..

[84]  M. S. Ulstad,et al.  An algorithm for estimating small scale differences between two digital images , 1973, Pattern Recognit..

[85]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[86]  Til Aach,et al.  Statistical model-based change detection in moving video , 1993, Signal Process..

[87]  Sei-Wang Chen,et al.  Automatic change detection of driving environments in a vision-based driver assistance system , 2003, IEEE Trans. Neural Networks.

[88]  Ashbindu Singh,et al.  Review Article Digital change detection techniques using remotely-sensed data , 1989 .

[89]  Louis Lemieux,et al.  The detection and significance of subtle changes in mixed-signal brain lesions by serial MRI scan matching and spatial normalization , 1998, Medical Image Anal..

[90]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[91]  Christof Koch,et al.  Automated event detection in underwater video , 2003, Oceans 2003. Celebrating the Past ... Teaming Toward the Future (IEEE Cat. No.03CH37492).

[92]  Til Aach,et al.  Illumination-Invariant Change Detection Using a Statistical Colinearity Criterion , 2001, DAGM-Symposium.

[93]  David J. Fleet,et al.  Robustly Estimating Changes in Image Appearance , 2000, Comput. Vis. Image Underst..

[94]  Elena Stringa Morphological Change Detection Algorithms for Surveillance Applications , 2000, BMVC.

[95]  Yoram Yakimovsky,et al.  Boundary and Object Detection in Real World Images , 1974, JACM.

[96]  Takeo Kanade,et al.  Introduction to the Special Section on Video Surveillance , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[97]  Georgios Tziritas,et al.  Adaptive detection and localization of moving objects in image sequences , 1999, Signal Process. Image Commun..

[98]  Paul C. Smits,et al.  Toward specification-driven change detection , 2000, IEEE Trans. Geosci. Remote. Sens..

[99]  Boon-Lock Yeo,et al.  Segmentation of Video by Clustering and Graph Analysis , 1998, Comput. Vis. Image Underst..

[100]  Ramesh C. Jain,et al.  Illumination independent change detection for real world image sequences , 1989, Comput. Vis. Graph. Image Process..

[101]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[102]  G. Nagy,et al.  Technique to measure 3D work-of-fracture of concrete in compression , 1999 .

[103]  Mark J. Carlotto Detection and analysis of change in remotely sensed imagery with application to wide area surveillance , 1997, IEEE Trans. Image Process..

[104]  M. Bauer,et al.  Digital change detection in forest ecosystems with remote sensing imagery , 1996 .

[105]  J W Berger,et al.  Computerized stereochronoscopy and alternation flicker to detect optic nerve head contour change. , 2000, Ophthalmology.

[106]  Chia-Ling Tsai,et al.  Automated Model-Based Segmentation, Tracing, and Analysis of Retinal Vasculature from Digital Fundus Images , 2003 .

[107]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[108]  Lorenzo Bruzzone,et al.  Automatic analysis of the difference image for unsupervised change detection , 2000, IEEE Trans. Geosci. Remote. Sens..

[109]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[110]  Joachim M. Buhmann,et al.  Topology Free Hidden Markov Models: Application to Background Modeling , 2001, ICCV.

[111]  Michael J. Brooks,et al.  Detecting suspicious background changes in video surveillance of busy scenes , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[112]  Visvanathan Ramesh,et al.  Sudden illumination change detection using order consistency , 2004, Image Vis. Comput..

[113]  PeterDeerandPeterEklund Schoolof,et al.  VALUES FOR THE FUZZY -MEANS CLASSIFIER IN CHANGE DETECTION FOR REMOTE SENSING , 2001 .

[114]  Touradj Ebrahimi,et al.  Change Detection by Nonlinear Grammian , 2001 .

[115]  J. S. Whorff,et al.  A video recording and analysis system used to sample intertidal communities , 1992 .

[116]  Christopher Justice,et al.  The impact of misregistration on change detection , 1992, IEEE Trans. Geosci. Remote. Sens..

[117]  Emanuele Trucco,et al.  Real-time automatic sea-floor change detection from video , 2000, OCEANS 2000 MTS/IEEE Conference and Exhibition. Conference Proceedings (Cat. No.00CH37158).

[118]  Fabrice Heitz,et al.  Automatic change detection in multimodal serial MRI: application to multiple sclerosis lesion evolution , 2003, NeuroImage.

[119]  J. Besag On the Statistical Analysis of Dirty Pictures , 1986 .

[120]  D. Stow Reducing the effects of misregistration on pixel-level change detection , 1999 .

[121]  Keith R. Matthews,et al.  Elementary Linear Algebra , 1998 .

[122]  Stan Z. Li,et al.  Markov Random Field Modeling in Image Analysis , 2001, Computer Science Workbench.

[123]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[124]  W. Malila Change Vector Analysis: An Approach for Detecting Forest Changes with Landsat , 1980 .