Computing optical flow across multiple scales: An adaptive coarse-to-fine strategy

Single-scale approaches to the determination of the optical flow field from the time-varying brightness pattern assume that spatio-temporal discretization is adequate for representing the patterns and motions in a scene. However, the choice of an appropriate spatial resolution is subject to conflicting, scene-dependent, constraints. In intensity-base methods for recovering optical flow, derivative estimation is more accurate for long wavelengths and slow velocities (with respect to the spatial and temporal discretization steps). On the contrary, short wavelengths and fast motions are required in order to reduce the errors caused by noise in the image acquisition and quantization process.Estimating motion across different spatial scales should ameliorate this problem. However, homogeneous multiscale approaches, such as the standard multigrid algorithm, do not improve this situation, because an optimal velocity estimate at a given spatial scale is likely to be corrupted at a finer scale. We propose an adaptive multiscale method, where the discretization scale is chosen locally according to an estimate of the relative error in the velocity estimation, based on image properties.Results for synthetic and video-acquired images show that our coarse-to-fine method, fully parallel at each scale, provides substantially better estimates of optical flow than do conventional algorithms, while adding little computational cost.

[1]  Christof Koch,et al.  A Multiscale Adaptive Network Model of Motion Computation in Primates , 1990, NIPS.

[2]  P. J. Burt,et al.  The Pyramid as a Structure for Efficient Computation , 1984 .

[3]  A. Verri,et al.  A computational approach to motion perception , 1988, Biological Cybernetics.

[4]  Norberto M. Grzywacz,et al.  A computational theory for the perception of coherent visual motion , 1988, Nature.

[5]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[7]  Jin Luo,et al.  Analog hardware for detecting discontinuities in early vision , 1990, International Journal of Computer Vision.

[8]  Roberto Battiti Multiscale methods, parallel computation, and neural networks for real-time computer vision , 1991 .

[9]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[10]  Christof Koch,et al.  Computing Optical Flow in the Primate Visual System , 1989, Neural Computation.

[11]  Joseph K. Kearney,et al.  Optical Flow Estimation: An Error Analysis of Gradient-Based Methods with Local Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  D Marr,et al.  Directional selectivity and its use in early visual processing , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[13]  A. Verri,et al.  Analysis of differential and matching methods for optical flow , 1989, [1989] Proceedings. Workshop on Visual Motion.

[14]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[15]  Shimon Ullman Analysis of visual motion by biological and computer systems , 1987 .

[16]  Demetri Terzopoulos,et al.  Image Analysis Using Multigrid Relaxation Methods , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  D. Brandt,et al.  Multi-level adaptive solutions to boundary-value problems math comptr , 1977 .

[18]  A. Verri,et al.  Constraints for the computation of optical flow , 1989, [1989] Proceedings. Workshop on Visual Motion.

[19]  W. Reichardt,et al.  Movement Detectors of the Correlation Type Provide Sufficient Information for Local Computation of the 2-D Velocity Field , 1988 .

[20]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[21]  R. Battiti,et al.  Surface Reconstruction and Discontinuity Detection: A Fast Hierarchical Approach on a Two-Dimensional Mesh , 1990, Proceedings of the Fifth Distributed Memory Computing Conference, 1990..

[22]  M. Egelhaaf,et al.  Movement detectors provide sufficient information for local computation of 2-D velocity field , 1988, Naturwissenschaften.

[23]  John B. Goodenough The Ada Compiler Validation Capability , 1981 .

[24]  Jin Luo,et al.  Computing motion using analog and binary resistive networks , 1988, Computer.

[25]  G. Wasilkowski,et al.  Computing optical flow , 1989, [1989] Proceedings. Workshop on Visual Motion.

[26]  C Koch,et al.  Computing motion in the primate's visual system. , 1989, The Journal of experimental biology.

[27]  J. van Santen,et al.  Temporal covariance model of human motion perception. , 1984, Journal of the Optical Society of America. A, Optics and image science.

[28]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[29]  Ellen C. Hildreth,et al.  Computations Underlying the Measurement of Visual Motion , 1984, Artif. Intell..

[30]  T Poggio,et al.  Parallel integration of vision modules. , 1988, Science.

[31]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[32]  F. Glazer Multilevel Relaxation in Low-Level Computer Vision , 1984 .

[33]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[34]  Behrooz Kamgar-Parsi,et al.  Evaluation of quantization error in computer vision , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  T. Poggio,et al.  Considerations on models of movement detection , 1973, Kybernetik.

[36]  Wilfried Enkelmann,et al.  Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences , 1988, Comput. Vis. Graph. Image Process..

[37]  B. Hassenstein,et al.  Systemtheoretische Analyse der Zeit-, Reihenfolgen- und Vorzeichenauswertung bei der Bewegungsperzeption des Rüsselkäfers Chlorophanus , 1956 .

[38]  J. Marroquín Surface Reconstruction Preserving Discontinuities , 1984 .

[39]  T. Poggio,et al.  A parallel algorithm for real-time computation of optical flow , 1989, Nature.

[40]  C. Koch,et al.  The analysis of visual motion: from computational theory to neuronal mechanisms. , 1986, Annual review of neuroscience.

[41]  Tomaso A. Poggio,et al.  Motion Field and Optical Flow: Qualitative Properties , 1989, IEEE Trans. Pattern Anal. Mach. Intell..