Filter selection model for motion segmentation and velocity integration

We present a new approach to computing from image sequences the two-dimensional velocities of moving objects that are occluded and transparent. The new motion model does not attempt to provide an accurate representation of the velocity flow field at fine resolutions but coarsely segments an image into regions of coherent motion, provides an estimate of velocity in each region, and actively selects the most reliable estimates. The model uses motion-energy filters in the first stage of processing and computes, in parallel, two different sets of retinotopically organized spatial arrays of unit responses: one set of units estimates the local velocity, and the second set selects from these local estimates those that support global velocities. Only the subset of local-velocity measurements that are the most reliable is included in estimation of the velocity of objects. The model is in agreement with many of the constraints imposed by the physiological response properties of cells in primate visual cortex, and its performance is similar to that of primates on motion transparency.

[1]  W. Reichardt,et al.  Autocorrelation, a principle for the evaluation of sensory information by the central nervous system , 1961 .

[2]  J. Gibson.  The Senses Considered As Perceptual Systems , 1967 .

[3]  O. Braddick.  A short-range process in apparent motion. , 1974, Vision research.

[4]  J. Movshon,et al.  Spatial and temporal contrast sensitivity of striate cortical neurones , 1975, Nature.

[5]  R. Shapley,et al.  Quantitative analysis of retinal ganglion cell classifications. , 1976, The Journal of physiology.

[6]  L. Maffei,et al.  Spatial frequency rows in the striate visual cortex , 1977, Vision Research.

[7]  D. Pollen,et al.  Relationship between spatial frequency selectivity and receptive field profile of simple cells. , 1979, The Journal of physiology.

[8]  J. Daugman.  Two-dimensional spectral analysis of cortical receptive field profiles , 1980, Vision Research.

[9]  D Marr,et al.  Directional selectivity and its use in early visual processing , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[10]  Steven W. Zucker,et al.  Continuous Relaxation and Local Maxima Selection: Conditions for Equivalence , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  R. Holub,et al.  Response of Visual Cortical Neurons of the cat to moving sinusoidal gratings: response-contrast functions and spatiotemporal interactions. , 1981, Journal of neurophysiology.

[12]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[13]  E. Adelson,et al.  Phenomenal coherence of moving visual patterns , 1982, Nature.

[14]  Steven W. Zucker,et al.  On the Foundations of Relaxation Labeling Processes , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Peter J. Burt,et al.  Fast algorithms for estimating local image properties , 1982, Comput. Graph. Image Process..

[16]  Geoffrey E. Hinton,et al.  Parallel visual computation , 1983, Nature.

[17]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  T. Albright.  Direction and orientation selectivity of neurons in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[19]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[20]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[21]  S. McKee,et al.  Sequential recruitment in the discrimination of velocity. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[22]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[23]  E. Adelson,et al.  The analysis of moving visual patterns , 1985 .

[24]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[25]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[26]  J. van Santen,et al.  Elaborated Reichardt detectors. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[27]  Geoffrey E. Hinton,et al.  Separating Figure from Ground with a Parallel Network , 1986, Perception.

[28]  J. Maunsell,et al.  Visual processing in monkey extrastriate cortex. , 1987, Annual review of neuroscience.

[29]  V. Ramachandran,et al.  Motion capture anisotropy , 1987, Vision Research.

[30]  Klein,et al.  Nonlinear directionally selective subunits in complex cells of cat striate cortex. , 1987, Journal of neurophysiology.

[31]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[32]  Hans-Hellmut Nagel,et al.  On the Estimation of Optical Flow: Relations between Different Approaches and Some New Results , 1987, Artif. Intell..

[33]  R. Shapley,et al.  Linear mechanisms of directional selectivity in simple cells of cat striate cortex. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[34]  K. Nakayama,et al.  The aperture problem—I. Perception of nonrigidity and motion direction in translating sinusoidal lines , 1988, Vision Research.

[35]  F. Mosteller,et al.  Exploring Data Tables, Trends and Shapes. , 1988 .

[36]  Robert M. Haralick,et al.  2D-3D pose estimation , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[37]  W. Newsome,et al.  A selective impairment of motion perception following lesions of the middle temporal visual area (MT) , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[38]  L. Palmer,et al.  Contribution of linear spatiotemporal receptive field structure to velocity selectivity of simple cells in area 17 of cat , 1989, Vision Research.

[39]  John Scott Bridle,et al.  Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition , 1989, NATO Neurocomputing.

[40]  Leslie Welch,et al.  The perception of moving plaids reveals two motion-processing stages , 1989, Nature.

[41]  C Koch,et al.  Computing motion in the primate's visual system. , 1989, The Journal of experimental biology.

[42]  A. B. Bonds.  Role of Inhibition in the Specification of Orientation Selectivity of Cells in the Cat Striate Cortex , 1989, Visual Neuroscience.

[43]  D. G. Albrecht,et al.  Visual cortical receptive fields in monkey and cat: Spatial and temporal phase transfer function , 1989, Vision Research.

[44]  S. Lehky,et al.  Neural model of stereoacuity and depth interpolation based on a distributed representation of stereo disparity [published erratum appears in J Neurosci 1991 Mar;11(3):following Table of Contents] , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[45]  G. Edelman,et al.  Signal and sense : local and global order in perceptual maps , 1990 .

[46]  A. Yuille,et al.  A model for the estimate of local image velocity by cells in the visual cortex , 1990, Proceedings of the Royal Society of London. B. Biological Sciences.

[47]  P. A. Sandon.  Simulating Visual Attention , 1990, Journal of Cognitive Neuroscience.

[48]  T. D. Albright,et al.  Transparency and coherence in human motion perception , 1990, Nature.

[49]  Dana H. Ballard,et al.  Animate Vision , 1991, Artif. Intell..

[50]  A. Pentland,et al.  Robust estimation of a multi-layered motion representation , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[51]  P. A. Sandon.  Logarithmic search in a winner-take-all network , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[52]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[53]  Terrence J. Sejnowski,et al.  Filter Selection Model for Generating Visual Motion Signals , 1992, NIPS.

[54]  E. Adelson,et al.  Directionally selective complex cells and the computation of motion energy in cat visual cortex , 1992, Vision Research.

[55]  H. Wilson,et al.  A psychophysically motivated model for two-dimensional motion perception , 1992, Visual Neuroscience.

[56]  Leif H. Finkel,et al.  Object Discrimination Based on Depth-from-Occlusion , 1992, Neural Computation.

[57]  T D Albright,et al.  Form-cue invariant motion processing in primate visual cortex. , 1992, Science.

[58]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[59]  Mary M. Conte,et al.  Coherence and transparency of moving plaids composed of Fourier and non-Fourier gratings , 1992, Perception & psychophysics.

[60]  A. Rosenfeld,et al.  Perceptual motion transparency : the role of geometrical information , 1992 .

[61]  Norberto M. Grzywacz,et al.  A Local Model for Transparent Motions Based on Spatio-Temporal Filters , 1993 .

[62]  O. Braddick.  Segmentation versus integration in visual motion processing , 1993, Trends in Neurosciences.

[63]  T. Sejnowski,et al.  A critique of pure vision , 1993 .

[64]  R A Andersen,et al.  Transparent motion perception as detection of unbalanced motion signals. I. Psychophysics , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[65]  David J. Heeger,et al.  Model of visual motion sensing , 1994 .

[66]  L. Palmer,et al.  Contribution of linear mechanisms to the specification of local motion by simple cells in areas 17 and 18 of the cat , 1994, Visual Neuroscience.

[67]  Mary J. Bravo,et al.  Evidence for two speed signals: a coarse local signal for segregation and a precise global signal for discrimination , 1995, Vision Research.

[68]  T. Sejnowski,et al.  A selection model for motion processing in area MT of primates , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.