Efficient coding of natural time varying images in the early visual system.

We investigate the hypothesis that the early visual system efficiently codes natural time varying images, first by tracking part of the image, then by matching the spatiotemporal properties of the neural pathway to those of the tracked image. A representation for the time varying image is formulated which consists of two spatiotemporal components, a velocity field component and a stationary component. We show, using digitized sequences of natural images, that the spatiotemporal spectrum and other attributes of the image markedly differ before and after tracking. The temporal frequency bandwidth and velocity distribution of the velocity field component are diminished in the region of tracking and broaden with increasing eccentricity from this region. On the other hand, the spectrum of the stationary component is unaffected by tracking. Comparison of the properties of the tracked image to those of the M and P pathways suggests that each pathway transmits different attributes of the tracked image. A retinal architecture which varies with eccentricity also matches the properties of the tracked image.

[1]  Brian J. Murphy,et al.  Pattern thresholds for moving and stationary gratings during smooth eye movement , 1978, Vision Research.

[2]  S. Kassam,et al.  Robust signal processing for communication systems , 1983, IEEE Communications Magazine.

[3]  D. H. Kelly.  Motion and vision. II. Stabilized spatio-temporal threshold surface. , 1979, Journal of the Optical Society of America.

[4]  Trichur Raman Vidyasagar,et al.  The responses of cells in macaque lateral geniculate nucleus to sinusoidal gratings. , 1983, The Journal of physiology.

[5]  A B Watson,et al.  Perceptual-components architecture for digital video. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[6]  Lawrence Stark,et al.  Predictive Control of Eye Tracking Movements , 1962 .

[7]  William H. Merigan,et al.  Spatio-temporal vision of macaques with severe loss of Pβ retinal ganglion cells , 1986, Vision Research.

[8]  C. J. Keemink,et al.  Contrast sensitivity for oscillating sine wave gratings during ocular fixation and pursuit , 1988, Vision Research.

[9]  Gershon Buchsbaum,et al.  A computational model of spatiochromatic image coding in early vision , 1991, J. Vis. Commun. Image Represent..

[10]  N. Logothetis,et al.  Role of the color-opponent and broad-band channels in vision , 1990, Visual Neuroscience.

[11]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[12]  E. J. Morris,et al.  Visual motion processing and sensory-motor integration for smooth pursuit eye movements. , 1987, Annual review of neuroscience.

[13]  Anil K. Jain,et al.  Displacement Measurement and Its Application in Interframe Image Coding , 1981, IEEE Trans. Commun..

[14]  B. B. Lee,et al.  Amplitude and phase of responses of macaque retinal ganglion cells to flickering stimuli. , 1989, The Journal of physiology.

[15]  Andrew B. Watson,et al.  A look at motion in the frequency domain , 1983 .

[16]  Allan W. Snyder,et al.  Information capacity of eyes , 1977, Vision Research.

[17]  B. B. Lee,et al.  Sensitivity of macaque retinal ganglion cells to chromatic and luminance flicker. , 1989, The Journal of physiology.

[18]  R. Shapley,et al.  Light adaptation in the primate retina: Analysis of changes in gain and dynamics of monkey retinal ganglion cells , 1990, Visual Neuroscience.

[19]  R. Shapley,et al.  X and Y cells in the lateral geniculate nucleus of macaque monkeys. , 1982, The Journal of physiology.

[20]  S. Laughlin,et al.  Matching Coding to Scenes to Enhance Efficiency , 1983 .

[21]  A. Hughes.  The Topography of Vision in Mammals of Contrasting Life Style: Comparative Optics and Retinal Organisation , 1977 .

[22]  Alex Pentland.  Photometric Motion , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  G R Barnes,et al.  Head‐free pursuit in the human of a visual target moving in a pseudo‐random manner. , 1989, The Journal of physiology.

[24]  C. Blakemore,et al.  Organization and post‐natal development of the monkey's lateral geniculate nucleus. , 1986, The Journal of physiology.

[25]  J. Koenderink,et al.  The distribution of human motion detector properties in the monocular visual field , 1986, Vision Research.

[26]  R. Shapley,et al.  Cat and monkey retinal ganglion cells and their visual functional roles , 1986, Trends in Neurosciences.

[27]  Curtis L. Baker,et al.  Eccentricity-dependent scaling of the limits for short-range apparent motion perception , 1985, Vision Research.

[28]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[29]  Andrew B. Watson,et al.  Separability of Spatiotemporal Spectra of Image Sequences , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  H B Barlow,et al.  The Ferrier lecture, 1980 , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[31]  G Buchsbaum,et al.  Inference of global spatiochromatic mechanisms from contrast sensitivity functions. , 1988, Journal of the Optical Society of America. A, Optics and image science.

[32]  B. B. Lee,et al.  Visual resolution of macaque retinal ganglion cells. , 1988, The Journal of physiology.

[33]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[34]  G. Buchsbaum,et al.  Trichromacy, opponent colours coding and optimum colour information transmission in the retina , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[35]  J. Maunsell,et al.  The effects of parvocellular lateral geniculate lesions on the acuity and contrast sensitivity of macaque monkeys , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[36]  Bernd Girod,et al.  The Efficiency of Motion-Compensating Prediction for Hybrid Coding of Video Sequences , 1987, IEEE J. Sel. Areas Commun..

[37]  P. Sterling,et al.  "Collective coding" of correlated cone signals in the retinal ganglion cell. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[38]  P. Lennie,et al.  Spatial and temporal contrast sensitivities of neurones in lateral geniculate nucleus of macaque. , 1984, The Journal of physiology.

[39]  S. McKee,et al.  The detection of motion in the peripheral visual field , 1984, Vision Research.

[40]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[41]  R. Young,et al.  Spatial summation and conduction latency classification of cells of the lateral geniculate nucleus of macaques , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[42]  J. Maunsell,et al.  Macaque vision after magnocellular lateral geniculate lesions , 1990, Visual Neuroscience.

[43]  Eileen Kowler,et al.  New directions for oculomotor research , 1990, Vision Research.

[44]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[45]  E. Ludvigh,et al.  The effect of relative motion on visual acuity. , 1962, Survey of ophthalmology.

[46]  S.A. Kassam,et al.  Robust techniques for signal processing: A survey , 1985, Proceedings of the IEEE.

[47]  J. van Santen,et al.  Elaborated Reichardt detectors. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[48]  N. Jayant,et al.  Digital Coding of Waveforms: Principles and Applications to Speech and Video , 1990 .