Emergence of simple-cell receptive field properties by learning a sparse code for natural images

THE receptive fields of simple cells in mammalian primary visual cortex can be characterized as being spatially localized, oriented1–4 and bandpass (selective to structure at different spatial scales), comparable to the basis functions of wavelet transforms5,6. One approach to understanding such response properties of visual neurons has been to consider their relationship to the statistical structure of natural images in terms of efficient coding7–12. Along these lines, a number of studies have attempted to train unsupervised learning algorithms on natural images in the hope of developing receptive fields with similar properties13–18, but none has succeeded in producing a full set that spans the image space and contains all three of the above properties. Here we investigate the proposal8,12 that a coding strategy that maximizes sparseness is sufficient to account for these properties. We show that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex. The resulting sparse image code provides a more efficient representation for later stages of processing because it possesses a higher degree of statistical independence among its outputs.

[1]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[2]  D. G. Albrecht,et al.  Spatial frequency selectivity of cells in macaque visual cortex , 1982, Vision Research.

[3]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[4]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[5]  A. Parker,et al.  Two-dimensional spatial structure of receptive fields in monkey striate cortex. , 1988, Journal of the Optical Society of America. A, Optics and image science.

[6]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[7]  J.G. Daugman,et al.  Entropy reduction and decorrelation in visual coding by oriented neural receptive fields , 1989, IEEE Transactions on Biomedical Engineering.

[8]  Nathan Intrator,et al.  Feature Extraction Using an Unsupervised Neural Network , 1992, Neural Computation.

[9]  Leslie S. Smith,et al.  The principal components of natural images , 1992 .

[10]  J. H. van Hateren,et al.  Real and optimal neural images in early vision , 1992, Nature.

[11]  Eric L. Schwartz,et al.  Computational Neuroscience , 1993, Neuromethods.

[12]  D. Ruderman The statistics of natural images , 1994 .

[13]  C. C. Law,et al.  Formation of receptive fields in realistic visual environments according to the Bienenstock, Cooper, and Munro (BCM) theory. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[14]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[15]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[16]  Geoffrey E. Hinton,et al.  The "wake-sleep" algorithm for unsupervised neural networks. , 1995, Science.

[17]  C. Fyfe,et al.  Finding compact and sparse-distributed representations of visual images , 1995 .

[18]  Eric Saund,et al.  A Multiple Cause Mixture Model for Unsupervised Learning , 1995, Neural Computation.

[19]  Jürgen Schmidhuber,et al.  Semilinear Predictability Minimization Produces Well-Known Feature Detectors , 1996, Neural Computation.