Convergent Algorithm for Sensory Receptive Field Development

An unsupervised developmental algorithm for linear maps is derived which reduces the pixel-entropy (using the measure introduced in previous work) at every update and thus removes pairwise correlations between pixels. Since the measure of pixel-entropy has only a global minimum the algorithm is guaranteed to converge to the minimum entropy map. Such optimal maps have recently been shown to possess cognitively desirable properties and are likely to be used by the nervous system to organize sensory information. The algorithm derived here turns out to be one proposed by Goodall for pairwise decorrelation. It is biologically plausible since in a neural network implementation it requires only data available locally to a neuron. In training over ensembles of two-dimensional input signals with the same spatial power spectrum as natural scenes, networks develop output neurons with center-surround receptive fields similar to those of ganglion cells in the retina. Some technical issues pertinent to developmental algorithms of this sort, such as symmetry fixing, are also discussed.

[1]  M. C. GOODALL,et al.  Performance of a Stochastic Net , 1960, Nature.

[2]  E. Oja Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[3]  R. Linsker,et al.  From basic network principles to neural architecture , 1986 .

[4]  H. Guy,et al.  Molecular model of the action potential sodium channel. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[5]  R Linsker,et al.  From basic network principles to neural architecture: emergence of spatial-opponent cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[6]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[7]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[8]  P. Foldiak,et al.  Adaptive network for optimal linear feature extraction , 1989, International 1989 Joint Conference on Neural Networks.

[9]  H. B. Barlow,et al.  Finding Minimum Entropy Codes , 1989, Neural Computation.

[10]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[11]  Richard Durbin,et al.  The computing neuron , 1989 .

[12]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[13]  Joseph J. Atick,et al.  Predicting Ganglion and Simple Cell Receptive Field Organizations , 1991, Int. J. Neural Syst..

[14]  J J Hopfield,et al.  Olfactory computation and object perception. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Joseph J. Atick,et al.  What Does the Retina Know about Natural Scenes? , 1992, Neural Computation.

[16]  Zhaoping Li,et al.  What does post-adaptation color appearance reveal about cortical color representation? , 1993, Vision Research.

[17]  Mark D. Plumbley Efficient information transfer and anti-Hebbian neural networks , 1993, Neural Networks.

[18]  A. Norman Redlich,et al.  Supervised Factorial Learning , 1993, Neural Computation.