A Nonlinear Hebbian Network that Learns to Detect Disparity in Random-Dot Stereograms

An intrinsic limitation of linear, Hebbian networks is that they are capable of learning only from the linear pairwise correlations within an input stream. To explore what higher forms of structure could be learned with a nonlinear Hebbian network, we constructed a model network containing a simple form of nonlinearity and we applied it to the problem of learning to detect the disparities present in random-dot stereograms. The network consists of three layers, with nonlinear sigmoidal activation functions in the second-layer units. The nonlinearities allow the second layer to transform the pixel-based representation in the input layer into a new representation based on coupled pairs of left-right inputs. The third layer of the network then clusters patterns occurring on the second-layer outputs according to their disparity via a standard competitive learning rule. Analysis of the network dynamics shows that the second-layer units' nonlinearities interact with the Hebbian learning rule to expand the region over which pairs of left-right inputs are stable. The learning rule is neurobiologically inspired and plausible, and the model may shed light on how the nervous system learns to use coincidence detection in general.

[1]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[2]  G. Poggio,et al.  Stereoscopic mechanisms in monkey visual cortex: binocular correlation and disparity selectivity , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  Geoffrey E. Hinton,et al.  Self-organizing neural network that discovers surfaces in random-dot stereograms , 1992, Nature.

[4]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[5]  K. Miller,et al.  Ocular dominance column development: analysis and simulation. , 1989, Science.

[6]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry, Expanded Edition , 1987 .

[7]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[8]  Terence D. Sanger,et al.  An Optimality Principle for Unsupervised Learning , 1988, NIPS.

[9]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[10]  M. Konishi,et al.  A circuit for detection of interaural time differences in the brain stem of the barn owl , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[11]  Ning Qian,et al.  Computing Stereo Disparity and Motion with Known Binocular Cell Properties , 1994, Neural Computation.

[12]  B. McNaughton,et al.  Synaptic enhancement in fascia dentata: Cooperativity among coactive afferents , 1978, Brain Research.

[13]  Harry G. Barrow,et al.  The Role of Weight Normalization in Competitive Learning , 1994, Neural Computation.

[14]  Margaret E. Sereno,et al.  Learning to See Rotation and Dilation with a Hebb Rule , 1990, NIPS.

[15]  Kenneth D. Miller,et al.  The Role of Constraints in Hebbian Learning , 1994, Neural Computation.

[16]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[17]  D. Mackay,et al.  Analysis of Linsker's application of Hebbian rules to linear networks , 1990 .

[18]  Tomaso Poggio,et al.  Cooperative computation of stereo disparity , 1988 .

[19]  B. Julesz.  Foundations of Cyclopean Perception , 1971 .

[20]  P. Dayan,et al.  A correlational model for the development of disparity selectivity in visual cortex that depends on prenatal and postnatal phases. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[21]  David J. Field,et al.  Sparse Coding of Natural Images Produces Localized, Oriented, Bandpass Receptive Fields , 1995 .

[22]  Kenneth D. Miller,et al.  Derivation of Linear Hebbian Equations from a Nonlinear Hebbian Model of Synaptic Plasticity , 1990, Neural Computation.

[23]  Klaus Schulten,et al.  Models of Orientation and Ocular Dominance Columns in the Visual Cortex: A Critical Comparison , 1995, Neural Computation.

[24]  W. Levy,et al.  Insights into associative long-term potentiation from computational models of NMDA receptor-mediated calcium influx and intracellular calcium concentration changes. , 1990, Journal of neurophysiology.

[25]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.

[26]  T. Bliss,et al.  A synaptic model of memory: long-term potentiation in the hippocampus , 1993, Nature.

[27]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[28]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[29]  David J. Fleet,et al.  Phase-based disparity measurement , 1991, CVGIP Image Underst..

[30]  N Suga,et al.  Combination-sensitive neurons in the medial geniculate body of the mustached bat: encoding of target range information. , 1991, Journal of neurophysiology.

[31]  M. Konishi.  The neural algorithm for sound localization in the owl. , 1990, Harvey lectures.