Learning attractors in an asynchronous, stochastic electronic neural network.

LANN27 is an electronic device implementing in discrete electronics a fully connected (full feedback) network of 27 neurons and 351 plastic synapses with stochastic Hebbian learning. Both neurons and synapses are dynamic elements, with two time constants--fast for neurons and slow for synapses. Learning, synaptic dynamics, is analogue and is driven in a Hebbian way by neural activities. Long-term memorization takes place on a discrete set of synaptic efficacies and is effected in a stochastic manner. The intense feedback between the nonlinear neural elements, via the learned synaptic structure, creates in an organic way a set of attractors for the collective retrieval dynamics of the neural system, akin to Hebbian learned reverberations. The resulting structure of the attractors is a record of the large-scale statistics in the uncontrolled, incoming flow of stimuli. As the statistics in the stimulus flow changes significantly, the attractors slowly follow it and the network behaves as a palimpsest--old is gradually replaced by new. Moreover, the slow learning creates attractors which render the network a prototype extractor: entire clouds of stimuli, noisy versions of a prototype, used in training, all retrieve the attractor corresponding to the prototype upon retrieval. Here we describe the process of studying the collective dynamics of the network, before, during and following learning, which is rendered complex by the richness of the possible stimulus streams and the large dimensionality of the space of states of the network. We propose sampling techniques and modes of representation for the outcome.

[1]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[2]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Stanislas Dehaene,et al.  Networks of Formal Neurons and Memory Palimpsests , 1986 .

[4]  G. Parisi.  A memory which forgets , 1986 .

[5]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[6]  Haim Sompolinsky,et al.  The Theory of Neural Networks: The Hebb Rule and Beyond , 1987 .

[7]  Y. Miyashita,et al.  Neuronal correlate of pictorial short-term memory in the primate temporal cortexYasushi Miyashita , 1988, Nature.

[8]  Y. Miyashita.  Neuronal correlate of visual associative long-term memory in the primate temporal cortex , 1988, Nature.

[9]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[10]  John Lazzaro.  Low-power silicon spiking neurons and axons , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[11]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[12]  T. Bliss,et al.  A synaptic model of memory: long-term potentiation in the hippocampus , 1993, Nature.

[13]  Y. Miyashita.  Inferior temporal cortex: where visual perception meets memory. , 1993, Annual review of neuroscience.

[14]  Daniel J. Amit,et al.  Learning in Neural Networks with Material Synapses , 1994, Neural Computation.

[15]  P. Goldman-Rakic,et al.  Modulation of memory fields by dopamine Dl receptors in prefrontal cortex , 1995, Nature.

[16]  Walter J. Freeman,et al.  The Hebbian paradigm reintegrated: Local reverberations as internal representations , 1995, Behavioral and Brain Sciences.

[17]  Nicolas Brunel,et al.  Global Spontaneous Activity and Local Structured (learned) Delay Activity in Cortex , 1995 .

[18]  Nicolas Brunel,et al.  Hebbian Learning of Context in Recurrent Neural Networks , 1996, Neural Computation.

[19]  Daniel J. Amit,et al.  Paradigmatic Working Memory (Attractor) Cell in IT Cortex , 1997, Neural Computation.

[20]  N Brunel,et al.  Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network. , 1998, Network.

[21]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .