THE PREDICTIVE BRAIN Computational Consequences of Volume Learning

Introduction Some forms of synaptic plasticity depend on the temporal coincidence of presynaptic activiiy and postsynaptic response. This requirement is consistent with the Hebbian, or correlational, type of learning rule used in many neural network models. Recent evidence suggests that synaptic plasticity may depend in pan on the production of a membrane permeant-diffusible signal so that spatial volume may also be involved in correlational learning rules. This latter form of synaptic change has been called volume learning. In both Hebbian and volume learning rules, interaction among synaptic inputs depends on the degree of coincidence of the inputs and is otherwise insensitive to theh exact temporal order. Conditioning experiments and psychophysical studies have shown, however, that most animals are highly sensitive to the temporal order of the sensory inputs. Although these experiments assay the behavior of the entire animal or perceptual system, they raise the possibility that nervous systems may he sensitive to temporally ordered events at many spatial and temporal scales. We suggest here the existence of a new class of learning rule, called apredictfve Hebbian learning rule, that is sensitive to the temporal ordering of synaptic inputs. We show how this predictive learning rule could act at single synaptic connections and through diffuse neuromodulatory systems. Most biologically feasible theories of how experience-dependent changes take place in real neuronal networks use some variant of the notion that the efficacy or "strength of a synaptic connection from one cell to another can be modikied on the basis of its history. In this theoretical work it is generally assumed that modikications of synaptic efficacy, by acting over a large population of synapses, can account for interesting forms of learning and memory. This theoretical assumption prevails primarily because of its intuitive appeal, its accessibility to analysis, some provocative relations to biological data, and a lack of good alternatives. Recent work demonstrates that simple abstract learning algorithms, if given appropriately coded input, can produce complicated mappings from input to output. These efforts include networks that learn to pronounce written text (Sejnowski and Rosenberg 1987), play master level backgammon (Tesauro 1994), and recognize handwritten characters (Le Cun et al. 1990). As pointed out by Crick (1989) and others, many of these efforts are not good models of the vertebrate brain; however, they can be quite valuable for identifying the informational requirements LEARNING & MEMORY 1 1-33 B 1994 by Cold Sprmg Harbor Laboratory Prerr lSSN1072-0502194 55 00 Montague and Sejnowski involved in specific tasks. Moreover, they point out some of the computational constraints to which brains are subject. An awareness of the computational constraints involved in a particular problem can guide theories that explain how real brains are constructed (Churchland and Sejnowski 1992). Although abstract networks have provided some insight into the topdown constraints that nervous systems face, these approaches are of limited use in gaining insight into how various problems have been solved by real brains. For example, the actual learning mechanisms that are used in biological systems also satisfy additional constraints that arise from the known properties of neurons and synapses. In this paper we focus on learning rules that are supported by biological data and consider the strengths and weaknesses of these rules by measuring them against both computational and biological constraints. Taking this dual approach, we show that computational concerns applicable to the behavior and survival of the animal can work hand in hand with biologically feasible synaptic mechanisms to explain and predict experimental data. Correlational Theoretical accounts of how neural activity actually changes synaptic Learning function typically rely on a local correlational learning rule to model Rules--Learning synaptic plasticity. A correlational learning rule, often called a Hebbian Driven by Temporal learning rule, uses the correlation between presynaptic activity and Coincidence postsynaptic response to drive changes in synaptic efficacy (Fig. 1 ) (Hertz et al. 1991; Churchland and Sejnowski 1992). One simple expression of a Hebbian learning rule is

[1]  Terrence J. Sejnowski,et al.  The Computational Brain , 1996, Artif. Intell..

[2]  Gerald Tesauro,et al.  TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play , 1994, Neural Computation.

[3]  Charles F. Stevens,et al.  Reversal of long-term potentiation by inhibitors of haem oxygenase , 1993, Nature.

[4]  E. Kandel,et al.  Nitric oxide and carbon monoxide produce activity-dependent long-term synaptic enhancement in hippocampus. , 1993, Science.

[5]  Tim V. P. Bliss,et al.  The search for retrograde messengers in long-term potentiation , 1993 .

[6]  D. O'Leary,et al.  N-methyl-D-aspartate receptor antagonists disrupt the formation of a mammalian neural map. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[7]  J. Kaas,et al.  Dynamic features of sensory and motor maps , 1992, Current Biology.

[8]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[9]  S. Lipton,et al.  Effect of nitric oxide production on the redox modulatory site of the NMDA receptor-channel complex , 1992, Neuron.

[10]  M. Bear,et al.  Homosynaptic long-term depression in area CA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[11]  T. Robbins,et al.  Forebrain norepinephrine: role in controlled information processing in the rat. , 1992, Neuropsychopharmacology : official publication of the American College of Neuropsychopharmacology.

[12]  T. Wiesel,et al.  Receptive field dynamics in adult primary visual cortex , 1992, Nature.

[13]  A. Damasio,et al.  Preserved access and processing of social knowledge in a patient with acquired sociopathy due to ventromedial frontal damage , 1991, Neuropsychologia.

[14]  J. Rauschecker,et al.  Mechanisms of visual plasticity: Hebb synapses, NMDA receptors, and beyond. , 1991, Physiological reviews.

[15]  G. Edelman,et al.  The NO hypothesis: possible effects of a short-lived, rapidly diffusible signal in the development and function of the nervous system. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[16]  D. O'Leary,et al.  Target control of collateral extension and directional axon growth in the mammalian brain. , 1990, Science.

[17]  W. Harris Local positional cues in the neuroepithelium guide retinal axons in embryonic Xenopus brain , 1989, Nature.

[18]  Francis Crick,et al.  The recent excitement about neural networks , 1989, Nature.

[19]  C. H. Bailey,et al.  The anatomy of a memory: convergence of results across a diversity of tests , 1988, Trends in Neurosciences.

[20]  M. Merzenich,et al.  Receptive fields in the body-surface map in adult cortex defined by temporally correlated inputs , 1988, Nature.

[21]  W. Singer,et al.  Blockade of "NMDA" receptors disrupts experience-dependent plasticity of kitten striate cortex. , 1987, Science.

[22]  Masao Ito Long-term depression as a memory process in the cerebellum , 1986, Neuroscience Research.

[23]  S. Kelso,et al.  Hebbian synapses in hippocampus. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[24]  A. Damasio,et al.  Knowledge without awareness: an autonomic index of facial recognition by prosopagnosics. , 1985, Science.

[25]  H. Wigström,et al.  On long-lasting potentiation in the hippocampus: a proposed mechanism for its dependence on coincident pre- and postsynaptic activity. , 1985, Acta physiologica Scandinavica.

[26]  A R Damasio,et al.  Amnesia following basal forebrain lesions. , 1985, Archives of neurology.

[27]  E. Kandel,et al.  Is there a cell-biological alphabet for simple forms of learning? , 1984, Psychological review.

[28]  M. Cynader,et al.  Somatosensory cortical map changes following digit amputation in adult monkeys , 1984, The Journal of comparative neurology.

[29]  R L Meyer,et al.  Tetrodotoxin blocks the formation of ocular dominance columns in goldfish. , 1982, Science.

[30]  R. Wise Neuroleptics and operant behavior: The anhedonia hypothesis , 1982, Behavioral and Brain Sciences.

[31]  G M Edelman,et al.  Selective networks capable of representative transformations, limited generalizations, and associative memory. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[32]  D. Willshaw The establishment and the subsequent elimination of polyneuronal innervation of developing muscle: theoretical considerations , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[33]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[34]  D J Willshaw,et al.  A marker induction mechanism for the establishment of ordered neural mappings: its application to the retinotectal problem. , 1979, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[35]  C. Malsburg,et al.  How to label nerve cells so that they can interconnect in an ordered fashion. , 1977, Proceedings of the National Academy of Sciences of the United States of America.

[36]  C. Malsburg,et al.  How patterned neural connections can be set up by self-organization , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[37]  C. Malsburg Self-organization of orientation sensitive cells in the striate cortex , 2004, Kybernetik.

[38]  P. Dayan,et al.  Volume Learning: Signaling Covariance Through Neural Tissue , 1993 .

[39]  J. Kaas Plasticity of sensory and motor maps in adult mammals. , 1991, Annual review of neuroscience.

[40]  Richard S. Sutton,et al.  Time-Derivative Models of Pavlovian Reinforcement , 1990 .

[41]  David J. C. MacKay,et al.  Analysis of Linsker's Simulations of Hebbian Rules , 1990, Neural Computation.

[42]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[43]  R. Rescorla A theory of pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement , 1972 .