Syntactic sequencing in Hebbian cell assemblies

Hebbian cell assemblies provide a theoretical framework for the modeling of cognitive processes that grounds them in the underlying physiological neural circuits. Recently we have presented an extension of cell assemblies by operational components which allows to model aspects of language, rules, and complex behaviour. In the present work we study the generation of syntactic sequences using operational cell assemblies timed by unspecific trigger signals. Syntactic patterns are implemented in terms of hetero-associative transition graphs in attractor networks which cause a directed flow of activity through the neural state space. We provide regimes for parameters that enable an unspecific excitatory control signal to switch reliably between attractors in accordance with the implemented syntactic rules. If several target attractors are possible in a given state, noise in the system in conjunction with a winner-takes-all mechanism can randomly choose a target. Disambiguation can also be guided by context signals or specific additional external signals. Given a permanently elevated level of external excitation the model can enter an autonomous mode, where it generates temporal grammatical patterns continuously.

[1]  L. A. Jeffress,et al.  Cerebral Mechanisms in Behavior , 1953 .

[2]  Daniel J. Amit Modeling brain function: Introduction , 1989 .

[3]  E. Bienenstock A model of neocortex , 1995 .

[4]  C. L. Giles,et al.  Sequence Learning - Paradigms, Algorithms, and Applications , 2001 .

[5]  Roland Heim,et al.  Theoretical Approaches to Complex Systems , 1978 .

[6]  Thomas Wennekers,et al.  Associative memory in networks of spiking neurons , 2001, Neural Networks.

[7]  Andreas Knoblauch,et al.  An associative model of cortical language and action processing , 2005 .

[8]  Thomas Wennekers,et al.  Synfire chains with conductance-based neurons: internal timing and coordination with timed input , 2005, Neurocomputing.

[9]  D. Margoliash,et al.  Neuronal populations and single cells representing learned auditory objects , 2003, Nature.

[10]  E. Koechlin,et al.  Broca's Area and the Hierarchical Organization of Human Behavior , 2006, Neuron.

[11]  Günther Palm,et al.  Sentence Understanding and Learning of New Words with Large-Scale Neural Networks , 2008, ANNPR.

[12]  J. A. Hertz,et al.  Analysis of Synnre Chains , 1995 .

[13]  M. Walton,et al.  Action sets and decisions in the medial frontal cortex , 2004, Trends in Cognitive Sciences.

[14]  Friedemann Pulvermüller,et al.  The Neuroscience of Language: On Brain Circuits of Words and Serial Order , 2003 .

[15]  Florentin Wörgötter,et al.  Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms , 2005, Neural Computation.

[16]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[17]  J. Feldman,et al.  Embodied meaning in a neural theory of language , 2004, Brain and Language.

[18]  W A Wickelgren,et al.  Webs, cell assemblies, and chunking in neural nets: introduction. , 1999, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[19]  Gerd Willwacher,et al.  Storage of a temporal pattern sequence in a network , 2004, Biological Cybernetics.

[20]  Gèunther Palm,et al.  Neural Assemblies: An Alternative Approach to Artificial Intelligence , 1982 .

[21]  Thomas Wennekers,et al.  Finite State Automata Resulting from Temporal Information Maximization and a Temporal Learning Rule , 2005, Neural Computation.

[22]  Friedemann Pulvermüller,et al.  A neuroanatomically grounded Hebbian-learning model of attention–language interactions in the human brain , 2008, The European journal of neuroscience.

[23]  Dean V Buonomano,et al.  Timing of neural responses in cortical organotypic slices , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Kristina M. Visscher,et al.  A Core System for the Implementation of Task Sets , 2006, Neuron.

[25]  M. Arbib From monkey-like action recognition to human language: An evolutionary framework for neurolinguistics , 2005, Behavioral and Brain Sciences.

[26]  Jeffrey D. Ullman,et al.  Formal languages and their relation to automata , 1969, Addison-Wesley series in computer science and information processing.

[27]  Günther Palm,et al.  An Associative Cortical Model of Language Understanding and Action Planning , 2005, IWINAC.

[28]  Günther Palm,et al.  Information capacity in recurrent McCulloch-Pitts networks with sparsely coded memory states , 1992 .

[29]  Giacomo Indiveri,et al.  Synaptic Plasticity and Spike-based Computation in VLSI Networks of Integrate-and-Fire Neurons , 2007 .

[30]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[31]  Günther Palm,et al.  Detecting Sequences and Understanding Language with Neural Associative Memories and Cell Assemblies , 2005, Biomimetic Neural Learning for Intelligent Robots.

[32]  Jan Mycielski,et al.  A model of the neocortex , 1988 .

[33]  William H. Press,et al.  Numerical recipes in C++: the art of scientific computing, 2nd Edition (C++ ed., print. is corrected to software version 2.10) , 1994 .

[34]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[35]  Shun-ichi Amari,et al.  Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements , 1972, IEEE Transactions on Computers.

[36]  E. Jarvis,et al.  Learned Birdsong and the Neurobiology of Human Language , 2004, Annals of the New York Academy of Sciences.

[37]  D. McCormick,et al.  Neocortical Network Activity In Vivo Is Generated through a Dynamic Balance of Excitation and Inhibition , 2006, The Journal of Neuroscience.

[38]  Jean,et al.  The Computer and the Brain , 1989, Annals of the History of Computing.

[39]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[40]  K. Meier,et al.  A new VLSI model of neural microcircuits including spike time dependent plasticity , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[41]  Thomas Wennekers,et al.  Operational Cell Assemblies as a Paradigm for Brain-Inspired Future Computing Architectures , 2006 .

[42]  Timothy Q. Gentner,et al.  Recursive syntactic pattern learning by songbirds , 2006, Nature.

[43]  J. Eggert,et al.  Unifying framework for neuronal assembly dynamics. , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[44]  A.C.C. Coolen,et al.  Chapter 15 Statistical mechanics of recurrent neural networks II — Dynamics , 2000, cond-mat/0006011.

[45]  Joe Z Tsien,et al.  Linking Hebb’s coincidence-detection to memory formation , 2000, Current Opinion in Neurobiology.

[46]  J. Tanji,et al.  Activity in the Lateral Prefrontal Cortex Reflects Multiple Steps of Future Events in Action Plans , 2006, Neuron.

[47]  Walter J. Freeman,et al.  The Hebbian paradigm reintegrated: Local reverberations as internal representations , 1995, Behavioral and Brain Sciences.

[48]  Michael A. Arbib,et al.  Synthetic brain imaging: grasping, mirror neurons and imitation , 2000, Neural Networks.

[49]  Friedemann Pulvermüller,et al.  Constituents of a neurological theory of language , 1992 .

[50]  E. Vaadia,et al.  Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. , 1993, Journal of neurophysiology.

[51]  R. Yuste,et al.  Attractor dynamics of network UP states in the neocortex , 2003, Nature.

[52]  Yuji Ikegaya,et al.  Synfire Chains and Cortical Songs: Temporal Modules of Cortical Activity , 2004, Science.

[53]  Horn,et al.  Neural networks with dynamical thresholds. , 1989, Physical review. A, General physics.

[54]  A. Coolen Statistical Mechanics of Recurrent Neural Networks I. Statics , 2000, cond-mat/0006010.

[55]  Piotr Dudek,et al.  Compact silicon neuron circuit with spiking and bursting behaviour , 2008, Neural Networks.

[56]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[57]  Adam Prügel-Bennett,et al.  Analysis of synfire chains , 1995 .

[58]  K. D. Punta,et al.  An ultra-sparse code underlies the generation of neural sequences in a songbird , 2002 .

[59]  Friedemann Pulvermüller,et al.  Language models based on Hebbian cell assemblies , 2006, Journal of Physiology-Paris.

[60]  Günther Palm,et al.  Modelling of syntactical processing in the cortex , 2007, Biosyst..

[61]  K. Lashley The problem of serial order in behavior , 1951 .

[62]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[63]  Günther Palm,et al.  Combining Visual Attention, Object Recognition and Associative Information Processing in a NeuroBotic System , 2005, Biomimetic Neural Learning for Intelligent Robots.

[64]  H. Sompolinsky,et al.  Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity , 1996, Science.

[65]  Wayne A. Wickelgren,et al.  I liked the postcard you sent Abe and I: Context-sensitive coding of syntax and other procedural knowledge , 1979 .

[66]  Thomas Wennekers,et al.  Dynamics of spatio-temporal patterns in associative networks of spiking neurons , 2000, Neurocomputing.

[67]  Jun Tani,et al.  Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment , 2008, PLoS Comput. Biol..

[68]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[69]  Martin Rehn,et al.  Sequence memory with dynamical synapses , 2004, Neurocomputing.

[70]  G. Rizzolatti,et al.  Motor and cognitive functions of the ventral premotor cortex , 2002, Current Opinion in Neurobiology.

[71]  Alessandro Treves,et al.  Free association transitions in models of cortical latching dynamics , 2008 .

[72]  V. Braitenberg Cell Assemblies in the Cerebral Cortex , 1978 .

[73]  J. Csicsvari,et al.  Organization of cell assemblies in the hippocampus , 2003, Nature.

[74]  Lawrence H. Snyder,et al.  Executive control and task-switching in monkeys , 2003, Neuropsychologia.