Processing of Time Series by Neural Circuits with Biologically Realistic Synaptic Dynamics

Experimental data show that biological synapses behave quite differently from the symbolic synapses in common artificial neural network models. Biological synapses are dynamic, i.e., their "weight" changes on a short time scale by several hundred percent in dependence of the past input to the synapse. In this article we explore the consequences that these synaptic dynamics entail for the computational power of feedforward neural networks. We show that gradient descent suffices to approximate a given (quadratic) filter by a rather small neural system with dynamic synapses. We also compare our network model to artificial neural networks designed for time series processing. Our numerical results are complemented by theoretical analysis which show that even with just a single hidden layer such networks can approximate a surprisingly large large class of nonlinear filters: all filters that can be characterized by Volterra series. This result is robust with regard to various changes in the model for synaptic dynamics.

[1]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[2]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[3]  Henry Markram,et al.  Neural Networks with Dynamic Synapses , 1998, Neural Computation.

[4]  M. Merzenich,et al.  Optimizing sound features for cortical neurons. , 1998, Science.

[5]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[6]  H. Markram,et al.  Redistribution of synaptic efficacy between neocortical pyramidal neurons , 1996, Nature.

[7]  R. Nicoll,et al.  Hippocampal Long-Term Potentiation Preserves the Fidelity of Postsynaptic Responses to Presynaptic Bursts , 1999, The Journal of Neuroscience.

[8]  Ah Chung Tsoi,et al.  A Simplified Gradient Algorithm for IIR Synapse Multilayer Perceptrons , 1993, Neural Computation.

[9]  M. Schetzen.  The Volterra and Wiener Theories of Nonlinear Systems , 1980 .

[10]  W A Little,et al.  A statistical theory of short and long term memory. , 1975, Behavioral biology.

[11]  Eduardo D. Sontag,et al.  Neural Systems as Nonlinear Filters , 2000, Neural Computation.

[12]  L. Abbott,et al.  A Quantitative Description of Short-Term Plasticity at Excitatory Synapses in Layer 2/3 of Rat Primary Visual Cortex , 1997, The Journal of Neuroscience.

[13]  Anthony M. Zador,et al.  The basic unit of computation , 2000, Nature Neuroscience.

[14]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.