Dreaming of mathematical neuroscience for half a century

Theoreticians have been enchanted by the secrets of the brain for many years: how and why does it work so well? There has been a long history of searching for its mechanisms. Theoretical or even mathematical scientists have proposed various models of neural networks which has led to the birth of a new field of research. We can think of the 'pre-historic' period of Rashevski and Wiener, and then the period of perceptrons which is the beginning of learning machines, neurodynamics approaches, and further connectionist approaches. Now is currently the period of computational neuroscience. I have been working in this field for nearly half a century, and have experienced its repeated rise and fall. Now having reached very old age, I would like to state my own endeavors on establishing mathematical neuroscience for half a century, from a personal, even biased, point of view. It would be my pleasure if my experiences could encourage young researchers to participate in mathematical neuroscience.

[1]  S. Amari,et al.  Characteristics of Random Nets of Analog Neuron-Like Elements , 1972, IEEE Trans. Syst. Man Cybern..

[2]  S. Amari,et al.  Singularities Affect Dynamics of Learning in Neuromanifolds , 2006, Neural Computation.

[3]  S.-I. Amari,et al.  Neural theory of association and concept-formation , 1977, Biological Cybernetics.

[4]  Shun-ichi Amari,et al.  Measure of Correlation Orthogonal to Change in Firing Rate , 2009, Neural Computation.

[5]  Shun-ichi Amari,et al.  Dynamics of learning near singularities in radial basis function networks , 2008, Neural Networks.

[6]  Shun-ichi Amari,et al.  Statistical neurodynamics of associative memory , 1988, Neural Networks.

[7]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[8]  Shun-ichi Amari,et al.  Four Types of Learning Curves , 1992, Neural Computation.

[9]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[10]  Anthony C. C. Coolen,et al.  Phase Diagram and Storage Capacity of Sequence-Storing Neural Networks , 1998, NIPS.

[11]  Shun-ichi Amari,et al.  A Theory of Adaptive Pattern Classifiers , 1967, IEEE Trans. Electron. Comput..

[12]  Shun-ichi Amari,et al.  A method of statistical neurodynamics , 1974, Kybernetik.

[13]  Shun-ichi Amari,et al.  Stochastic Reasoning, Free Energy, and Information Geometry , 2004, Neural Computation.

[14]  S. Amari,et al.  A Mathematical Foundation for Statistical Neurodynamics , 1977 .

[15]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[16]  Marc Timme,et al.  Synaptic Scaling in Combination with Many Generic Plasticity Mechanisms Stabilizes Circuit Connectivity , 2011, Front. Comput. Neurosci..

[17]  Shun-ichi Amari,et al.  Characteristics of randomly connected threshold-element networks and network systems , 1971 .

[18]  Masato Okada,et al.  A hierarchy of macrodynamical equations for associative memory , 1995, Neural Networks.

[19]  J. Cowan,et al.  A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue , 1973, Kybernetik.

[20]  Shun-ichi Amari,et al.  Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements , 1972, IEEE Transactions on Computers.

[21]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[23]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[24]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[25]  Shun-ichi Amari,et al.  Information geometry on hierarchy of probability distributions , 2001, IEEE Trans. Inf. Theory.

[26]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[27]  Masato Okada,et al.  Estimating Spiking Irregularities Under Changing Environments , 2006, Neural Computation.

[28]  Yutaka Sakai,et al.  Synchronous Firing and Higher-Order Interactions in Neuron Pool , 2003, Neural Computation.

[29]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[30]  C. Malsburg,et al.  How patterned neural connections can be set up by self-organization , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[31]  S. Amari,et al.  Formation of topographic maps and columnar microstructures in nerve fields , 1979, Biological Cybernetics.

[32]  Shun-ichi Amari,et al.  A universal theorem on learning curves , 1993, Neural Networks.

[33]  S. Kauffman.  Metabolic stability and epigenesis in randomly constructed genetic nets. , 1969, Journal of theoretical biology.

[34]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[35]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[36]  S. Amari,et al.  State concentration exponent as a measure of quickness in Kauffman-type networks. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[37]  Shun-ichi Amari,et al.  State concentration measure of quickness in Kauffman-type networks , 2012 .

[38]  C. von der Malsburg.  Self-organization of orientation sensitive cells in the striate cortex. , 1973, Kybernetik.

[39]  Shun-ichi Amari,et al.  Information-Geometric Measure for Neural Spikes , 2002, Neural Computation.

[40]  Shun-ichi Amari,et al.  Blind source separation-semiparametric statistical approach , 1997, IEEE Trans. Signal Process..

[41]  Takafumi Kanamori,et al.  Information Geometry of U-Boost and Bregman Divergence , 2004, Neural Computation.

[42]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[43]  Shun-ichi Amari,et al.  Mathematical foundations of neurocomputing , 1990, Proc. IEEE.

[44]  Sherrington,et al.  Dynamics of fully connected attractor neural networks near saturation. , 1993, Physical review letters.

[45]  Heskes,et al.  Learning processes in neural networks. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[46]  Masato Okada,et al.  Information Loss Associated with Imperfect Observation and Mismatched Decoding , 2011, Front. Comput. Neurosci..

[47]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[48]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[49]  Shun-ichi Amari,et al.  Information geometry of Boltzmann machines , 1992, IEEE Trans. Neural Networks.