Difficulty of Singularity in Population Coding

Fisher information has been used to analyze the accuracy of neural population coding. This works well when the Fisher information does not degenerate, but when two stimuli are presented to a population of neurons, a singular structure emerges by their mutual interactions. In this case, the Fisher information matrix degenerates, and the regularity condition ensuring the Cramr-Rao paradigm of statistics is violated. An animal shows pathological behavior in such a situation. We present a novel method of statistical analysis to understand information in population coding in which algebraic singularity plays a major role. The method elucidates the nature of the pathological case by calculating the Fisher information. We then suggest that synchronous firing can resolve singularity and show a method of analyzing the binding problem in terms of the Fisher information. Our method integrates a variety of disciplines in population coding, such as nonregular statistics, Bayesian statistics, singularity in algebraic geometry, and synchronous firing, under the theme of Fisher information.

[1]  Alexandre Pouget,et al.  Probabilistic Interpretation of Population Codes , 1996, Neural Computation.

[2]  Si Wu,et al.  Information processing in a neuron ensemble with the multiplicative correlation structure , 2004, Neural Networks.

[3]  Y. Shao,et al.  Asymptotics for likelihood ratio tests under loss of identifiability , 2003 .

[4]  Stefan Treue,et al.  Seeing multiple directions of motion—physiology and psychophysics , 2000, Nature Neuroscience.

[5]  Shun-ichi Amari,et al.  On Density Estimation under Relative Entropy Loss Criterion , 2002, Probl. Inf. Transm..

[6]  Shun-ichi Amari,et al.  Singularities Affect Dynamics of Learning in Neuromanifolds , 2006, Neural Comput..

[7]  D. Ingle.  Visual Releasers of Prey-Catching Behavior in Frogs and Toads , 1968 .

[8]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[9]  Terrence J. Sejnowski,et al.  Neuronal Tuning: To Sharpen or Broaden? , 1999, Neural Computation.

[10]  Richard S. Zemel,et al.  A Probabilistic Network Model of Population Responses , 2007 .

[11]  S. Amari,et al.  Differential and Algebraic Geometry of Multilayer Perceptrons , 2001 .

[12]  Christian W. Eurich,et al.  Multidimensional Encoding Strategy of Spiking Neurons , 2000, Neural Computation.

[13]  Yutaka Sakai,et al.  Synchronous Firing and Higher-Order Interactions in Neuron Pool , 2003, Neural Computation.

[14]  K. Fukumizu.  Likelihood ratio of unidentifiable models and multilayer neural networks , 2003 .

[15]  H. Hotelling.  Tubes and Spheres in n-Spaces, and a Class of Statistical Problems , 1939 .

[16]  S. Amari,et al.  Competition and Cooperation in Neural Nets , 1982 .

[17]  Si Wu,et al.  Asymptotic behaviors of population codes , 2002, Neurocomputing.

[18]  Christoph von der Malsburg,et al.  The What and Why of Binding: Review The Modeler's Perspective , 1999 .

[19]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[20]  Si Wu,et al.  Population Coding and Decoding in a Neural Field: A Computational Study , 2002, Neural Computation.

[21]  Andrew R. Barron,et al.  Information-theoretic asymptotics of Bayes methods , 1990, IEEE Trans. Inf. Theory.

[22]  F. Komaki.  On asymptotic properties of predictive distributions , 1996 .

[23]  Sumio Watanabe.  Algebraic geometrical methods for hierarchical learning machines , 2001, Neural Networks.

[24]  Shun-ichi Amari,et al.  On Some Singularities in Parameter Estimation Problems , 2003, Probl. Inf. Transm..

[25]  Sumio Watanabe.  Algebraic Analysis for Non-identifiable Learning Machines , 2000 .

[26]  Si Wu,et al.  Attention Modulation of Neural Tuning Through Peak and Base Rate , 2001, Neural Computation.

[27]  Jianfeng Feng,et al.  Decoding spike train ensembles: tracking a moving stimulus , 2006, Biological Cybernetics.

[28]  Shun-ichi Amari,et al.  Attention modulation of neural tuning through peak and base rate in correlated firing , 2002, Neural Networks.

[29]  Sumio Watanabe.  Algebraic Information Geometry for Learning Machines with Singularities , 2000, NIPS.

[30]  Si Wu,et al.  Sequential Bayesian Decoding with a Population of Neurons , 2003, Neural Computation.

[31]  H Sompolinsky,et al.  Simple models for reading neuronal population codes. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[32]  Wolf Singer,et al.  Neuronal Synchrony: A Versatile Code for the Definition of Relations? , 1999, Neuron.

[33]  Werner Reichardt,et al.  A theory of the pattern induced flight orientation of the fly Musca domestica II , 1975, Biological Cybernetics.

[34]  H. Weyl.  On the Volume of Tubes , 1939 .

[35]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[36]  Sumio Watanabe,et al.  Algebraic Analysis for Nonidentifiable Learning Machines , 2001, Neural Computation.

[37]  Haim Sompolinsky,et al.  The Effect of Correlations on the Fisher Information of Population Codes , 1998, NIPS.

[38]  E. Gassiat,et al.  Testing in locally conic models, and application to mixture models , 1997 .

[39]  Kenji Fukumizu,et al.  Local minima and plateaus in hierarchical structures of multilayer perceptrons , 2000, Neural Networks.

[40]  S. Amari.  Dynamics of pattern formation in lateral-inhibition type neural fields , 1977, Biological Cybernetics.

[41]  Peter E. Latham,et al.  Statistically Efficient Estimation Using Population Coding , 1998, Neural Computation.

[42]  Peter Dayan,et al.  Doubly Distributional Population Codes: Simultaneous Representation of Uncertainty and Multiplicity , 2003, Neural Computation.

[43]  Si Wu,et al.  Population Coding with Correlation and an Unfaithful Model , 2001, Neural Computation.