Mutual Information, Fisher Information, and Population Coding

In the context of parameter estimation and model selection, it is only quite recently that a direct link between the Fisher information and information-theoretic quantities has been exhibited. We give an interpretation of this link within the standard framework of information theory. We show that in the context of population coding, the mutual information between the activity of a large array of neurons and a stimulus to which the neurons are tuned is naturally related to the Fisher information. In the light of this result, we consider the optimization of the tuning curves parameters in the case of neurons responding to a stimulus represented by an angular variable.

[1]  Claude E. Shannon,et al.  The mathematical theory of communication , 1950 .

[2]  L. Goddard.  Information Theory , 1962, Nature.

[3]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[4]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[5]  R. Stein,et al.  The information capacity of nerve cells using a frequency code. , 1967, Biophysical journal.

[6]  R. Rao,et al.  Normal Approximation and Asymptotic Expansions , 1976 .

[7]  S. Laughlin.  A Simple Coding Procedure Enhances a Neuron's Information Capacity , 1981, Zeitschrift fur Naturforschung. Section C, Biosciences.

[8]  A P Georgopoulos,et al.  On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[9]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[10]  Richard E. Blahut,et al.  Principles and practice of information theory , 1987 .

[11]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[12]  G. Parisi,et al.  Statistical Field Theory , 1988 .

[13]  H. B. Barlow,et al.  Finding Minimum Entropy Codes , 1989, Neural Computation.

[14]  William Bialek,et al.  Reading a Neural Code , 1991, NIPS.

[15]  R. Muller,et al.  Head-direction cells recorded from the postsubiculum in freely moving rats. II. Effects of environmental manipulations , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  Andrew R. Barron,et al.  Information-theoretic asymptotics of Bayes methods , 1990, IEEE Trans. Inf. Theory.

[17]  R U Muller,et al.  Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[18]  Alexandre Pouget,et al.  Connectionist Models of Orientation Identification , 1991 .

[19]  H Sompolinsky,et al.  Simple models for reading neuronal population codes. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[20]  William R. Softky,et al.  The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[21]  J. Nadal,et al.  Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer Network 5 , 1994 .

[22]  Daniel L. Ruderman,et al.  Designing receptive fields for highest fidelity , 1994 .

[23]  J. Taube.  Head direction cells recorded in the anterior thalamic nuclei of freely moving rats , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  Herman P. Snippe,et al.  Parameter Extraction from Population Codes: A Critical Assessment , 1996, Neural Computation.

[25]  Alexandre Pouget,et al.  Statistically Efficient Estimations Using Cortical Lateral Connections , 1996, NIPS.

[26]  Jorma Rissanen,et al.  Fisher information and stochastic complexity , 1996, IEEE Trans. Inf. Theory.

[27]  B L McNaughton,et al.  Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells. , 1998, Journal of neurophysiology.

[28]  Stefano Panzeri,et al.  On Decoding the Responses of a Population of Neurons from Short Time Windows , 1999, Neural Computation.

[29]  J. Nadal,et al.  Unsupervised and supervised learning: Mutual information between parameters and observations , 1999 .