Neural coding of categories: information efficiency and optimal population codes

This paper deals with the analytical study of coding a discrete set of categories by a large assembly of neurons. We consider population coding schemes, which can also be seen as instances of exemplar models proposed in the literature to account for phenomena in the psychophysics of categorization. We quantify the coding efficiency by the mutual information between the set of categories and the neural code, and we characterize the properties of the most efficient codes, considering different regimes corresponding essentially to different signal-to-noise ratio. One main outcome is to find that, in a high signal-to-noise ratio limit, the Fisher information at the population level should be the greatest between categories, which is achieved by having many cells with the stimulus-discriminating parts (steepest slope) of their tuning curves placed in the transition regions between categories in stimulus space. We show that these properties are in good agreement with both psychophysical data and with the neurophysiology of the inferotemporal cortex in the monkey, a cortex area known to be specifically involved in classification tasks.

[1]  J. Hillenbrand,et al.  Acoustic characteristics of American English vowels. , 1994, The Journal of the Acoustical Society of America.

[2]  Shaowen Bao,et al.  Early experience impairs perceptual discrimination , 2007, Nature Neuroscience.

[3]  Yuhong Yang Elements of Information Theory (2nd ed.). Thomas M. Cover and Joy A. Thomas , 2008 .

[4]  Jean-Pierre Nadal,et al.  Formal Neural Networks: From Supervised to Unsupervised Learning , 1994 .

[5]  T. Poggio A theory of how the brain might work. , 1990, Cold Spring Harbor symposia on quantitative biology.

[6]  Nicolas Brunel,et al.  Mutual Information, Fisher Information, and Population Coding , 1998, Neural Computation.

[7]  N. Sigala Visual categorization and the inferior temporal cortex , 2004, Behavioural Brain Research.

[8]  Martha Wilson,et al.  Inferotemporal cortex and categorical perception of visual stimuli by monkeys , 1981, Neuropsychologia.

[9]  I. Gauthier,et al.  Visual object understanding , 2004, Nature Reviews Neuroscience.

[10]  M. V. Van Hulle,et al.  Encoding of Categories by Noncategory-Specific Neurons in the Inferior Temporal Cortex , 2001, Journal of Cognitive Neuroscience.

[12]  R. Vogels Categorization of complex visual images by rhesus monkeys. Part 1: behavioural study , 1999, The European journal of neuroscience.

[13]  William R. Softky,et al.  The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  R. Vogels Categorization of complex visual images by rhesus monkeys. Part 2: single‐cell study , 1999, The European journal of neuroscience.

[15]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[16]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[17]  Keiji Tanaka,et al.  Inferotemporal cortex and object vision. , 1996, Annual review of neuroscience.

[18]  M. Riesenhuber,et al.  Categorization Training Results in Shape- and Category-Selective Human Neural Plasticity , 2007, Neuron.

[19]  Kenji Kawano,et al.  Global and fine information coded by single neurons in the temporal visual cortex , 1999, Nature.

[20]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[21]  J.C. Principe,et al.  A methodology for information theoretic feature extraction , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[22]  G. Orban,et al.  How well do response changes of striate neurons signal differences in orientation: a study in the discriminating monkey , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[23]  M. Cohen,et al.  Effects of categorization and discrimination training on auditory perceptual space. , 1999, The Journal of the Acoustical Society of America.

[24]  Peter E. Latham,et al.  Statistically Efficient Estimation Using Population Coding , 1998, Neural Computation.

[25]  A. Pouget,et al.  Tuning curve sharpening for orientation selectivity: coding efficiency and the impact of correlations , 2004, Nature Neuroscience.

[26]  R U Muller,et al.  Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  D. Butts,et al.  Tuning Curves, Neuronal Variability, and Sensory Coding , 2006, PLoS biology.

[28]  Richard E. Blahut,et al.  Principles and practice of information theory , 1987 .

[29]  J. Movshon,et al.  The statistical reliability of signals in single neurons in cat and monkey visual cortex , 1983, Vision Research.

[30]  Eric Goles,et al.  Cellular automata, dynamical systems, and neural networks , 1994 .

[31]  H. Sompolinsky,et al.  Population coding in neuronal systems with correlated noise. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[32]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[33]  M. Riesenhuber,et al.  Categorization in IT and PFC: Model and Experiments , 2002 .

[34]  Tomaso Poggio,et al.  Models of object recognition , 2000, Nature Neuroscience.

[35]  S. Yau Mathematics and its applications , 2002 .

[36]  Tomaso Poggio,et al.  Fast Readout of Object Identity from Macaque Inferior Temporal Cortex , 2005, Science.

[37]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[38]  Robert L. Goldstone Influences of categorization on perceptual discrimination. , 1994, Journal of experimental psychology. General.

[39]  B. C. Griffith,et al.  The discrimination of speech sounds within and across phoneme boundaries. , 1957, Journal of experimental psychology.

[40]  A. Pouget,et al.  Neural correlations, population coding and computation , 2006, Nature Reviews Neuroscience.

[41]  Bruno A. Olshausen,et al.  Book Review , 2003, Journal of Cognitive Neuroscience.

[42]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[43]  Haim Sompolinsky,et al.  The Effect of Correlations on the Fisher Information of Population Codes , 1998, NIPS.

[44]  R. Shapley,et al.  Information Tuning of Populations of Neurons in Primary Visual Cortex , 2004, The Journal of Neuroscience.

[45]  P. Kuhl,et al.  Enhanced discriminability at the phonetic boundaries for the place feature in macaques. , 1983, The Journal of the Acoustical Society of America.

[46]  Brian J. Spiering,et al.  The neurobiology of category learning. , 2004, Behavioral and cognitive neuroscience reviews.

[47]  Keiji Tanaka,et al.  Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. , 2007, Journal of neurophysiology.

[48]  N. Sigala,et al.  Visual categorization shapes feature selectivity in the primate temporal cortex , 2002, Nature.

[49]  Peter Dayan,et al.  The Effect of Correlated Variability on the Accuracy of a Population Code , 1999, Neural Computation.

[50]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[51]  David J. Freedman,et al.  Categorical representation of visual stimuli in the primate prefrontal cortex. , 2001, Science.

[52]  H. Cohen,et al.  Handbook of Categorization , 2005 .

[53]  H. Komatsu,et al.  Effects of task demands on the responses of color-selective neurons in the inferior temporal cortex , 2007, Nature Neuroscience.

[54]  Keiji Tanaka,et al.  Effects of shape-discrimination training on the selectivity of inferotemporal cells in adult monkeys. , 1998, Journal of neurophysiology.

[55]  R. Nosofsky Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[56]  M. Young,et al.  Sparse population coding of faces in the inferotemporal cortex. , 1992, Science.

[57]  William M. Campbell,et al.  Mutual Information in Learning Feature Transformations , 2000, ICML.

[58]  Kenneth R. Livingston,et al.  Categorical Perception Effects Induced by Category Learning , 1998 .

[59]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[60]  R. Stein,et al.  The information capacity of nerve cells using a frequency code. , 1967, Biophysical journal.

[61]  David J. Freedman,et al.  A Comparison of Primate Prefrontal and Inferior Temporal Cortices during Visual Categorization , 2003, The Journal of Neuroscience.

[62]  Stevan Harnad,et al.  Cognition is categorization , 2005 .

[63]  R. Nosofsky Attention, similarity, and the identification-categorization relationship. , 1986 .

[64]  Douglas L. Hintzman,et al.  "Schema Abstraction" in a Multiple-Trace Memory Model , 1986 .

[65]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[66]  G. Humphreys,et al.  Hierarchies, similarity, and interactivity in object recognition: “Category-specific” neuropsychological deficits , 2001, Behavioral and Brain Sciences.

[67]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[68]  Haim Sompolinsky,et al.  Erratum: Population coding in neuronal systems with correlated noise [Phys. Rev. E 64, 051904 (2001)] , 2002 .

[69]  P. Kuhl Human adults and human infants show a “perceptual magnet effect” for the prototypes of speech categories, monkeys do not , 1991, Perception & psychophysics.

[70]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[71]  H Sompolinsky,et al.  Simple models for reading neuronal population codes. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[72]  C. Gilbert,et al.  Perceptual learning and top-down influences in primary visual cortex , 2004, Nature Neuroscience.

[73]  S. Harnad Categorical Perception: The Groundwork of Cognition , 1990 .

[74]  M. Paradiso,et al.  A theory for the use of visual orientation information which exploits the columnar structure of striate cortex , 2004, Biological Cybernetics.

[75]  R. Vogels,et al.  Inferotemporal neurons represent low-dimensional configurations of parameterized shapes , 2001, Nature Neuroscience.

[76]  H. Sompolinsky,et al.  Mutual information of population codes and distance measures in probability space. , 2001, Physical review letters.

[77]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .