Learning complex cell invariance from natural videos: A plausibility proof

Abstract : One of the most striking features of the cortex is its ability to wire itself. Understanding how the visual cortex wires up through development and how visual experience refines connections into adulthood is a key question for Neuroscience. While computational models of the visual cortex are becoming increasingly detailed, the question of how such architecture could self-organize through visual experience is often overlooked. Here we focus on the class of hierarchical feed-forward models of the ventral stream of the visual cortex, which extend the classical simple-to-complex cells model by Hubel and Wiesel to extra-striate areas, and have been shown to account for a host of experimental data. Such models assume two functional classes of simple and complex cells with specific predictions about their respective wiring and resulting functionalities. In these networks, the issue of learning, especially for complex cells, is perhaps the least well understood. In fact, in most of these models, the connectivity between simple and complex cells is not learned but rather hard-wired. Several algorithms have been proposed for learning invariances at the complex cell level based on a trace rule to exploit the temporal continuity of sequences of natural images.

[1]  James V. Stone,et al.  A learning rule for extracting spatio-temporal invariances , 1995 .

[2]  Konrad Paul Kording,et al.  How are complex cell properties adapted to the statistics of natural stimuli? , 2004, Journal of neurophysiology.

[3]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[4]  D. Ruderman,et al.  Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[5]  J. K. Hietanen,et al.  The effects of lighting conditions on responses of cells selective for face views in the macaque temporal cortex , 2004, Experimental Brain Research.

[6]  Rudy Guyonneau Codage par latence et STDP : des stratégies temporelles pour expliquer le traitement visuel rapide , 2006 .

[7]  Martin Rehn,et al.  A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields , 2007, Journal of Computational Neuroscience.

[8]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[9]  Edmund T. Rolls,et al.  A Model of Invariant Object Recognition in the Visual System: Learning Rules, Activation Functions, Lateral Inhibition, and Information-Based Performance Measures , 2000, Neural Computation.

[10]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[11]  I. Ohzawa,et al.  Receptive field structure in the visual cortex: does selective stimulation induce plasticity? , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Werner Reichardt,et al.  Figure-ground discrimination by relative movement in the visual system of the fly , 2004, Biological Cybernetics.

[13]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[14]  Arnaud Delorme,et al.  Networks of integrate-and-fire neurons using Rank Order Coding B: Spike timing dependent plasticity and emergence of orientation selectivity , 2001, Neurocomputing.

[15]  S. Grossberg Contour Enhancement , Short Term Memory , and Constancies in Reverberating Neural Networks , 1973 .

[16]  Lisa R. Betts,et al.  Distributed Neural Plasticity for Shape Learning in the Human Visual Cortex , 2005, PLoS biology.

[17]  C. Malsburg Self-organization of orientation sensitive cells in the striate cortex , 2004, Kybernetik.

[18]  S. Thorpe,et al.  Seeking Categories in the Brain , 2001, Science.

[19]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.

[20]  D Purves,et al.  The distribution of oriented contours in the real world. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Edmund T. Rolls,et al.  Invariant visual object recognition: A model, with lighting invariance , 2006, Journal of Physiology-Paris.

[22]  D. Hubel,et al.  Receptive fields of single neurones in the cat's striate cortex , 1959, The Journal of physiology.

[23]  Y. Amit,et al.  An integrated network for invariant visual detection and recognition , 2003, Vision Research.

[24]  A. J. Mistlin,et al.  Neurones responsive to faces in the temporal cortex: studies of functional organization, sensitivity to identity and relation to perception. , 1984, Human neurobiology.

[25]  T. Sato,et al.  Interactions of visual stimuli in the receptive fields of inferior temporal neurons in awake macaques , 2004, Experimental Brain Research.

[26]  Christoph Kayser,et al.  Learning the invariance properties of complex cells from their responses to natural stimuli , 2002, The European journal of neuroscience.

[27]  S. Thorpe,et al.  Spike times make sense , 2005, Trends in Neurosciences.

[28]  Y. Miyashita Neuronal correlate of visual associative long-term memory in the primate temporal cortex , 1988, Nature.

[29]  F. Attneave Some informational aspects of visual perception. , 1954, Psychological review.

[30]  J. Maunsell,et al.  Physiological correlates of perceptual learning in monkey V1 and V2. , 2002, Journal of neurophysiology.

[31]  D. Ferster,et al.  Computational Diversity in Complex Cells of Cat Primary Visual Cortex , 2007, The Journal of Neuroscience.

[32]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[33]  T. Poggio,et al.  Cognitive neuroscience: Neural mechanisms for the recognition of biological movements , 2003, Nature Reviews Neuroscience.

[34]  E. Rolls,et al.  INVARIANT FACE AND OBJECT RECOGNITION IN THE VISUAL SYSTEM , 1997, Progress in Neurobiology.

[35]  Laurenz Wiskott,et al.  Slowness: An Objective for Spike-Timing–Dependent Plasticity? , 2007, PLoS Comput. Biol..

[36]  R. L. Valois,et al.  The orientation and direction selectivity of cells in macaque visual cortex , 1982, Vision Research.

[37]  P. Fldik,et al.  The Speed of Sight , 2001, Journal of Cognitive Neuroscience.

[38]  E. Rolls Learning mechanisms in the temporal lobe visual cortex , 1995, Behavioural Brain Research.

[39]  Mriganka Sur,et al.  Visual activity and cortical rewiring: activity-dependent plasticity of cortical networks. , 2006, Progress in brain research.

[40]  Peter Ftildidk Learning constancies for object perception , 2001 .

[41]  Heiko Wersing,et al.  Learning Optimized Features for Hierarchical Models of Invariant Object Recognition , 2003, Neural Computation.

[42]  Y. Frégnac,et al.  Activity-dependent regulation of receptive field properties of cat area 17 by supervised Hebbian learning. , 1999, Journal of neurobiology.

[43]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[44]  R Van Rullen,et al.  Face processing using one spike per neurone. , 1998, Bio Systems.

[45]  Tomaso Poggio,et al.  Fast Readout of Object Identity from Macaque Inferior Temporal Cortex , 2005, Science.

[46]  P S Goldman-Rakic,et al.  Functional synergism between putative gamma-aminobutyrate-containing neurons and pyramidal neurons in prefrontal cortex. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[47]  Martin A. Giese,et al.  Biophysiologically Plausible Implementations of the Maximum Operation , 2002, Neural Computation.

[48]  Stephen Grossberg,et al.  Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks , 1973 .

[49]  G. Orban,et al.  Practising orientation identification improves orientation coding in V1 neurons , 2001, Nature.

[50]  Geoffrey M Ghose,et al.  Learning in mammalian sensory cortex , 2004, Current Opinion in Neurobiology.

[51]  M. Behrmann,et al.  Impact of learning on representation of parts and wholes in monkey inferotemporal cortex , 2002, Nature Neuroscience.

[52]  T. Gawne,et al.  Responses of primate visual cortical V4 neurons to simultaneously presented stimuli. , 2002, Journal of neurophysiology.

[53]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[54]  S. Hochstein,et al.  View from the Top Hierarchies and Reverse Hierarchies in the Visual System , 2002, Neuron.

[55]  Guy M. Wallis,et al.  Using Spatio-temporal Correlations to Learn Invariant Object Recognition , 1996, Neural Networks.

[56]  S. Grossberg,et al.  Laminar cortical dynamics of visual form and motion interactions during coherent object motion perception. , 2007, Spatial vision.

[57]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[58]  J. DiCarlo,et al.  Learning and neural plasticity in visual object recognition , 2006, Current Opinion in Neurobiology.

[59]  David J. Freedman,et al.  A Comparison of Primate Prefrontal and Inferior Temporal Cortices during Visual Categorization , 2003, The Journal of Neuroscience.

[60]  W. Richards,et al.  Perception as Bayesian Inference , 2008 .

[61]  L. Palmer,et al.  Plasticity of neuronal response properties in adult cat striate cortex , 1998, Visual Neuroscience.

[62]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[63]  V. TORREt,et al.  A synaptic mechanism possibly underlying directional , 1978 .

[64]  Keiji Tanaka,et al.  Effects of shape-discrimination training on the selectivity of inferotemporal cells in adult monkeys. , 1998, Journal of neurophysiology.

[65]  Timothée Masquelier,et al.  Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity , 2007, PLoS Comput. Biol..

[66]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[67]  Y. Frégnac,et al.  Visual input evokes transient and strong shunting inhibition in visual cortical neurons , 1998, Nature.

[68]  Bartlett W. Mel SEEMORE: Combining Color, Shape, and Texture Histogramming in a Neurally Inspired Approach to Visual Object Recognition , 1997, Neural Computation.

[69]  M. Riesenhuber,et al.  Categorization Training Results in Shape- and Category-Selective Human Neural Plasticity , 2007, Neuron.

[70]  D. Ferster,et al.  Neural mechanisms of orientation selectivity in the visual cortex. , 2000, Annual review of neuroscience.

[71]  Tomaso Poggio,et al.  Generalization in vision and motor control , 2004, Nature.

[72]  J. Leo van Hemmen,et al.  Temporal association , 1991 .

[73]  Konrad P. Körding,et al.  The world from a cat’s perspective – statistics of natural videos , 2003, Biological Cybernetics.

[74]  Michel Vidal-Naquet,et al.  Visual features of intermediate complexity and their use in classification , 2002, Nature Neuroscience.

[75]  P. Földiák,et al.  Forming sparse representations by local anti-Hebbian learning , 1990, Biological Cybernetics.

[76]  P. Schiller,et al.  Quantitative studies of single-cell properties in monkey striate cortex. II. Orientation specificity and ocular dominance. , 1976, Journal of neurophysiology.

[77]  R. Desimone,et al.  Clustering of perirhinal neurons with similar properties following visual experience in adult monkeys , 2000, Nature Neuroscience.

[78]  D Sagi,et al.  Where practice makes perfect in texture discrimination: evidence for primary visual cortex plasticity. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[79]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[80]  Thomas Serre,et al.  Realistic Modeling of Simple and Complex Cell Tuning in the HMAX Model, and Implications for Invariant Object Recognition in Cortex , 2004 .

[81]  D I Perrett,et al.  Organization and functions of cells responsive to faces in the temporal cortex. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[82]  M. Mishkin,et al.  Learning increases stimulus salience in anterior inferior temporal cortex of the macaque. , 2001, Journal of neurophysiology.

[83]  R. Douglas,et al.  A functional microcircuit for cat visual cortex. , 1991, The Journal of physiology.

[84]  Gustavo Deco,et al.  Computational neuroscience of vision , 2002 .

[85]  D. G. Albrecht,et al.  Spatial frequency selectivity of cells in macaque visual cortex , 1982, Vision Research.

[86]  T J Sejnowski,et al.  Learning viewpoint-invariant face representations from visual experience in an attractor network. , 1998, Network.

[87]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[88]  Karl J. Friston,et al.  How the brain learns to see objects and faces in an impoverished context , 1997, Nature.

[89]  P. Goldman-Rakic,et al.  Functional synergism between putative y-aminobutyrate-containing neurons and pyramidal neurons in prefrontal cortex ( fast spike / monkey / memory / interneurons / vislon ) , 2022 .

[90]  Michael S. Lewicki,et al.  Efficient auditory coding , 2006, Nature.

[91]  Tomaso A. Poggio,et al.  Biophysical Models of Neural Computation: Max and Tuning Circuits , 2006, WImBI.

[92]  Peter Földiák,et al.  Learning Invariance from Transformation Sequences , 1991, Neural Comput..

[93]  Geoffrey E. Hinton,et al.  Self-organizing neural network that discovers surfaces in random-dot stereograms , 1992, Nature.

[94]  Peter Földiák,et al.  Learning generalisation and localisation: Competition for stimulus type and receptive field , 1996, Neurocomputing.

[95]  D. Ruderman The statistics of natural images , 1994 .

[96]  Y. Frégnac,et al.  A cellular analogue of visual cortical plasticity , 1988, Nature.

[97]  Aapo Hyvärinen,et al.  A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images , 2001, Vision Research.

[98]  M. A. Repucci,et al.  Spatial Structure and Symmetry of Simple-Cell Receptive Fields in Macaque Primary Visual Cortex , 2002 .

[99]  V. Mountcastle Modality and topographic properties of single neurons of cat's somatic sensory cortex. , 1957, Journal of neurophysiology.

[100]  Y. Dan,et al.  Stimulus Timing-Dependent Plasticity in Cortical Processing of Orientation , 2001, Neuron.

[101]  Thomas Serre,et al.  A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex , 2005 .

[102]  Allan D. Jepson,et al.  From Features to Perceptual Categories , 1992 .

[103]  E M Callaway,et al.  Visual scenes and cortical neurons: what you see is what you get. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[104]  Laurenz Wiskott,et al.  Slow feature analysis yields a rich repertoire of complex cell properties. , 2005, Journal of vision.

[105]  D. Perrett,et al.  Evidence accumulation in cell populations responsive to faces: an account of generalisation of recognition without mental transformations , 1998, Cognition.

[106]  Y. Dan,et al.  Receptive-Field Modification in Rat Visual Cortex Induced by Paired Visual Stimulation and Single-Cell Spiking , 2006, Neuron.

[107]  N. Logothetis,et al.  The Effect of Learning on the Function of Monkey Extrastriate Visual Cortex , 2004, PLoS biology.

[108]  E. Rolls,et al.  Neurodynamics of biased competition and cooperation for attention: a model with spiking neurons. , 2005, Journal of neurophysiology.

[109]  E. Miller,et al.  Effects of Visual Experience on the Representation of Objects in the Prefrontal Cortex , 2000, Neuron.

[110]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[111]  T. Poggio,et al.  The Mathematics of Learning: Dealing with Data , 2005, 2005 International Conference on Neural Networks and Brain.

[112]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[113]  G. Wallis,et al.  Learning invariant responses to the natural transformations of objects , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[114]  E. Rolls,et al.  A Neurodynamical cortical model of visual attention and invariant object recognition , 2004, Vision Research.

[115]  Tomaso Poggio,et al.  Intracellular measurements of spatial integration and the MAX operation in complex cells of the cat primary visual cortex. , 2004, Journal of neurophysiology.

[116]  David I. Perrett,et al.  Neurophysiology of shape processing , 1993, Image Vis. Comput..

[117]  E. Rolls,et al.  View-invariant representations of familiar objects by neurons in the inferior temporal visual cortex. , 1998, Cerebral cortex.

[118]  Tomaso Poggio,et al.  Role of learning in three-dimensional form perception , 1996, Nature.

[119]  D. Perrett,et al.  Recognition of objects and their component parts: responses of single units in the temporal cortex of the macaque. , 1994, Cerebral cortex.

[120]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[121]  Edmund T. Rolls,et al.  Position invariant recognition in the visual system with cluttered environments , 2000, Neural Networks.

[122]  N. Sigala,et al.  Visual categorization shapes feature selectivity in the primate temporal cortex , 2002, Nature.

[123]  Peter Földiák,et al.  Sparse coding in the primate cortex , 1998 .

[124]  U Yinon,et al.  Evidence for long‐term functional plasticity in the visual cortex of adult cats , 1982, The Journal of physiology.

[125]  M. Stryker Temporal associations , 1991, Nature.

[126]  Thomas Serre,et al.  Object recognition with features inspired by visual cortex , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[127]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[128]  T. Bonhoeffer,et al.  Pairing-Induced Changes of Orientation Maps in Cat Visual Cortex , 2001, Neuron.

[129]  P. Schiller,et al.  Quantitative studies of single-cell properties in monkey striate cortex. III. Spatial frequency. , 1976, Journal of neurophysiology.

[130]  V. Mountcastle The columnar organization of the neocortex. , 1997, Brain : a journal of neurology.

[131]  J. Maunsell,et al.  The Effect of Perceptual Learning on Neuronal Responses in Monkey Visual Area V4 , 2004, The Journal of Neuroscience.

[132]  M. Tarr,et al.  Activation of the middle fusiform 'face area' increases with expertise in recognizing novel objects , 1999, Nature Neuroscience.

[133]  C. Gilbert,et al.  Learning to see: experience and attention in primary visual cortex , 2001, Nature Neuroscience.

[134]  Niraj S. Desai,et al.  Plasticity in the intrinsic excitability of cortical pyramidal neurons , 1999, Nature Neuroscience.

[135]  Tomaso Poggio,et al.  Standard model v2.0: How visual cortex might learn a universal dictionary of shape components , 2005 .

[136]  A. Sillito Functional Considerations of the Operation of GABAergic Inhibitory Processes in the Visual Cortex , 1984 .

[137]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[138]  Simon J. Thorpe,et al.  Sparse spike coding in an asynchronous feed-forward multi-layer neural network using matching pursuit , 2004, Neurocomputing.

[139]  Michael W. Spratling Learning viewpoint invariant perceptual representations from cluttered images , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[140]  N. Kanwisher,et al.  Discrimination Training Alters Object Representations in Human Extrastriate Cortex , 2006, The Journal of Neuroscience.

[141]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[142]  P. Schiller,et al.  Quantitative studies of single-cell properties in monkey striate cortex. I. Spatiotemporal organization of receptive fields. , 1976, Journal of neurophysiology.

[143]  E. Miller,et al.  Experience-dependent sharpening of visual shape selectivity in inferior temporal cortex. , 2005, Cerebral cortex.