BIOLOGICAL CONSTRAINTS ON CONNECTIONIST MODELLING

Many researchers interested in connectionist models accept that such models are "neurally inspired" but do not worry too much about whether their models are biologically realistic. While such a position may be perfectly justifiable, the present paper attempts to illustrate how biological information can be used to constrain connectionist models. Two particular areas are discussed. The first section deals with visual information processing in the primate and human visual system. It is argued that speed with which visual information is processed imposes major constraints on the architecture and operation of the visual system. In particular, it seems that a great deal of processing must depend on a single bottum-up pass. The second section deals with biological aspects of learning algorithms. It is argued that although there is good evidence for certain coactivation related synaptic modification schemes, other learning mechanisms, including back-propagation, are not currently supported by experimental data.

[1]  J. M. Cattell.  THE TIME TAKEN UP BY CEREBRAL OPERATIONS , 1886 .

[2]  J. M. Cattell.  THE TIME IT TAKES TO SEE AND NAME OBJECTS , 1886 .

[3]  S. Kaplan.  The Physiology of Thought , 1950 .

[4]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[5]  A Wingfield,et al.  Response Latencies in Naming Objects , 1965, The Quarterly journal of experimental psychology.

[6]  R. C. Oldfield.  Things, Words and the Brain* , 1966, The Quarterly journal of experimental psychology.

[7]  M. Potter.  Meaning in visual search. , 1975, Science.

[8]  M. Potter,et al.  Time to understand pictures and words , 1975, Nature.

[9]  tephen E. Palmer.  The effects of contextual scenes on the identification of objects , 1975, Memory & cognition.

[10]  E. T. Rolls,et al.  Hypothalamic neuronal responses associated with the sight of food , 1976, Brain Research.

[11]  M. Potter.  Short-term conceptual memory for pictures. , 1976, Journal of experimental psychology. Human learning and memory.

[12]  T. Sejnowski,et al.  Storing covariance with nonlinearly interacting neurons , 1977, Journal of mathematical biology.

[13]  Teuvo Kohonen,et al.  Associative memory. A system-theoretical approach , 1977 .

[14]  E. Rolls,et al.  The latency of activation of neurones in the lateral hypothalamus and substantia innominata during feeding in the monkey , 1979, Brain Research.

[15]  H. Intraub,et al.  Presentation rate and the representation of briefly glimpsed pictures in memory. , 1980, Journal of experimental psychology. Human learning and memory.

[16]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[17]  H. Intraub.  Rapid conceptual identification of sequentially presented pictures. , 1981 .

[18]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[19]  E T Rolls,et al.  Neuronal responses related to visual recognition. , 1982, Brain : a journal of neurology.

[20]  Stephen Grossberg,et al.  Studies of mind and brain , 1982 .

[21]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Kunihiko Fukushima,et al.  Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position , 1982, Pattern Recognit..

[23]  D. C. Essen,et al.  Hierarchical organization and functional streams in the visual cortex , 1983, Trends in Neurosciences.

[24]  Leslie G. Ungerleider,et al.  Object vision and spatial vision: two cortical pathways , 1983, Trends in Neurosciences.

[25]  J. Kulikowski,et al.  Primate cortical area V4 important for colour constancy but not wavelength discrimination , 1985, Nature.

[26]  Jerome A. Feldman,et al.  Connectionist models and parallelism in high level vision , 1985, Comput. Vis. Graph. Image Process..

[27]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[28]  J. Feldman.  Four frames suffice: A provisional model of vision and space , 1985, Behavioral and Brain Sciences.

[29]  R Linsker,et al.  From basic network principles to neural architecture: emergence of orientation columns. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[31]  C Koch,et al.  Analog "neuronal" networks in early vision. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[32]  R Linsker,et al.  From basic network principles to neural architecture: emergence of orientation-selective cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[33]  S. Kelso,et al.  Hebbian synapses in hippocampus. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[34]  B. Strehler,et al.  Evidence on precise time-coded symbols and memory of patterns in monkey cortical neuronal spike trains. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[35]  B. Baldwin,et al.  The activity of neurones in the lateral hypothalamus and zona incerta of the sheep responding to the sight or approach of food is modified by learning and satiety and reflects food preference , 1986, Brain Research.

[36]  R Linsker,et al.  From basic network principles to neural architecture: emergence of spatial-opponent cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[37]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[38]  J. Maunsell,et al.  Visual processing in monkey extrastriate cortex. , 1987, Annual review of neuroscience.

[39]  K. Rayner,et al.  Effects of Foveal Priming and Extrafoveal Preview on Object Identification , 1987 .

[40]  B. A. Baldwin,et al.  Cells in temporal cortex of conscious sheep can respond preferentially to the sight of faces. , 1987, Science.

[41]  L. Optican,et al.  Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. III. Information theoretic analysis. , 1987, Journal of neurophysiology.

[42]  H. Spitzer,et al.  Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. I. Response characteristics. , 1987, Journal of neurophysiology.

[43]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[44]  C. Koch,et al.  The analysis of visual motion: from computational theory to neuronal mechanisms. , 1986, Annual review of neuroscience.

[45]  M. Posner,et al.  Selective attention and cognitive control , 1987, Trends in Neurosciences.

[46]  B J Richmond,et al.  Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. II. Quantification of response waveform. , 1987, Journal of neurophysiology.

[47]  Elie Bienenstock,et al.  A neural network for invariant pattern recognition. , 1987 .

[48]  Elie Bienenstock,et al.  A neural network for the retrieval of superimposed connection patterns , 1987 .

[49]  Glyn W. Humphreys,et al.  Normal and pathological processes in visual object constancy. , 1987 .

[50]  T. Poggio,et al.  Synapses that compute motion. , 1987, Scientific American.

[51]  A. Cowey,et al.  On the role of cortical area V4 in the discrimination of hue and pattern in macaque monkeys , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[52]  W. Singer,et al.  Long-term potentiation and NMDA receptors in rat visual cortex , 1987, Nature.

[53]  E I Knudsen,et al.  Computational maps in the brain. , 1987, Annual review of neuroscience.

[54]  M J Farah,et al.  Electrophysiological evidence for a shared representational medium for visual images and visual percepts. , 1988, Journal of experimental psychology. General.

[55]  Walter Heiligenberg,et al.  Temporal hyperacuity in single neurons of electric fish , 1988, Nature.

[56]  A. Ganong,et al.  Excitatory amino acid neurotransmission: NMDA receptors and Hebb-type synaptic plasticity. , 1988, Annual review of neuroscience.

[57]  D. C. Van Essen,et al.  Concurrent processing streams in monkey visual cortex , 1988, Trends in Neurosciences.

[58]  Y. Frégnac,et al.  A cellular analogue of visual cortical plasticity , 1988, Nature.

[59]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[60]  Terrence J. Sejnowski,et al.  Neural network learning algorithms , 1988 .

[61]  La vision naturelle. Le traitement neuronal de l'information visuelle , 1988 .

[62]  M. Stryker,et al.  Neural plasticity without postsynaptic action potentials: less-active inputs become dominant when kitten visual cortical cells are pharmacologically inhibited. , 1988, Proceedings of the National Academy of Sciences of the United States of America.

[63]  Tony Owen.  Brains, Machines, and Mathematics by Michael A. Arbib Springer-Verlag, New York, 1987 (Price DM55, circa £18.50) , 1988, Robotica.

[64]  R. Wurtz,et al.  Probing visual cortical function with discrete chemical lesions , 1988, Trends in Neurosciences.

[65]  S. Shipp,et al.  The functional logic of cortical connections , 1988, Nature.

[66]  Richard A. Andersen,et al.  A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons , 1988, Nature.

[67]  Marvin Minsky,et al.  Perceptrons: expanded edition , 1988 .

[68]  D. Schacter.  Progress in cognitive neuropsychology. Review of G. Humphreys & M.J. Riddoch (Eds.), Visual object processing: A cognitive neuropsychological approach , 1989 .

[69]  Geoffrey E. Hinton,et al.  Parallel Models of Associative Memory , 1989 .

[70]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.

[71]  Eric L. Schwartz,et al.  Computational Neuroscience , 1993, Neuromethods.

[72]  Kunihiko Fukushima,et al.  Cognitron: A self-organizing multilayered neural network , 1975, Biological Cybernetics.

[73]  S.-I. Amari,et al.  Neural theory of association and concept-formation , 1977, Biological Cybernetics.

[74]  G. Henry,et al.  Physiological studies on the feedback connection to the striate cortex from cortical areas 18 and 19 of the cat , 1988, Experimental Brain Research.

[75]  D. Perrett,et al.  Visual neurones responsive to faces in the monkey temporal cortex , 2004, Experimental Brain Research.

[76]  S. Thorpe,et al.  The orbitofrontal cortex: Neuronal activity in the behaving monkey , 2004, Experimental Brain Research.

[77]  B. Strehler,et al.  Differences between monkey visual cortex cells in triplet and ghost doublet informational symbols relationships , 1988, Biological Cybernetics.

[78]  H. Barlow,et al.  Single Units and Sensation: A Neuron Doctrine for Perceptual Psychology? , 1972, Perception.