Understanding normal and impaired word reading: computational principles in quasi-regular domains.

A connectionist approach to processing in quasi-regular domains, as exemplified by English word reading, is developed. Networks using appropriately structured orthographic and phonological representations were trained to read both regular and exception words, and yet were also able to read pronounceable nonwords as well as skilled readers. A mathematical analysis of a simplified system clarifies the close relationship of word frequency and spelling-sound consistency in influencing naming latencies. These insights were verified in subsequent simulations, including an attractor network that accounted for latency data directly in its time to settle on a response. Further analyses of the ability of networks to reproduce data on acquired surface dyslexia support a view of the reading system that incorporates a graded division of labor between semantic and phonological processes, and contrasts in important ways with the standard dual-route account.

[1]  E. I. Salisbury The study of children , 1937 .

[2]  D. Whitteridge,et al.  Learning and Relearning , 1959, Science's STKE.

[3]  R. Venezky The Structure of English Orthography , 1965 .

[4]  J. Marshall,et al.  Syntactic and semantic errors in paralexia , 1966 .

[5]  H. Kucera,et al.  Computational analysis of present-day American English , 1967 .

[6]  N. Silberberg,et al.  Hyperlexia—Specific Word Recognition Skills in Young Children , 1967, Exceptional children.

[7]  Wayne A. Wickelgran Context-sensitive coding, associative memory, and serial order in (speech) behavior. , 1969 .

[8]  D. Marr A theory of cerebellar cortex , 1969, The Journal of physiology.

[9]  John J. L. Morton,et al.  Interaction of information in word recognition. , 1969 .

[10]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[11]  C. C. Mehegan,et al.  Hyperlexia: Exceptional reading ability in brain‐damaged children , 1972, Neurology.

[12]  J. Marshall,et al.  Patterns of paralexia: a psycholinguistic approach. , 1995, Journal of psycholinguistic research.

[13]  P. Huttenlocher,et al.  A study of children with hyperlexia , 1973, Neurology.

[14]  K. Forster,et al.  Lexical Access and Naming Time. , 1973 .

[15]  Marvin Minsky,et al.  A framework for representing knowledge , 1974 .

[16]  R. Schvaneveldt,et al.  Functions of graphemic and phonemic codes in visual word-recognition , 1974, Memory & cognition.

[17]  Marvin Minsky,et al.  A framework for representing knowledge" in the psychology of computer vision , 1975 .

[18]  J. Richardson The effects of stimulus attributes upon latency of word recognition. , 1976, British journal of psychology.

[19]  Patrick Henry Winston,et al.  The psychology of computer vision , 1976, Pattern Recognit..

[20]  R. G. Baker,et al.  The influence of English spelling patterns on pronunciation , 1976 .

[21]  J. Baron,et al.  Use of orthographic and word-specific knowledge in reading words aloud. , 1976 .

[22]  J. Frederiksen,et al.  Spelling and sound: Approaches to the internal lexicon. , 1976 .

[23]  Stephen A. Ritz,et al.  Distinctive features, categorical perception, and probability learning: some applications of a neural model , 1977 .

[24]  W. Kintsch,et al.  Memory and cognition , 1977 .

[25]  J. Marshall,et al.  Semantic Errors or Statistical Flukes? a Note on Allport's “On Knowing the Meaning of Words We are Unable to Report” , 1978 .

[26]  M. Coltheart Lexical access in simple reading tasks , 1978 .

[27]  S. Hains,et al.  Individual differences in word recognition latency , 1979 .

[28]  M. Beauvois,et al.  Phonological alexia: three dissociations. , 1979, Journal of neurology, neurosurgery, and psychiatry.

[29]  R. Glushko The Organization and Activation of Orthographic Knowledge in Reading Aloud. , 1979 .

[30]  M. Schwartz,et al.  Dissociations of language function in dementia: A case study , 1979, Brain and Language.

[31]  P. Bertelson,et al.  Does awareness of speech as a sequence of phones arise spontaneously? , 1979, Cognition.

[32]  G. Underwood Strategies of information processing , 1980 .

[33]  J. Requin Attention and Performance VII , 1980 .

[34]  Morris Halle,et al.  The rules of language , 1980, IEEE Transactions on Professional Communication.

[35]  M. Coltheart Disorders of reading and their implications for models of normal reading. , 1981 .

[36]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[37]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: Part 2. The contextual enhancement effect and some tests and extensions of the model. , 1982, Psychological review.

[38]  A. Parkin Phonological recoding in lexical decision: Effects of spelling-to-sound regularity depend on how regularity is defined , 1982, Memory & cognition.

[39]  L. Henderson Orthography and Word Recognition in Reading , 1982 .

[40]  S. Andrews Phonological recoding: Is the regularity effect consistent? , 1982 .

[41]  A. Baddeley,et al.  Developmental and acquired dyslexia: A comparison , 1982, Cognition.

[42]  Joan L. Bybee,et al.  Rules and schemas in the development and use of the English past tense , 1982 .

[43]  M. Coltheart,et al.  Surface dyslexia. , 1983, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[44]  E. Funnell Phonological processes in reading: new evidence from acquired dyslexia. , 1983, British journal of psychology.

[45]  E. Funnell Phonological processes in reading: new evidence from acquired dyslexia. , 1983 .

[46]  H. Whitaker,et al.  Dyslexia: A Global Issue , 1983 .

[47]  Mark S. Seidenberg,et al.  When does irregular spelling or pronunciation influence word recognition , 1984 .

[48]  Roger Lass,et al.  Phonology: An Introduction to Basic Concepts , 1984 .

[49]  J. Marshall Toward a Rational Taxonomy of the Developmental Dyslexias , 1984 .

[50]  Mark S. Seidenberg,et al.  Acquisition and use of spelling-sound correspondences in reading , 1984 .

[51]  Ian Maddieson,et al.  Patterns of sounds , 1986 .

[52]  Steven Pinker,et al.  Language learnability and language development , 1985 .

[53]  G. Humphreys,et al.  Are there independent lexical and nonlexical routes in word processing? An evaluation of the dual-route theory of reading , 1985, Behavioral and Brain Sciences.

[54]  D. Shankweiler,et al.  Phonology and the Problems of Learning to Read and Write , 1985 .

[55]  Gregory V. Jones Deep dyslexia, imageability, and ease of predication , 1985, Brain and Language.

[56]  Mark S. Seidenberg,et al.  Spelling-sound effects in reading: Time-course and decision criteria , 1985, Memory & cognition.

[57]  Mark S. Seidenberg The time course of phonological code activation in two writing systems , 1985, Cognition.

[58]  Joan L. Bybee Morphology: A study of the relation between meaning and form , 1985 .

[59]  James L. McClelland,et al.  The TRACE model of speech perception , 1986, Cognitive Psychology.

[60]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[61]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[62]  P. Bertelson,et al.  Literacy training and speech segmentation , 1986, Cognition.

[63]  C. Clifton,et al.  The independence of syntactic processing , 1986 .

[64]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[65]  G. O. Stone,et al.  An analysis of the delta rule and the learning of statistical associations , 1986 .

[66]  James L. McClelland,et al.  Psychological and biological models , 1986 .

[67]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[68]  Roger K. Moore Computer Speech and Language , 1986 .

[69]  James L. McClelland,et al.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition : Psychological and Biological Models , 1986 .

[70]  E. Warrington,et al.  Phonological Reading: Phenomena and Paradoxes , 1986, Cortex.

[71]  M. Harris Language processing in children and adults , 1986 .

[72]  D. Neary,et al.  Deep Dyslexia , 1986 .

[73]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[74]  G. V. van Orden A ROWS is a ROSE: spelling, sound, and reading. , 1987, Memory & cognition.

[75]  D. Besner,et al.  Reading pseudohomophones: Implications for models of pronunciation assembly and the locus of word-frequency effects in naming. , 1987 .

[76]  M. Coltheart,et al.  Attention and Performance XII: The Psychology of Reading , 1987 .

[77]  R. Treiman,et al.  Are there onset- and rime-like units in printed words? , 1987 .

[78]  James L. McClelland,et al.  Conspiracy effects in word pronunciation. , 1987 .

[79]  Geoffrey E. Hinton,et al.  Learning sets of filters using back-propagation , 1987 .

[80]  A. Parkin,et al.  Further explorations of the consistency effect in word and nonword pronunciation , 1987, Memory & cognition.

[81]  G. C. Orden A ROWS is a ROSE: Spelling, sound, and reading , 1987 .

[82]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[83]  James L. McClelland The Case for Interactionism in Language Processing. , 1987 .

[84]  S. Pinker,et al.  On language and connectionism: Analysis of a parallel distributed processing model of language acquisition , 1988, Cognition.

[85]  James A. Reggia,et al.  Competitive Dynamics in a Dual-route Connectionist Model of Print-to-sound Transformation , 1988, Complex Syst..

[86]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[87]  S. Black,et al.  Semantic encoding of pictures and words: Some neuropsychological observations , 1988 .

[88]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[89]  T. Bever,et al.  The relation between linguistic structure and associative theories of language learning—A constructive critique of some connectionist learning models , 1988, Cognition.

[90]  Steven Pinker,et al.  On language and connectionism , 1988 .

[91]  James L. McClelland,et al.  Constituent Attachment and Thematic Role Assignment in Sentence Processing: Influences of Content-Based Expectations , 1988 .

[92]  G. Bower,et al.  Evaluating an adaptive network model of human learning , 1988 .

[93]  D. Massaro Some criticisms of connectionist models of human performance , 1988 .

[94]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[95]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: part 1.: an account of basic findings , 1988 .

[96]  D. Massaro Testing between the TRACE model and the fuzzy logical model of speech perception , 1989, Cognitive Psychology.

[97]  T. Whitlock Dyslexia , 1989, The Lancet.

[98]  Barak A. Pearlmutter Learning State Space Trajectories in Recurrent Neural Networks , 1988, Neural Computation.

[99]  Paul Smolensky,et al.  Virtual Memories and Massive Generalization in Connectionist Combinatorial Learning ; CU-CS-431-89 , 1989 .

[100]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[101]  Richard J. Brown Neuropsychology Mental Structure , 1989 .

[102]  James L. McClelland,et al.  Connections and disconnections: Acquired dyslexia in a computational model of reading processes. , 1989 .

[103]  G. Stone,et al.  Are words represented by nodes? , 1989, Memory & cognition.

[104]  Geoffrey E. Hinton,et al.  Parallel Models of Associative Memory , 1989 .

[105]  Geoffrey E. Hinton Connectionist Learning Procedures , 1989, Artif. Intell..

[106]  James L. McClelland,et al.  Sentence comprehension: A parallel distributed processing approach , 1989, Language and Cognitive Processes.

[107]  Geoffrey E. Hinton,et al.  Proceedings of the 1988 Connectionist Models Summer School , 1989 .

[108]  Michael C. Mozer Discovering Faithful `Wickelfeature' Representations in a Connectionist Network , 1990 .

[109]  James L. McClelland,et al.  More Words but Still No Lexicon : Reply to Besner et al . ( 1990 ) , 1990 .

[110]  Jing Peng,et al.  An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories , 1990, Neural Computation.

[111]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[112]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[113]  James L. McClelland,et al.  More Words but Still No Lexicon: Reply to Besner et al. (1990) , 1990 .

[114]  Bruce F. Pennington,et al.  Word identification in reading and the promise of subsymbolic psycholinguistics. , 1990, Psychological review.

[115]  Geoffrey E. Hinton Mapping Part-Whole Hierarchies into Connectionist Networks , 1990, Artif. Intell..

[116]  Stephen José Hanson,et al.  What connectionist models learn: Learning and representation in connectionist networks , 1990, Behavioral and Brain Sciences.

[117]  Mark S. Seidenberg,et al.  Naming multisyllabic words. , 1990, Journal of experimental psychology. Human perception and performance.

[118]  Ken N. Seergobin,et al.  On the association between connectionism and data: Are a few words necessary? , 1990 .

[119]  D. Burr,et al.  What connectionist models learn , 1990 .

[120]  David E. Rumelhart,et al.  Brain style computation: learning and generalization , 1990 .

[121]  R. Friedman,et al.  The Continuum of Deep/Phonological Alexia , 1990, Cortex.

[122]  Mark S. Seidenberg,et al.  The basis of consistency effects in word naming , 1990 .

[123]  G. Stone,et al.  Word identification in reading and the promise of subsymbolic psycholinguistics. , 1990, Psychological review.

[124]  Mark Derthick,et al.  Mundane Reasoning by Settling on a Plausible Model , 1990, Artif. Intell..

[125]  V. Marchman,et al.  U-shaped learning and frequency effects in a multi-layered perception: Implications for child language acquisition , 1991, Cognition.

[126]  James L. McClelland Stochastic interactive processes and the effect of context on perception , 1991, Cognitive Psychology.

[127]  Michael C. Mozer,et al.  Perception of multiple objects - a connectionist approach , 1991, Neural network modeling and connectionism.

[128]  K. Paap,et al.  Dual-route models of print to sound: Still a good horse race , 1991 .

[129]  B. MacWhinney,et al.  Implementations are not conceptualizations: Revising the verb learning model , 1991, Cognition.

[130]  H. Coslett Read but not write “Idea”: Evidence for a third reading mechanism , 1991, Brain and Language.

[131]  D. Besner,et al.  Visual Word Recognition: Evidence for Strategic Control of Lexical and Nonlexical Routines in Oral Reading , 1991 .

[132]  Geoffrey E. Hinton,et al.  Lesioning an attractor network: investigations of acquired dyslexia. , 1991, Psychological review.

[133]  M. McCloskey Networks and Theories: The Place of Connectionism in Cognitive Science , 1991 .

[134]  Philip T. Quinlan,et al.  Connectionsim and psychology - a psychological perspective on new connectionist research , 1991 .

[135]  M. Snowling,et al.  The Nonword Reading Deficit in Developmental Dyslexia: A Review. , 1992 .

[136]  J. Hodges,et al.  Deterioration of word meaning: Implications for reading , 1992, Neuropsychologia.

[137]  S. Monsell,et al.  Lexical and sublexical translation of spelling to sound : Strategic anticipation of lexical status , 1992 .

[138]  Mark F. St. John,et al.  The Story Gestalt: A Model of Knowledge-Intensive Processes in Text Comprehension , 1992, Cogn. Sci..

[139]  Mark S. Seidenberg,et al.  Rules or connections? The past tense revisited , 1992 .

[140]  H. Diesfeldt Impaired and preserved semantic memory functions in dementia , 1992 .

[141]  James L. McClelland,et al.  READING EXCEPTION WORDS AND PSEUDOWORDS - ARE 2 ROUTES REALLY NECESSARY , 1992 .

[142]  L. Bäckman Memory functioning in dementia , 1992 .

[143]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[144]  M. Behrmann,et al.  Surface dyslexia and dysgraphia: dual routes, single lexicon , 1992 .

[145]  A. Berti Handbook of neuropsychology Edited by F. Boller, J. Grafman, G. Rizzolatti and H. Goodglass. Elsevier, New York, 1988, 441 pp , 1992, Neuropsychologia.

[146]  K. Patterson Phonological ALEXIA or PHONOLOGICAL alexia , 1995 .

[147]  James Hoeffner,et al.  Are Rules a Thing of the Past?: The Acquisition of Verbal Morphology by an Attractor Network. , 1992 .

[148]  Richard S. Sutton,et al.  Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta , 1992, AAAI.

[149]  D. Holender,et al.  Analytic approaches to human cognition , 1992 .

[150]  S Pinker,et al.  Overregularization in language acquisition. , 1992, Monographs of the Society for Research in Child Development.

[151]  M. C. Smith,et al.  Models of visual word recognition: When obscuring the stimulus yields a clearer view. , 1992 .

[152]  D. Besner,et al.  The process of lexical decision: More words about a parallel distributed processing model. , 1992 .

[153]  T. Shallice,et al.  Deep Dyslexia: A Case Study of , 1993 .

[154]  A. Caramazza,et al.  The assignment of word stress in oral reading: Evidence from a case of acquired dyslexia. , 1993 .

[155]  James L. McClelland,et al.  Can a perceptual processing deficit explain the impairment of inflectional morphology in developmental dysphasia? A computational investigation. , 1993 .

[156]  Peter M. Todd,et al.  Learning and connectionist representations , 1993 .

[157]  D. Besner,et al.  Reading Aloud: Evidence for the Use of a Whole Word Nonsemantic Pathway , 1993 .

[158]  A. H. Kawamoto Nonlinear dynamics in the resolution of lexical ambiguity: A parallel distributed processing account. , 1993 .

[159]  M. Coltheart,et al.  Varieties of developmental dyslexia , 1993, Cognition.

[160]  G. Humphreys,et al.  Basic processes in reading : visual word recognition , 1993 .

[161]  James L. McClelland,et al.  Generalization with Componential Attractors: Word and Nonword Reading in an Attractor Network , 1993 .

[162]  Mark S. Seidenberg Connectionist Models and Cognitive Theory , 1993 .

[163]  D. Balota,et al.  A dissociation of frequency and regularity effects in pronunciation performance across young adults, older adults, and individuals with senile dementia of the Alzheimer type. , 1993 .

[164]  Paul W. B. Atkins,et al.  Models of reading aloud: Dual-route and parallel-distributed-processing approaches. , 1993 .

[165]  Eric L. Schwartz,et al.  Computational Neuroscience , 1993, Neuromethods.

[166]  V. Marchman Constraints on Plasticity in a Connectionist Model of the English Past Tense , 1993, Journal of Cognitive Neuroscience.

[167]  Nigel H. Goddard,et al.  Proceedings of the 15th Annual Conference of the Cognitive Science Society , 1993 .

[168]  David C. Plaut,et al.  Deep Dyslexia: A Case Study of , 1993 .

[169]  V. Marchman,et al.  From rote learning to system building: acquiring verb morphology in children and connectionist nets , 1993, Cognition.

[170]  William A. Phillips,et al.  Lexicality and pronunciation in a simulated neural net , 1993 .

[171]  Javier R. Movellan,et al.  Learning Continuous Probability Distributions with Symmetric Diffusion Networks , 1993, Cogn. Sci..

[172]  M K Tanenhaus,et al.  A constraint-based lexicalist account of the subject/object attachment preference , 1994, Journal of psycholinguistic research.

[173]  C. Kello,et al.  Locus of the exception effect in naming , 1994 .

[174]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .

[175]  G. C. Orden,et al.  Interdependence of form and function in cognitive systems explains perception of printed words. , 1994, Journal of experimental psychology. Human perception and performance.

[176]  Maryellen C. MacDonald,et al.  Probabilistic constraints and syntactic ambiguity resolution , 1994 .

[177]  J. Hodges,et al.  Reading in dementia of the Alzheimer type: A preserved ability? , 1994 .

[178]  Susan M. Garnsey,et al.  Semantic Influences On Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution , 1994 .

[179]  D. Norris A quantitative multiple-levels model of reading aloud. , 1994 .

[180]  G. Kane Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[181]  James L. McClelland,et al.  Nonword pronunciation and models of word recognition. , 1994, Journal of experimental psychology. Human perception and performance.

[182]  J. Hodges,et al.  The relationship between comprehension and oral reading in progressive fluent aphasia , 1994, Neuropsychologia.

[183]  L. Katz,et al.  Evidence of flexible coding in visual word recognition. , 1994, Journal of experimental psychology. Human perception and performance.

[184]  Franz Kurfess,et al.  Connectionist Symbol Processing , 1994 .

[185]  Garrison W. Cottrell,et al.  Acquiring the Mapping from Meaning to Sounds , 1994, Connect. Sci..

[186]  M. Coltheart,et al.  Serial processing in reading aloud: Evidence for dual-route models of reading. , 1994 .

[187]  K. Forster Computational modeling and elementary process analysis in visual word recognition. , 1994, Journal of experimental psychology. Human perception and performance.

[188]  G O Stone,et al.  Building a resonance framework for word recognition using design and system principles. , 1994, Journal of experimental psychology. Human perception and performance.

[189]  Yves Chauvin,et al.  Backpropagation: the basic theory , 1995 .

[190]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[191]  J. Elman,et al.  Learning and morphological change , 1995, Cognition.

[192]  Mark S. Seidenberg,et al.  Semantic effects in single-word naming. , 1995, Journal of experimental psychology. Learning, memory, and cognition.

[193]  D. Plaut Double dissociation without modularity: evidence from connectionist neuropsychology. , 1995, Journal of clinical and experimental neuropsychology.

[194]  J. Bullinaria Representation , Learning , Generalization and Damage in Neural Network Models of Reading Aloud , 1995 .

[195]  Joel L. Davis,et al.  An Introduction to Neural and Electronic Networks , 1995 .

[196]  T. Shallice,et al.  Reading without Semantics , 1983 .

[197]  Mark S. Seidenberg,et al.  The special role of rimes in the description, use, and acquisition of English orthography. , 1995, Journal of experimental psychology. General.

[198]  E. Warrington,et al.  Semantic memory and reading abilities: A case report , 1995, Journal of the International Neuropsychological Society.

[199]  S. Sasanuma,et al.  Progressive aphasia and surface Alexia in Japanese , 1995 .

[200]  Mark S. Seidenberg,et al.  Pseudohomophone effects and models of word recognition. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[201]  Pseudohomophone effects and models of word recognition. , 1996 .

[202]  Mark S. Seidenberg,et al.  On the bases of two subtypes of development dyslexia , 1996, Cognition.

[203]  R. Friedman Recovery from Deep Alexia to Phonological Alexia: Points on a Continuum , 1996, Brain and Language.

[204]  Huaiyu Zhu On Information and Sufficiency , 1997 .

[205]  M. Behrmann,et al.  Frequency and consistency effects in a pure surface dyslexic patient. , 1997 .

[206]  James L. McClelland,et al.  Impaired Oral Reading in Surface Dyslexia: Detailed Comparison of a Patient and a Connectionist Model , 1998 .

[207]  M. Zorzi,et al.  Two routes or one in reading aloud? A connectionist dual-process model. , 1998 .