Sentence comprehension: A parallel distributed processing approach

Abstract In this paper, we review basic aspects of conventional approaches to sentence comprehension and point out some of the difficulties faced by models that take these approaches. We then describe an alternative approach, based on the principles of parallel distributed processing, and show how it offers different answers to basic questions about the nature of the language processing mechanism. We describe an illustrative simulation model that captures the key characteristics of the approach, and illustrate how it can cope with the difficulties faced by conventional models. We describe alternative ways of conceptualising basic aspects of language processing within the framework of this approach, consider how it can address several arguments that might be brought to bear against it, and suggest avenues for future development.

[1]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[2]  S. Blumstein,et al.  Phonological processing and lexical access in aphasia , 1988, Brain and Language.

[3]  Geoffrey E. Hinton,et al.  A Distributed Connectionist Production System , 1988, Cogn. Sci..

[4]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[5]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[6]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[7]  James L. McClelland,et al.  Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences , 1986 .

[8]  David Caplan,et al.  Disorders of Syntactic Comprehension , 1988 .

[9]  James L. McClelland,et al.  Learning Subsequential Structure in Simple Recurrent Networks , 1988, NIPS.

[10]  Geoffrey E. Hinton.  Learning distributed representations of concepts. , 1989 .

[11]  James L. McClelland.  The Case for Interactionism in Language Processing. , 1987 .

[12]  Mark Steedman,et al.  Interaction with context during human sentence processing , 1988, Cognition.

[13]  Mark S. Seidenberg,et al.  Evidence for Multiple Stages in the Processing of Ambiguous Words in Syntactic Contexts. , 1979 .

[14]  Mitchell P. Marcus,et al.  A theory of syntactic recognition for natural language , 1979 .

[15]  M. Mattson,et al.  From words to meaning: A semantic illusion , 1981 .

[16]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[17]  Charles J. Fillmore,et al.  THE CASE FOR CASE. , 1967 .

[18]  James L. McClelland.  Parallel Distributed Processing: Implications for Cognition and Development , 1988 .

[19]  D. Swinney.  Lexical access during sentence comprehension: (Re)consideration of context effects , 1979 .

[20]  R. Miikkulainen,et al.  A modular neural network architecture for sequential paraphrasing of script-based stories , 1989, International 1989 Joint Conference on Neural Networks.

[21]  Lyn Frazier,et al.  Theories of sentence processing , 1987 .

[22]  James L. McClelland.  On the time relations of mental processes: An examination of systems of processes in cascade. , 1979 .

[23]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: part 1.: an account of basic findings , 1988 .

[24]  Gregg C. Oden,et al.  Semantic constraints and judged preference for interpretations of ambiguous sentences , 1978 .

[25]  Roger C. Schank,et al.  Language and Memory , 1986, Cogn. Sci..

[26]  B. MacWhinney,et al.  The Crosslinguistic Study of Sentence Processing. , 1992 .

[27]  Geoffrey E. Hinton,et al.  Schemata and Sequential Thought Processes in PDP Models , 1986 .

[28]  Alan H. Kawamoto,et al.  Dynamic processes in the (re)solution of lexical ambiguity , 1985 .

[29]  Gerry T. M. Altmann,et al.  Ambiguity, parsing strategies, and computational models , 1988 .

[30]  H. H. Clark,et al.  Psychology and language : an introduction to psycholinguistics , 1979 .

[31]  Geoffrey E. Hinton,et al.  A Distributed Connectionist Production System , 1988, Cogn. Sci..

[32]  J. Elman.  Representation and structure in connectionist models , 1991 .

[33]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[34]  James L. McClelland,et al.  Constituent Attachment and Thematic Role Assignment in Sentence Processing: Influences of Content-Based Expectations , 1988 .

[35]  C. Clifton,et al.  The independence of syntactic processing , 1986 .

[36]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: Part 2. The contextual enhancement effect and some tests and extensions of the model. , 1982, Psychological review.

[37]  David S. Touretzky,et al.  A distributed connectionist representation for concept structures , 1987 .

[38]  J. Woolley,et al.  Paradigms and processes in reading comprehension. , 1982, Journal of experimental psychology. General.

[39]  Lyn Frazier,et al.  The interaction of syntax and semantics during sentence processing: eye movements in the analysis of semantically biased sentences , 1983 .

[40]  A. H. Kawamoto.  Distributed Representations of Ambiguous Words and Their Resolution in a Connectionist Network , 1988 .

[41]  Marcel Adam Just,et al.  Paradigms and processes in reading comprehension , 1982 .

[42]  Robert B. Allen,et al.  Several Studies on Natural Language ·and Back-Propagation , 1987 .

[43]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[44]  Eric Wanner,et al.  Language acquisition: the state of the art , 1982 .

[45]  Mark Steedman,et al.  The use of context by the psychological parser , 1981 .

[46]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .