Subsymbolic Case-Role Analysis of Sentences with Embedded Clauses

A distributed neural network model called SPEC for processing sentences with recursive relative clauses is described. The model is based on separating the tasks of segmenting the input word sequence into clauses, forming the case-role representations, and keeping track of the recursive embeddings into different modules. The system needs to be trained only with the basic sentence constructs, and it generalizes not only to new instances of familiar relative clause structures, but to novel structures as well. SPEC exhibits plausible memory degradation as the depth of the center embedding increases, its memory is primed by earlier constituents, and its performance is aided by semantic constraints between the constituents. The ability to process structure is largely due to a central executive network that monitors and control the execution of the entire system. This way, in contrast to earlier subsymbolic systems, parsing is modeled as a controlled high- level process rather than one based on automatic reflex responses.

[1]  David J. Chalmers,et al.  Syntactic Transformations on Distributed Representations , 1990 .

[2]  D. Norman,et al.  Attention to action: Willed and automatic control , 1980 .

[3]  Donald A. Norman,et al.  Attention to Action , 1986 .

[4]  Douglas S. Blank,et al.  Exploring the Symbolic/Subsymbolic Continuum: A case study of RAAM , 1992 .

[5]  M. Huang A Developmental Study of Children's Comprehension of Embedded Sentences with and without Semantic Constraints , 1983 .

[6]  Risto Miikkulainen,et al.  Natural Language Processing With Modular PDP Networks and Distributed Lexicon , 1991, Cogn. Sci..

[7]  Mark C. Detweiler,et al.  A Connectionist/Control Architecture for Working Memory , 1988 .

[8]  Geoffrey E. Hinton,et al.  A general framework for parallel distributed processing , 1986 .

[9]  Risto Miikkulainen,et al.  Subsymbolic natural language processing - an integrated model of scripts, lexicon, and memory , 1993, Neural network modeling and connectionism.

[10]  M. Braine,et al.  Short-term memory limitations on decoding self-embedded sentences , 1974 .

[11]  Paul Smolensky,et al.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1990, Artif. Intell..

[12]  T. Shallice Specific impairments of planning. , 1982, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[13]  D Burns,et al.  Sentence comprehension and memory for embedded structure , 1977, Memory & cognition.

[14]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[15]  T. Shallice From Neuropsychology to Mental Structure , 1988 .

[16]  Donald J. Foss,et al.  Some effects of memory limitation upon sentence comprehension and recall , 1970 .

[17]  Charles J. Fillmore,et al.  THE CASE FOR CASE. , 1967 .

[18]  R. Shiffrin,et al.  Automatic and controlled processing revisited. , 1984, Psychological review.

[19]  Walter Anthony Cook Case Grammar Theory , 1979 .

[20]  Risto Miikkulainen A PDP Architecture For Processing Sentences With Relative Clauses , 1990, COLING.

[21]  N. Cowan Evolving conceptions of memory storage, selective attention, and their mutual constraints within the human information-processing system. , 1988, Psychological bulletin.

[22]  David S. Touretzky Connectionism and Compositional Semantics , 1989 .

[23]  Paul W. Munro,et al.  A Network for Encoding, Decoding and Translating Locative Prepositions , 1991 .

[24]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[25]  Walter S. Stolz,et al.  A study of the ability to decode grammatically novel sentences , 1967 .

[26]  Robert A. Boakes,et al.  Prompted recall of sentences , 1967 .

[27]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[28]  Lonnie Chrisman,et al.  Learning Recursive Distributed Representations for Holistic Computation , 1991 .

[29]  G. Logan On the ability to inhibit thought and action , 1984 .

[30]  A. Caramazza,et al.  Dissociation of algorithmic and heuristic processes in language comprehension: Evidence from aphasia , 1976, Brain and Language.

[31]  Walter Schneider,et al.  Controlled and Automatic Human Information Processing: 1. Detection, Search, and Attention. , 1977 .

[32]  M. Posner,et al.  Attention and cognitive control. , 1975 .

[33]  Cynthia L. Cosic,et al.  Learning to Represent and Understand Locative Prepositional Phrases , 1988 .

[34]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[35]  Jordan B. Pollack,et al.  High-level connectionist models , 1988 .

[36]  George A. Miller,et al.  Free Recall of Self-Embedded English Sentences , 1964, Inf. Control..

[37]  Robert F Simmons,et al.  Training a Neural Network to be a Context Sensitive Grammar , 1990 .

[38]  Michael G. Dyer,et al.  Learning Distributed Representations of Conceptual Knowledge and their Application to Script-based Story Processing , 1990 .

[39]  Michael I. Jordan,et al.  Task Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks , 1990, Cogn. Sci..

[40]  Ajay N. Jain Parsing Complex Sentences with Structured Connectionist Networks , 1991, Neural Computation.

[41]  Mark F. St. John,et al.  The Story Gestalt: A Model of Knowledge-Intensive Processes in Text Comprehension , 1992, Cogn. Sci..

[42]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[43]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[44]  James L. McClelland,et al.  Learning Subsequential Structure in Simple Recurrent Networks , 1988, NIPS.

[45]  Jeffrey L. Elman,et al.  Distributed Representations, Simple Recurrent Networks, and Grammatical Structure , 1991, Mach. Learn..

[46]  Robert B. Allen,et al.  Several Studies on Natural Language ·and Back-Propagation , 1987 .

[47]  I. M. Schlesinger Sentence structure and the reading process. , 1968 .

[48]  Geoffrey E. Hinton Learning distributed representations of concepts. , 1989 .

[49]  George Berg,et al.  A Connectionist Parser with Recursive Sentence Structure and Lexical Disambiguation , 1992, AAAI.

[50]  C. P. Dolan Tensor manipulation networks: connectionist and symbolic approaches to comprehension, learning, and planning , 1989 .

[51]  James L. McClelland,et al.  Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences , 1986 .

[52]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.