Natural Language Processing With Modular PDP Networks and Distributed Lexicon

An approach to connectionist natural language processing is proposed, which is based on hierarchically organized modular Parallel Distributed Processing (PDP) networks and a central lexicon of distributed input/output representations. The modules communicate using these representations, which are global and publicly available in the system. The representations are developed automatically by all networks while they are learning their processing tasks. The resulting representations re ect the regularities in the subtasks, which facilitates robust processing in the face of noise and damage, supports improved generalization, and provides expectations about possible contexts. The lexicon can be extended by cloning new instances of the items, that is, by generating a number of items with known processing properties and distinct identities. This technique combinatorially increases the processing power of the system. The recurrent FGREP module, together with a central lexicon, is used as a basic building block in modeling higher level natural language tasks. A single module is used to form case-role representations of sentences from word-by-word sequential natural language input. A hierarchical organization of four recurrent FGREP modules (the DISPAR system) is trained to produce fully expanded paraphrases of script-based stories, where unmentioned events and role llers are inferred.

[1]  J. Piaget,et al.  The Growth Of Logical Thinking From Childhood To Adolescence: An Essay On The Construction Of Formal Operational Structures , 1958 .

[2]  M R Quillian,et al.  Word concepts: a theory and simulation of some basic semantic capabilities. , 1967, Behavioral science.

[3]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[4]  Logical Abilities in Children , 1974 .

[5]  Richard Edward Cullingford,et al.  Script application: computer understanding of newspaper stories. , 1977 .

[6]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[7]  Ii Gerald Francis Dejong Skimming stories in real time: an experiment in integrated understanding. , 1979 .

[8]  John B. Black,et al.  Scripts in memory for text , 1979, Cognitive Psychology.

[9]  R. Schank,et al.  Inside Computer Understanding , 1981 .

[10]  M. Dyer In-Depth Understanding: A Computer Model of Integrated Processing for Narrative Comprehension , 1983 .

[11]  Bruce G. Buchanan,et al.  The MYCIN Experiments of the Stanford Heuristic Programming Project , 1985 .

[12]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[13]  Geoffrey E. Hinton,et al.  The appeal of parallel distributed processing , 1986 .

[14]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[15]  Jerome A. Feldman,et al.  Neural Representation of Conceptual Knowledge. , 1986 .

[16]  James L. McClelland,et al.  Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences , 1986 .

[17]  James L. McClelland,et al.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition : Psychological and Biological Models , 1986 .

[18]  Noel E. Sharkey,et al.  Mixing Binary and Continuous Connection Schemes for Knowledge Access , 1986, AAAI.

[19]  Geoffrey E. Hinton,et al.  A general framework for parallel distributed processing , 1986 .

[20]  Richard E. Korf,et al.  Planning as Search: A Quantitative Approach , 1987, Artif. Intell..

[21]  Dana H. Ballard,et al.  Modular Learning in Neural Networks , 1987, AAAI.

[22]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[23]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[24]  Charles P. Dolan,et al.  Implementing a Connectionist Production System Using Tensor Products ; CU-CS-411-88 , 1988 .

[25]  R. Miikkulainen,et al.  Forming global representations with extended backpropagation , 1988, IEEE 1988 International Conference on Neural Networks.

[26]  James L. McClelland,et al.  Learning Subsequential Structure in Simple Recurrent Networks , 1988, NIPS.

[27]  H. Ritter Combining self-organizing maps , 1989, International 1989 Joint Conference on Neural Networks.

[28]  C. P. Dolan Tensor manipulation networks: connectionist and symbolic approaches to comprehension, learning, and planning , 1989 .

[29]  David S. Touretzky Connectionism and Compositional Semantics , 1989 .

[30]  Ajay N. Jain A connectionist architecture for sequential symbolic domains , 1989 .

[31]  Geoffrey E. Hinton Learning distributed representations of concepts. , 1989 .

[32]  Paul Smolensky,et al.  Virtual Memories and Massive Generalization in Connectionist Combinatorial Learning ; CU-CS-431-89 , 1989 .

[33]  R. Miikkulainen,et al.  A modular neural network architecture for sequential paraphrasing of script-based stories , 1989, International 1989 Joint Conference on Neural Networks.

[34]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[35]  Michael G. Dyer,et al.  Storing and Generalizing Multiple Instances While Maintaining Knowledge-Level Parallelism , 1989, IJCAI.

[36]  Michael I. Jordan,et al.  Task Decomposition through Competition in A , 1990 .

[37]  Michael G. Dyer,et al.  Argument comprehension and retrieval for editorial text , 1990, Knowl. Based Syst..

[38]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[39]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[40]  Risto Miikkulainen,et al.  Discern: a distributed artificial neural network model of script processing and memory , 1990 .

[41]  Risto Miikkulainen A PDP Architecture For Processing Sentences With Relative Clauses , 1990, COLING.

[42]  Michael I. Jordan Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[43]  G. Reeke The society of mind , 1991 .

[44]  Michael I. Jordan,et al.  Task Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks , 1990, Cogn. Sci..

[45]  Risto Miikkulainen,et al.  Script Recognition with Hierarchical Feature Maps , 1992 .

[46]  Vincent Kanade,et al.  Clustering Algorithms , 2021, Wireless RF Energy Transfer in the Massive IoT Era.