Text and Discourse Understanding: The DISCERN System

The subsymbolic approach to natural language processing (NLP) captures a number of intriguing properties of human-like information processing such as learning from examples, context sensitivity, generalization, robustness of behavior, and intuitive reasoning. Within this new paradigm, the central issues are quite different from (even incompatible with) the traditional issues in symbolic NLP, and the research has proceeded without much in common with the past. However, the ultimate goal is still the same: to understand how humans process language. Even if NLP is being built on a new foundation, as can be argued, many of the results obtained through symbolic research are still valid, and could be used as a guide for developing subsymbolic models of natural language processing. This is where DISCERN (DIstributed SCript processing and Episodic memoRy Network (Miikkulainen 1993), a subsymbolic neural network model of script-based story understanding, fits in. DISCERN is purely a subsymbolic model, but at the high level it consists of modules and information structures similar to those of symbolic systems, such as scripts, lexicon, and episodic memory. At the highest level of natural language processing such as text and discourse understanding, the symbolic and subsymbolic paradigms have to address the same basic issues. Outlining a subsymbolic approach to those issues is the purpose of DISCERN. In more specific terms, DISCERN aims: (1) to demonstrate that distributed artificial neural networks can be used to build a large-scale natural language processing system that performs approximately at the level of symbolic models; (2) to show that several cognitive phenomena can be explained at the subsymbolic level using the special properties of these networks; and (3) to identify central issues in subsymbolic NLP and to develop well-motivated techniques to deal with them. To the extent that DISCERN is successful in these areas, it constitutes a first step towards building text and discourse understanding systems within the subsymbolic paradigm.

[1]  Richard Edward Cullingford,et al.  Script application: computer understanding of newspaper stories. , 1977 .

[2]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[3]  Arthur C. Graesser,et al.  Recognition memory for typical and atypical actions in scripted activities: Tests of a script pointer + tag hypothesis , 1979 .

[4]  John B. Black,et al.  Scripts in memory for text , 1979, Cognitive Psychology.

[5]  Janet L. Kolodner,et al.  Retrieval and organizational strategies in conceptual memory: a computer model , 1980 .

[6]  A. Graesser,et al.  Memory for typical and atypical actions in scripted activities. , 1980 .

[7]  James D. Hollan,et al.  The Process of Retrieval from Very Long-Term Memory , 1978, Cogn. Sci..

[8]  M. Dyer.  In-Depth Understanding: A Computer Model of Integrated Processing for Narrative Comprehension , 1983 .

[9]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[10]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[11]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[12]  A. Caramazza.  Some aspects of language processing revealed through the analysis of acquired aphasia: the lexical system. , 1988, Annual review of neuroscience.

[13]  E. Warrington,et al.  Cognitive Neuropsychology: A Clinical Introduction , 1990 .

[14]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[15]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[16]  Michael G. Dyer,et al.  Argument representation for editorial text , 1990, Knowl. Based Syst..

[17]  Risto Mukkulainen,et al.  Script Recognition with Hierarchical Feature Maps , 1990 .

[18]  John F. Reeves,et al.  Computational morality: a process model of belief conflict and resolution for story understanding , 1991 .

[19]  Risto Miikkulainen,et al.  Natural Language Processingwith Modular Neural Networks and Distributed Lexicon , 1991 .

[20]  Risto Miikkulainen,et al.  Subsymbolic natural language processing - an integrated model of scripts, lexicon, and memory , 1993, Neural network modeling and connectionism.

[21]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[22]  R. Miikkulainen.  Dyslexic and Category-Specific Aphasic Impairments in a Self-Organizing Feature Map Model of the Lexicon , 1997, Brain and Language.

[23]  A. Collins.  The Psychology of Memory. , 2001 .

[24]  Risto Miikkulainen,et al.  Trace feature map: a model of episodic associative memory , 2004, Biological Cybernetics.

[25]  Risto Miikkulainen,et al.  Script-based inference and memory retrieval in subsymbolic story processing , 1995, Applied Intelligence.