Bifurcation analysis of a Gradient Symbolic Computation model of incremental processing

Language is ordered in time and an incremental processing system encounters temporary ambiguity in the middle of sentence comprehension. An optimal incremental processing system must solve two computational problems: On the one hand, it has to keep multiple possible interpretations without choosing one over the others. On the other hand, it must reject interpretations inconsistent with context. We propose a recurrent neural network model of incremental processing that does stochastic optimization of a set of soft, local constraints to build a globally coherent structure successfully. Bifurcation analysis of the model makes clear when and why the model parses a sentence successfully and when and why it does not— the garden path and local coherence effects are discussed. Our model provides neurally plausible solutions of the computational problems arising in incremental processing

[1]  Matthew Goldrick,et al.  Optimization and Quantization in Gradient Symbol Systems: A Framework for Integrating the Continuous and the Discrete in Cognition , 2014, Cogn. Sci..

[2]  Geoffrey E. Hinton.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[3]  Daniel C. Richardson,et al.  Effects of merely local syntactic coherence on sentence processing , 2004 .

[4]  A Prince,et al.  Optimality: From Neural Networks to Universal Grammar , 1997, Science.

[5]  D. Lathrop.  Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering , 2015 .

[6]  Mark A. Miller,et al.  Archetypal energy landscapes , 1998, Nature.

[7]  Michael J. Spivey,et al.  Syntactic ambiguity resolution in discourse: modeling the effects of referential context and lexical frequency. , 1998, Journal of experimental psychology. Learning, memory, and cognition.

[8]  G. Kempen,et al.  Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar , 2000, Cognition.

[9]  M. Karplus,et al.  The topology of multidimensional potential energy surfaces: Theory and application to peptide structure and kinetics , 1997 .

[10]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[11]  C. Hwang,et al.  Diffusion for global optimization in R n , 1987 .

[12]  M. Coltheart.  Attention and Performance XII: The Psychology of Reading , 1987 .

[13]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .

[14]  Hil Meijer,et al.  Numerical Bifurcation Analysis , 2009, Encyclopedia of Complexity and Systems Science.

[15]  M. Pickering,et al.  Adjunct attachment is not a form of lexical ambiguity resolution , 1998 .

[16]  John Hale,et al.  A Probabilistic Earley Parser as a Psycholinguistic Model , 2001, NAACL.

[17]  W. Tabor,et al.  Evidence for self-organized sentence processing: digging-in effects. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[18]  Lyn Frazier,et al.  Sentence processing: A tutorial review. , 1987 .

[19]  K. Rayner,et al.  Making and correcting errors during sentence comprehension: Eye movements in the analysis of structurally ambiguous sentences , 1982, Cognitive Psychology.

[20]  R. Levy.  Expectation-based syntactic comprehension , 2008, Cognition.

[21]  Richard L. Lewis,et al.  Computational principles of working memory in sentence comprehension , 2006, Trends in Cognitive Sciences.

[22]  Stefan L. Frank,et al.  Surprisal-based comparison between a symbolic and a connectionist model of sentence processing , 2009 .