Harmony in Linguistic Cognition

In this article, I survey the integrated connectionist/symbolic (ICS) cognitive architecture in which higher cognition must be formally characterized on two levels of description. At the microlevel, parallel distributed processing (PDP) characterizes mental processing; this PDP system has special organization in virtue of which it can be characterized at the macrolevel as a kind of symbolic computational system. The symbolic system inherits certain properties from its PDP substrate; the symbolic functions computed constitute optimization of a well-formedness measure called Harmony. The most important outgrowth of the ICS research program is optimality theory (Prince & Smolensky, 1993/2004), an optimization-based grammatical theory that provides a formal theory of cross-linguistic typology. Linguistically, Harmony maximization corresponds to minimization of markedness or structural ill-formedness. Cognitive explanation in ICS requires the collaboration of symbolic and connectionist principles. ICS is developed in detail in Smolensky and Legendre (2006a); this article is a précis of and guide to those volumes.

[1]  Richard M. Saenz,et al.  University of Massachusetts occasional papers in linguistics , 1976 .

[2]  J J Hopfield,et al.  Learning algorithms and probability distributions in feed-forward and feed-back networks. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Whitney Tabor,et al.  Fractal encoding of context‐free grammars in connectionist networks , 2000, Expert Syst. J. Knowl. Eng..

[4]  R. Jakobson,et al.  Selected Writings: I. Phonological Studies , 1965 .

[5]  J. Stemberger,et al.  Optimality Theory , 2003 .

[6]  P. Smolensky,et al.  Optimality in Sentence Processing , 2003 .

[7]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[8]  Paul Smolensky,et al.  Schema Selection and Stochastic Inference in Modular Environments , 1983, AAAI.

[9]  James L. McClelland.  On the time relations of mental processes: An examination of systems of processes in cascade. , 1979 .

[10]  J. Grimshaw.  Projection, heads, and optimality , 1997 .

[11]  Geoffrey E. Hinton,et al.  Parallel Models of Associative Memory , 1989 .

[12]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[13]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[14]  James L. McClelland.  Explorations In Parallel Distributed Processing , 1988 .

[15]  Stephen Grossberg,et al.  Absolute stability of global pattern formation and parallel memory storage by competitive neural networks , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[16]  Nick Chater,et al.  Connectionist natural language processing: the state of the art , 1999 .

[17]  Petra Hendriks,et al.  Guest Edito's Introduction , 2000, J. Semant..

[18]  A Prince,et al.  Optimality: From Neural Networks to Universal Grammar , 1997, Science.

[19]  Tony A. Plate,et al.  Holographic Reduced Representation: Distributed Representation for Cognitive Structures , 2003 .

[20]  John J. McCarthy,et al.  A Thematic Guide to Optimality Theory , 2001 .

[21]  J. Fodor.  Connectionism and the problem of systematicity (continued): why Smolensky's solution still doesn't work , 1997, Cognition.

[22]  Henk Zeevat,et al.  Where is pragmatics in optimality theory , 2008 .

[23]  Teuvo Kohonen,et al.  Associative memory. A system-theoretical approach , 1977 .

[24]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[25]  D. Dinnsen.  Constraints in phonological acquisition , 2004 .

[26]  G. Miller,et al.  Cognitive science. , 1981, Science.

[27]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[28]  John J. McCarthy,et al.  Optimality-Theoretic Syntax , 2008 .

[29]  R. Golden.  A unified framework for connectionist systems , 1988, Biological Cybernetics.

[30]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[31]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[32]  Michael D. Byrne,et al.  Cognitive Architecture , 2003 .

[33]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[34]  P. Smolensky,et al.  Optimality Theory: Constraint Interaction in Generative Grammar , 2004 .

[35]  P. Smolensky.  On the proper treatment of connectionism , 1988, Behavioral and Brain Sciences.

[36]  Y. Miyata,et al.  Harmonic grammar: A formal multi-level connectionist theory of linguistic well-formedness: Theoretic , 1990 .

[37]  Robert Cummins,et al.  Connectionism, computation, and cognition , 1991 .

[38]  W. Freeman.  Second Commentary: On the proper treatment of connectionism by Paul Smolensky (1988) - Neuromachismo Rekindled , 1989 .

[39]  Stephen A. Ritz,et al.  Distinctive features, categorical perception, and probability learning: some applications of a neural model , 1977 .

[40]  S. Pinker,et al.  On language and connectionism: Analysis of a parallel distributed processing model of language acquisition , 1988, Cognition.

[41]  Paul Smolensky,et al.  Harmonic Grammars for Formal Languages , 1992, NIPS.

[42]  R. Golden.  The :20Brain-state-in-a-box Neural model is a gradient descent algorithm , 1986 .

[43]  Paul Smolensky,et al.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1990, Artif. Intell..

[44]  Nick Chater,et al.  Special issue - Connectionist models of human language processing: Progress and prospects , 1999 .

[45]  Tony Plate,et al.  Holographic Reduced Representations: Convolution Algebra for Compositional Distributed Representations , 1991, IJCAI.

[46]  Richard Reviewer-Granger.  Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[47]  Daniel Jurafsky,et al.  A Probabilistic Model of Lexical and Syntactic Access and Disambiguation , 1996, Cogn. Sci..

[48]  S. Phillips,et al.  Processing capacity defined by relational complexity: implications for comparative, developmental, and cognitive psychology. , 1998, The Behavioral and brain sciences.

[49]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[50]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[51]  Colin Wilson,et al.  Experimental Investigation of Phonological Naturalness * , 2003 .

[52]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[53]  Maryellen C. MacDonald,et al.  A probabilistic constraints approach to language acquisition and processing , 1999, Cogn. Sci..

[54]  J. Stemberger,et al.  Handbook of Phonological Development: From the Perspective of Constraint-Based Nonlinear Phonology , 1999 .

[55]  John R. Anderson,et al.  Combining Optimality Theory and a Cognitive Architecture , 2003 .

[56]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[57]  C. P. Dolan.  Tensor manipulation networks: connectionist and symbolic approaches to comprehension, learning, and planning , 1989 .

[58]  Geoffrey E. Hinton,et al.  Distributed representations and nested compositional structure , 1994 .

[59]  S. Grossberg.  Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[60]  Mitsuhiko Ota,et al.  Optimality Theory: an overview , 2000 .

[61]  G. Altmann.  Ambiguity in sentence processing , 1998, Trends in Cognitive Sciences.

[62]  Géraldine Legendre,et al.  The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 2: Linguistic and philosophical implications). , 2006 .

[63]  J. Fodor,et al.  Connectionism and the problem of systematicity: Why Smolensky's solution doesn't work , 1990, Cognition.

[64]  E. Battistella.  Markedness: The Evaluative Superstructure of Language , 1990 .

[65]  Richard,et al.  The “Brain-State-in-a-Box” Neural Model Is a Gradient Descent Algorithm , 2003 .

[66]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[67]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[68]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[69]  T. Horgan,et al.  Connectionism and the Philosophy of Mind , 1991 .

[70]  T. Sejnowski,et al.  Spatial Transformations in the Parietal Cortex Using Basis Functions , 1997, Journal of Cognitive Neuroscience.

[71]  Alan S. Prince,et al.  Faithfulness and reduplicative identity , 1995 .

[72]  守屋 悦朗,et al.  J.E.Hopcroft, J.D. Ullman 著, "Introduction to Automata Theory, Languages, and Computation", Addison-Wesley, A5変形版, X+418, \6,670, 1979 , 1980 .

[73]  William D. Raymond,et al.  An Optimality-Theoretic Typology of Case and Grammatical Voice Systems , 1993 .