Cognitive Biases, Linguistic Universals, and Constraint-Based Grammar Learning

According to classical arguments, language learning is both facilitated and constrained by cognitive biases. These biases are reflected in linguistic typology-the distribution of linguistic patterns across the world's languages-and can be probed with artificial grammar experiments on child and adult learners. Beginning with a widely successful approach to typology (Optimality Theory), and adapting techniques from computational approaches to statistical learning, we develop a Bayesian model of cognitive biases and show that it accounts for the detailed pattern of results of artificial grammar experiments on noun-phrase word order (Culbertson, Smolensky, & Legendre, 2012). Our proposal has several novel properties that distinguish it from prior work in the domains of linguistic theory, computational cognitive science, and machine learning. This study illustrates how ideas from these domains can be synthesized into a model of language learning in which biases range in strength from hard (absolute) to soft (statistical), and in which language-specific and domain-general biases combine to account for data from the macro-level scale of typological distribution to the micro-level scale of learning by individuals.

[1]  Thomas L. Griffiths,et al.  The evolution of frequency distributions: Relating regularization to inductive biases through iterated learning , 2009, Cognition.

[2]  J. Grimshaw Projection, heads, and optimality , 1997 .

[3]  John J. McCarthy,et al.  A Thematic Guide to Optimality Theory , 2001 .

[4]  Colin Wilson,et al.  Learning Phonology With Substantive Bias: An Experimental and Computational Study of Velar Palatalization , 2006, Cogn. Sci..

[5]  Jennifer Culbertson Learning biases, regularization, and the emergence of typological universals in syntax , 2010 .

[6]  Noam Chomsky,et al.  Lectures on Government and Binding , 1981 .

[7]  David A. Smith,et al.  Bootstrapping Feature-Rich Dependency Parsers with Entropic Priors , 2007, EMNLP-CoNLL.

[8]  Vieri Samek-Lodovici,et al.  Prosody–Syntax Interaction in the Expression of Focus , 2005 .

[9]  Jennifer Culbertson,et al.  Typological Universals as Reflections of Biased Learning: Evidence from Artificial Language Learning , 2012, Lang. Linguistics Compass.

[10]  Jennifer Culbertson,et al.  A Bayesian Model of Biases in Artificial Language Learning: The Case of a Word-Order Universal , 2012, Cogn. Sci..

[11]  Joy Philip,et al.  (Dis)harmony, the Head-Proximate Filter, and linkers1 , 2012, Journal of Linguistics.

[12]  P. Smolensky,et al.  Optimality Theory: Constraint Interaction in Generative Grammar , 2004 .

[13]  Michael Meeuwis,et al.  Order of adjective and noun , 2013 .

[14]  Joseph V. Pater Emergent Systemic Simplicity (and Complexity) , 2012 .

[15]  M. W. Weir Probability Performance: Reinforcement Procedure and Number of Alternatives , 1972 .

[16]  S. Levinson,et al.  The myth of language universals: language diversity and its importance for cognitive science. , 2009, The Behavioral and brain sciences.

[17]  Theresa Biberauer,et al.  Structure and linearization in disharmonic word orders , 2008 .

[18]  Ka Cormier,et al.  Annual Meeting of the Linguistic Society of America , 2004 .

[19]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[20]  P. Smolensky The Initial State and 'Richness of the Base' in Optimality Theory , 1996 .

[21]  Lisa deMena Travis,et al.  Parameters and effects of word order variation , 1984 .

[22]  Robert Forkel,et al.  The World Atlas of Language Structures Online , 2009 .

[23]  Guglielmo Cinque,et al.  On the Evidence for Partial N-Movement in the Romance DP , 1993 .

[24]  E. Moreton,et al.  Structure and substance in artificial-phonology learning , 2012 .

[25]  Joe Pater,et al.  Weighted Constraints in Generative Linguistics , 2009, Cogn. Sci..

[26]  Theresa Biberauer,et al.  A Syntactic Universal and Its Consequences , 2014, Linguistic Inquiry.

[27]  Peter Sells,et al.  Structure, Alignment and Optimality in Swedish , 2001 .

[28]  B. Bickel Typology in the 21st century: Major current developments , 2007 .

[29]  Padraic Monaghan,et al.  Relationships Between Language Structure and Language Learning: The Suffixing Preference and Grammatical Categorization , 2009, Cogn. Sci..

[30]  E. Newport,et al.  Getting it right by getting it wrong: When learners change languages , 2009, Cognitive Psychology.

[31]  Bruce Hayes,et al.  A Maximum Entropy Model of Phonotactics and Phonotactic Learning , 2008, Linguistic Inquiry.

[32]  Mark C. Baker The atoms of language: The mind''s hidden rules of grammar , 1987 .

[33]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[34]  P. Smolensky,et al.  Learning biases predict a word order universal , 2012, Cognition.

[35]  Barbara C. Scholz,et al.  Empirical assessment of stimulus poverty arguments , 2002 .

[36]  Christopher D. Manning,et al.  Hierarchical Bayesian Domain Adaptation , 2009, NAACL.

[37]  Richard S. Kayne The Antisymmetry of Syntax , 1994 .

[38]  Karen Jesney,et al.  Biases in Harmonic Grammar: the road to restrictive learning , 2011 .

[39]  Mark Johnson,et al.  Learning OT constraint rankings using a maximum entropy model , 2003 .

[40]  Elliott Moreton,et al.  Structure and Substance in Artificial-phonology Learning, Part I: Structure , 2012, Lang. Linguistics Compass.

[41]  Christopher D. Manning,et al.  Probabilistic models of language processing and acquisition , 2006, Trends in Cognitive Sciences.

[42]  G. M. Horn,et al.  On ‘On binding’ , 1981 .

[43]  Joseph H. Greenberg,et al.  Some Universals of Grammar with Particular Reference to the Order of Meaningful Elements , 1990, On Language.

[44]  V. Samek-Lodovici,et al.  On the Ungrammaticality of Remnant Movement in the Derivation of Greenberg's Universal 20 , 2011, Linguistic Inquiry.

[45]  B. Hayes,et al.  Natural and Unnatural Constraints in Hungarian Vowel Harmony , 2009 .

[46]  Bruce Tesar,et al.  Constraints in Phonological Acquisition: Learning phonotactic distributions , 2004 .

[47]  時崎 久夫,et al.  The Universals Archiveによる音韻と統語の相関研究 , 2012 .

[48]  Alan S. Prince,et al.  The emergence of the unmarked: Optimality in prosodic morphology , 1994 .

[49]  William D. Raymond,et al.  An Optimality-Theoretic Typology of Case and Grammatical Voice Systems , 1993 .

[50]  P. Smolensky,et al.  Analytic bias and phonological typology * , 2008 .

[51]  G. Cinque Deriving Greenberg's Universal 20 and Its Exceptions , 2005, Linguistic Inquiry.

[52]  Noam Chomsky,et al.  Minimalist inquiries : the framework , 1998 .

[53]  Christopher Laenzlinger,et al.  French adjective ordering: perspectives on DP-internal movement types , 2005 .

[54]  Bruce Tesar,et al.  Learning phonotactic distributions , 2004 .

[55]  Ronald Rosenfeld,et al.  A survey of smoothing techniques for ME models , 2000, IEEE Trans. Speech Audio Process..

[56]  Pawel M. Nowak,et al.  Phonological Rule-Learning and Its Implications for a Theory of Vowel Harmony , 2003 .

[57]  A. Anttila Deriving Variation from Grammar , 1997 .

[58]  Tom M. Mitchell,et al.  The Need for Biases in Learning Generalizations , 2007 .

[59]  Simon J. Greenhill,et al.  Evolved structure of language shows lineage-specific trends in word-order universals , 2011, Nature.

[60]  Ralf Vogel,et al.  Free relative constructions in OT-syntax , 2002 .

[61]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[62]  Wim Zonneveld,et al.  Learning phonotactic distributions , 2004 .

[63]  Bart de Boer,et al.  The Atoms of Language: The Mind's Hidden Rules of Grammar; Foundations of Language: Brain, Meaning, Grammar, Evolution , 2002, Artificial Life.

[64]  Y. Miyata,et al.  Harmonic grammar: A formal multi-level connectionist theory of linguistic well-formedness: Theoretic , 1990 .

[65]  Gaja Jarosz,et al.  Implicational markedness and frequency in constraint-based computational models of phonological learning. , 2010, Journal of child language.