Modelling Parsing Constraints with High-dimensional Context Space

Deriving representations of meaning has been a long-standing problem in cognitive psychology and psycholinguistics. The lack of a m odel for representing semantic and grammatical knowledge has been a handicap in attempting to model the effects of semantic constraints in hum an syntactic processing. A computational model of high-dim ensional context space, the Hyperspace A nalogue to Language (H AL), is presented with a series of simulations modelling a variety of human empirical results. HAL learns its representations from the unsupervised processing of 300 million words of conversational text. W e propose that HAL's high-dim ensional context space can be used to (1) provide a basic categorisation of semantic and grammatical concepts, (2) model certain aspects of morphological ambiguity in verbs, and (3) provide an account of semantic context effects in syntactic processing. W e propose that the distributed and contextually derived representations that HAL acquires provide a basis for the subconceptual kn...

[1]  K. Nelson,et al.  The syntagmatic-paradigmatic shift revisited: a review of research and theory. , 1977, Psychological bulletin.

[2]  Bernard Mérialdo,et al.  Tagging English Text with a Probabilistic Model , 1994, CL.

[3]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[4]  Fernanda Ferreira,et al.  Ambiguity in context , 1989 .

[5]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[6]  Paul Gorrell.  Establishing the loci of serial and parallel effects in syntactic processing , 1989 .

[7]  L. Komatsu.  Recent views of conceptual structure , 1992 .

[8]  S. Ervin.  Changes with age in the verbal determinants of word association. , 1961, The American journal of psychology.

[9]  S. Ervin.  Correlates of associative frequency , 1963 .

[10]  M. Tanenhaus,et al.  Verb-specific constraints in sentence processing: separating effects of lexical preference from garden-paths. , 1993, Journal of experimental psychology. Learning, memory, and cognition.

[11]  J. H. Neely.  Semantic priming effects in visual word recognition: A selective review of current findings and theories. , 1991 .

[12]  Lorraine K. Tyler,et al.  A Distributed Memory Model of the Associative Boost in Semantic Priming , 1994, Connect. Sci..

[13]  Susan M. Garnsey,et al.  Semantic Influences On Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution , 1994 .

[14]  M. Masson.  A distributed memory model of semantic priming. , 1995 .

[15]  J. Deese.  The structure of associations in language and thought , 1966 .

[16]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[17]  Mats Rooth,et al.  Structural Ambiguity and Lexical Relations , 1991, ACL.

[18]  W. Marslen-Wilson,et al.  Accessing Different Types of Lexical Semantic Information: Evidence From Priming , 1995 .

[19]  I. Fischler.  Semantic facilitation without association in a lexical decision task , 1977, Memory & cognition.

[20]  Janet D. Fodor,et al.  The sausage machine: A new two-stage parsing model , 1978, Cognition.

[21]  M. MacDonald,et al.  Individual Differences and Probabilistic Constraints in Syntactic Ambiguity Resolution , 1995 .

[22]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .

[23]  Roger W. Schvaneveldt,et al.  Pathfinder associative networks: studies in knowledge organization , 1990 .

[24]  Frank Smadja,et al.  Retrieving Collocations from Text: Xtract , 1993, CL.

[25]  Geoffrey E. Hinton,et al.  Lesioning an attractor network: investigations of acquired dyslexia , 1991 .

[26]  H S Kurtzman,et al.  On-Line Probe Recognition Latencies Within Complex Sentences , 1985, Language and speech.

[27]  C. Osgood,et al.  The Measurement of Meaning , 1958 .

[28]  Bruce F. Pennington,et al.  Word identification in reading and the promise of subsymbolic psycholinguistics. , 1990, Psychological review.

[29]  C. Clifton,et al.  The independence of syntactic processing , 1986 .

[30]  Lyn Frazier,et al.  The interaction of syntax and semantics during sentence processing: eye movements in the analysis of semantically biased sentences , 1983 .

[31]  Stephen I. Gallant.  A Practical Approach for Representing Context and for Performing Word Sense Disambiguation Using Neural Networks , 1991, Neural Computation.

[32]  Michael K. Tanenhaus,et al.  Lexical structure and language comprehension , 1989 .

[33]  Randi C. Martin,et al.  How semantic is automatic semantic priming , 1992 .

[34]  Lance J. Rips,et al.  Structure and process in semantic memory: A featural model for semantic decisions. , 1974 .

[35]  Lyn Frazier,et al.  ON COMPREHENDING SENTENCES: SYNTACTIC PARSING STRATEGIES. , 1979 .

[36]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[37]  R. Schvaneveldt,et al.  Facilitation in recognizing pairs of words: evidence of a dependence between retrieval operations. , 1971, Journal of experimental psychology.

[38]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[39]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[40]  James L. McClelland,et al.  Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences , 1986 .

[41]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[42]  H. Schütze,et al.  Dimensions of meaning , 1992, Supercomputing '92.

[43]  S. Lupker.  Semantic Priming without Association: A Second Look. , 1984 .

[44]  D. Spence,et al.  Lexical co-occurrence and association strength , 1990 .

[45]  Garland Cannon,et al.  The Holt guide to English , 1972 .

[46]  Michael G. Dyer,et al.  Distributed symbol formation and processing in connectionist networks , 1990, J. Exp. Theor. Artif. Intell..

[47]  Maryellen C. MacDonald,et al.  Probabilistic constraints and syntactic ambiguity resolution , 1994 .

[48]  G. Mandler,et al.  From association to structure. , 1962, Psychological review.

[49]  A. H. Kawamoto.  Nonlinear dynamics in the resolution of lexical ambiguity: A parallel distributed processing account. , 1993 .

[50]  Curt Burgess,et al.  A Computational Model of Syntactic Ambiguity as a Lexical Process , 1989 .

[51]  M. Ross Quillian,et al.  Retrieval time from semantic memory , 1969 .