Learning Human-like Knowledge by Singular Value Decomposition: A Progress Report

Singular value decomposition (SVD) can be viewed as a method for unsupervised training of a network that associates two classes of events reciprocally by linear connections through a single hidden layer. SVD was used to learn and represent relations among very large numbers of words (20k-60k) and very large numbers of natural text passages (1k- 70k) in which they occurred. The result was 100-350 dimensional "semantic spaces" in which any trained or newly added word or passage could be represented as a vector, and similarities were measured by the cosine of the contained angle between vectors. Good accuracy in simulating human judgments and behaviors has been demonstrated by performance on multiple-choice vocabulary and domain knowledge tests, emulation of expert essay evaluations, and in several other ways. Examples are also given of how the kind of knowledge extracted by this method can be applied.

[1]  Carla H. Lagorio,et al.  Psychology , 1929, Nature.

[2]  J. M. Anglin.  The Growth of Word Meaning , 1961 .

[3]  E. Clark,et al.  The Growth of Word Meaning , 1961 .

[4]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[5]  A. Tversky,et al.  Judgments of and by Representativeness , 1981 .

[6]  A. Tversky,et al.  Judgment under Uncertainty , 1982 .

[7]  T. Shallice,et al.  Category specific semantic impairments , 1984 .

[8]  A. Tversky,et al.  Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment , 1983 .

[9]  W. Kintsch,et al.  Time course of priming for associate and inference words in a discourse context , 1988, Memory & cognition.

[10]  Edward E. Smith,et al.  Typicality and reasoning fallacies , 1990, Memory & cognition.

[11]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[12]  Susan T. Dumais,et al.  Improving the retrieval of information from external sources , 1991 .

[13]  Michael W. Berry,et al.  Large-Scale Sparse Singular Value Computations , 1992 .

[14]  Susan T. Dumais,et al.  Latent Semantic Indexing (LSI) and TREC-2 , 1993, TREC.

[15]  Bob Rehder,et al.  How Well Can Passage Meaning be Derived without Using Word Order? A Comparison of Latent Semantic Analysis and Humans , 1997 .

[16]  Darrell Laham,et al.  Latent Semantic Analysis Approaches to Categorization , 1997 .

[17]  Susan T. Dumais,et al.  The latent semantic analysis theory of knowledge , 1997 .

[18]  Bob Rehder,et al.  Using latent semantic analysis to assess knowledge: Some technical considerations , 1998 .

[19]  Peter W. Foltz,et al.  Analysis of text coherence using Latent Semantic Analysis , 1998 .

[20]  Peter W. Foltz,et al.  Learning from text: Matching readers and texts by latent semantic analysis , 1998 .