Using matrices to model symbolic relationship

We describe a way of learning matrix representations of objects and relationships. The goal of learning is to allow multiplication of matrices to represent symbolic relationships between objects and symbolic relationships between relationships, which is the main novelty of the method. We demonstrate that this leads to excellent generalization in two different domains: modular arithmetic and family relationships. We show that the same system can learn first-order propositions such as (2, 5) ∈ +3 or (Christopher, Penelope) ∈ has_wife, and higher-order propositions such as (3, +3) ∈ plus and (+3, -3) ∈ inverse or (has_husband, has_wife) ∈ higher_oppsex. We further demonstrate that the system understands how higher-order propositions are related to first-order ones by showing that it can correctly answer questions about first-order propositions involving the relations +3 or has_wife even though it has not been trained on any first-order examples involving these relations.

[1]  Leonidas A A Doumas,et al.  A theory of the discovery and predication of relational concepts. , 2008, Psychological review.

[2]  Geoffrey E. Hinton,et al.  A general framework for parallel distributed processing , 1986 .

[3]  Geoffrey E. Hinton.  Learning distributed representations of concepts. , 1989 .

[4]  Thomas G. Dietterich,et al.  Readings in Machine Learning , 1991 .

[5]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[6]  K. Holyoak,et al.  A symbolic-connectionist theory of relational inference and generalization. , 2003, Psychological review.

[7]  Geoffrey E. Hinton,et al.  Learning Distributed Representations of Concepts Using Linear Relational Embedding , 2001, IEEE Trans. Knowl. Data Eng..

[8]  Tom M. Mitchell,et al.  The Need for Biases in Learning Generalizations , 2007 .

[9]  Geoffrey E. Hinton,et al.  Learning Distributed Representations of Relational Data using Linear Relational Embedding , 2001, WIRN.

[10]  Rajesh P. N. Rao,et al.  Development of localized oriented receptive fields by learning a translation-invariant code for natural images. , 1998, Network.

[11]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[12]  Geoffrey E. Hinton,et al.  Unsupervised Learning of Image Transformations , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Joshua B. Tenenbaum,et al.  Separating Style and Content with Bilinear Models , 2000, Neural Computation.