We consider the problem of determining whether certain input patterns share any on-trivial abstract features. We present a neural system that invents non-trivial distributed representations of input patterns such that patterns having something in common are represented by the same distinct and informative output pattern. The system consists of two modules that learn in an unsupervised manner (no teacher provides target outputs): At a given time, each module sees an input pattern. Each module tries to represent different input patterns by different output patterns, but at the same time both modules try to emit output patterns that match. This procedure tends to create informative representations of non-trivial abstract features shared by both patterns. The approach can be related to the IMAX method of Hinton, Becker and Zemel (1989, 1991). Experiments include a stereo task proposed by Becker and Hinton, which can be solved more easily by our system.
[1]
Jürgen Schmidhuber,et al.
A novel unsupervised classification method
,
1993
.
[2]
Jürgen Schmidhuber,et al.
Discovering Predictable Classifications
,
1993,
Neural Computation.
[3]
H. B. Barlow,et al.
Finding Minimum Entropy Codes
,
1989,
Neural Computation.
[4]
Geoffrey E. Hinton,et al.
Learning internal representations by error propagation
,
1986
.
[5]
Jürgen Schmidhuber,et al.
Learning Factorial Codes by Predictability Minimization
,
1992,
Neural Computation.
[6]
Geoffrey E. Hinton,et al.
Discovering Viewpoint-Invariant Relationships That Characterize Objects
,
1990,
NIPS.
[7]
P. Werbos,et al.
Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences
,
1974
.