论文信息 - Discovering Multiple Constraints that are Frequently Approximately Satisfied

Discovering Multiple Constraints that are Frequently Approximately Satisfied

Some high-dimensional datasets can be modelled by assuming that there are many different linear constraints, each of which is Frequently Approximately Satisfied (FAS) by the data. The probability of a data vector under the model is then proportional to the product of the probabilities of its constraint violations. We describe three methods of learning products of constraints using a heavy-tailed probability distribution for the violations.

Yee Whye Teh | Geoffrey E. Hinton | Y. Teh

[1] J. Besag. Statistical Analysis of Non-Lattice Data , 1975 .

[2] J J Hopfield,et al. Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[3] Paul Smolensky,et al. Information processing in dynamical systems: foundations of harmony theory , 1986 .

[4] Geoffrey E. Hinton,et al. Learning and relearning in Boltzmann machines , 1986 .

[5] David Haussler,et al. Unsupervised learning of distributions on binary vectors using two layer networks , 1991, NIPS 1991.

[6] Yoav Freund,et al. A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[7] Terrence J. Sejnowski,et al. Edges are the Independent Components of Natural Scenes , 1996, NIPS.

[8] David J. Field,et al. Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[9] Yoav Freund,et al. A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[10] Geoffrey E. Hinton. Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.