论文信息 - Deep belief networks

Deep belief networks

The important aspect of this layer-wise training procedure is that, provided the number of features per layer does not decrease, [6] showed that each extra layer increases a variational lower bound on the log probability of data. So layer-by-layer training can be repeated several times1 to learn a deep, hierarchical model in which each layer of features captures strong high-order correlations between the activities of features in the layer below. We will discuss three ideas based on greedily learning a hierarchy of features:

Geoffrey E. Hinton

[1] Nicole Immorlica,et al. Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[2] A. Gámez,et al. Nonlinear dimensionality reduction in climate data , 2004 .

[3] Geoffrey E. Hinton,et al. Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure , 2007, AISTATS.

[4] Geoffrey E. Hinton,et al. Non-linear dimensionality reduction using neural networks , 2006 .

[5] Bernhard Schölkopf,et al. Training Invariant Support Vector Machines , 2002, Machine Learning.

[6] R Hecht-Nielsen,et al. Replicator neural networks for universal optimal source coding. , 1995, Science.

[7] Yee Whye Teh,et al. A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.