The "wake-sleep" algorithm for unsupervised neural networks.

An unsupervised learning algorithm for a multilayer network of stochastic neurons is described. Bottom-up "recognition" connections convert the input into representations in successive hidden layers, and top-down "generative" connections reconstruct the representation in one layer from the representation in the layer above. In the "wake" phase, neurons are driven by recognition connections, and generative connections are adapted to increase the probability that they would reconstruct the correct activity vector in the layer below. In the "sleep" phase, neurons are driven by generative connections, and recognition connections are adapted to increase the probability that they would produce the correct activity vector in the layer above.

[1]  Ulf Grenander,et al.  Lectures in pattern theory , 1978 .

[2]  H. J. Gamble.  Trends in Neurosciences , 1980 .

[3]  G. Miller,et al.  Cognitive science. , 1981, Science.

[4]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[5]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[6]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[7]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[8]  M. Hasselmo,et al.  Acetylcholine and memory , 1993, Trends in Neurosciences.