Chaitin-Kolmogorov Complexity and Generalization in Neural Networks
暂无分享,去创建一个
We present a unified framework for a number of different ways of failing to generalize properly. During learning, sources of random information contaminate the network, effectively augmenting the training data with random information. The complexity of the function computed is therefore increased, and generalization is degraded. We analyze replicated networks, in which a number of identical networks are independently trained on the same data and their results averaged. We conclude that replication almost always results in a decrease in the expected complexity of the network, and that replication therefore increases expected generalization. Simulations confirming the effect are also presented.
[1] Leslie G. Valiant,et al. A theory of the learnable , 1984, STOC '84.
[2] Alexander H. Waibel,et al. A novel objective function for improved phoneme recognition using time delay neural networks , 1990, International 1989 Joint Conference on Neural Networks.
[3] Nathan Intrator,et al. A Neural Network for Feature Extraction , 1989, NIPS.
[4] Josef Skrzypek,et al. Synergy of Clustering Multiple Back Propagation Networks , 1989, NIPS.