Abstract : The Cascade-Correlation learning algorithm constructs a multi-layer artificial neural network as it learns to perform a given task. The resulting network's size and topology are chosen specifically for this task. In the resulting 'cascade' networks, each new hidden unit receives incoming connections from all input and pre-existing hidden units. In effect, each new unit adds a new layer to the network. This allows Cascade-Correlation to create complex feature detectors, but it typically results in a network that is deeper, in terms of the longest path from input to output, than is necessary to solve the problem efficiently. In this paper we investigate a simple variation of Cascade-Correlation that will build deep nets if necessary, but that is biased toward minimizing network depth. We demonstrate empirically, across a range of problems, that this simple technique can reduce network depth, often dramatically. However, we show that this technique does not, in general, reduce the total number of weights or improve the generalization ability of the resulting networks.
[1]
Kurt Hornik,et al.
Multilayer feedforward networks are universal approximators
,
1989,
Neural Networks.
[2]
R. Palmer,et al.
Introduction to the theory of neural computation
,
1994,
The advanced book program.
[3]
Terrence J. Sejnowski,et al.
Analysis of hidden units in a layered network trained to classify sonar targets
,
1988,
Neural Networks.
[4]
Dhananjay S. Phatak,et al.
Connectivity and performance tradeoffs in the cascade correlation learning architecture
,
1994,
IEEE Trans. Neural Networks.
[5]
Anders Krogh,et al.
Introduction to the theory of neural computation
,
1994,
The advanced book program.
[6]
Christian Lebiere,et al.
The Cascade-Correlation Learning Architecture
,
1989,
NIPS.
[7]
Sam Waugh,et al.
Connection Strategies in Cascade Correlation
,
1994
.