Recursive dynamic node creation in multilayer neural networks

The derivations of a novel approach for simultaneous recursive weight adaptation and node creation in multilayer backpropagation neural networks are presented. The method uses time and order update formulations in the orthogonal projection method to derive a recursive weight updating procedure for the training process of the neural network and a recursive node creation algorithm for weight adjustment of a layer with added nodes during the training process. The proposed approach allows optimal dynamic node creation in the sense that the mean-squared error is minimized for each new topology. The effectiveness of the algorithm is demonstrated on several benchmark problems (the multiplexer and the decoder problems) as well as a real world application for detection and classification of buried dielectric anomalies using a microwave sensor.