Adaptive learning algorithm for principal component analysis with partial data

In this paper a fast and ecient adaptive learning algorithm for estimation of the principal components is developed. It seems to be especially useful in applications with changing environment , where the learning process has to be repeated in on{line manner. The approach can be called the cascade recursive least square (CRLS) method, as it combines a cascade (hierarchical) neural network scheme for input signal reduction with the RLS (recursive least square) lter for adaptation of learning rates. Successful application of the CRLS method for 2{D image compression{reconstruction and its performance in comparison to other known PCA adaptive algorithms are also documented. 1 Introduction Principal Component Analysis (PCA) is a powerful data analysis tool in multivariate statistics [Jollie, 1986; Amari, 1977]. Up to now many neural network learning algorithms have been proposed for the PCA and its generalizations (for an overview see (the RLS method). In recent papers several new the-oretic developments in neural network based PCA have been described (e.g. The main purpose of this paper is to develop a reliable on{line adaptive learning algorithm for neural network. Its reliability means it converges for every signal of given class with well quality, and it precisely extracts an arbitrary number of principal components with high convergence speed. This fast learning is performed in one epoch of the input sample or even with partial data only, and it still can provide a better trade{o between learning speed and performance than the known adaptive algorithms. The application eld in mind for the proposed algorithm is image processing { image restoration, compression and classi-cation { if the observed scene is steadily changing (due to illumination change in case of a stationary camera or due to background or foreground change in case of an active camera). This assumption requires that the learning process is repeated in on{line manner. The paper is organized in ve sections. First a short introduction of the neural network based PCA is given and a discussion of previous and recent developments takes place. In section 3 we present a new method { the CRLS algorithm with adaptive learning rule. In section 4 test results for applying the above methods to single still images are summarized. Conclusions are drawn in the last section. 2 Neural Network Based PCA The standard PCA, called also Karhunen{Loeve transformation, (KL) determines an optimal linear transformation of an input vector x y = Wx (1) where …