Low complexity adaptive algorithms for Principal and Minor Component Analysis

This article introduces new low cost algorithms for the adaptive estimation and tracking of principal and minor components. The proposed algorithms are based on the well-known OPAST method which is adapted and extended in order to achieve the desired MCA or PCA (Minor or Principal Component Analysis). For the PCA case, we propose efficient solutions using Givens rotations to estimate the principal components out of the weight matrix given by OPAST method. These solutions are then extended to the MCA case by using a transformed data covariance matrix in such a way the desired minor components are obtained from the PCA of the new (transformed) matrix. Finally, as a byproduct of our PCA algorithm, we propose a fast adaptive algorithm for data whitening that is shown to overcome the recently proposed RLS-based whitening method.

[1]  Joos Vandewalle,et al.  Updating Singular Value Decompositions. A Parallel Implementation. , 1989, Optics & Photonics.

[2]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..

[3]  Liping Sun,et al.  Orthonormal Subspace Tracking Algorithm for Space–Time Multiuser Detection in Multipath CDMA Channels , 2007, IEEE Transactions on Vehicular Technology.

[4]  Jean Pierre Delmas,et al.  Subspace Tracking for Signal Processing , 2010 .

[5]  Rong Wang,et al.  Stable and Orthonormal OJA Algorithm With Low Complexity , 2011, IEEE Signal Processing Letters.

[6]  George V. Moustakides,et al.  Fast and Stable Subspace Tracking , 2008, IEEE Transactions on Signal Processing.

[7]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[8]  Hideaki Sakai,et al.  A new adaptive algorithm for minor component analysis , 1998, Signal Process..

[9]  Y. Hua,et al.  Fast orthonormal PAST algorithm , 2000, IEEE Signal Processing Letters.

[10]  Zheng Bao,et al.  Fast principal component extraction by a weighted information criterion , 2002, IEEE Trans. Signal Process..

[11]  Nishan Canagarajah,et al.  Improving the direction-of-arrival resolution via double code filtering in WCDMA , 2000 .

[12]  Weidong Jiao,et al.  DOA Estimation of Multiple Convolutively Mixed Sources Based on Principle Component Analysis , 2009, ICONIP.

[13]  William H. Press,et al.  Numerical Recipes 3rd Edition: The Art of Scientific Computing , 2007 .

[14]  Mao Ye,et al.  A Class of Self-Stabilizing MCA Learning Algorithms , 2006, IEEE Transactions on Neural Networks.

[15]  Karim Abed-Meraim,et al.  Low-cost adaptive algorithm for noise subspace estimation , 2002 .

[17]  Karim Abed-Meraim,et al.  Fast principal component analysis and data whitening algorithms , 2011, International Workshop on Systems, Signal Processing and their Applications, WOSSPA.

[18]  Simon Haykin,et al.  Adaptive Signal Processing: Next Generation Solutions , 2010 .

[19]  Roland Badeau,et al.  Yast Algorithm for Minor Subspace Tracking , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[20]  Zhang Yi,et al.  A Modified Oja–Xu MCA Learning Algorithm and Its Convergence Analysis , 2007, IEEE Transactions on Circuits and Systems II: Express Briefs.

[21]  Rong Wang,et al.  A Novel Orthonormalization Matrix Based Fast and Stable DPM Algorithm for Principal and Minor Subspace Tracking , 2012, IEEE Transactions on Signal Processing.

[22]  Gene H. Golub,et al.  Matrix computations , 1983 .

[23]  Xianda Zhang,et al.  Adaptive RLS Implementation of Non-negative PCA Algorithm for Blind Source Separation , 2004, ISNN.

[24]  Zhang Xianda,et al.  Natural gradient-based recursive least-squares algorithm for adaptive blind source separation , 2004 .

[25]  Vwani P. Roychowdhury,et al.  Algorithms for accelerated convergence of adaptive PCA , 2000, IEEE Trans. Neural Networks Learn. Syst..

[26]  Samir Attallah,et al.  The generalized Rayleigh's quotient adaptive noise subspace algorithm: a householder transformation-based implementation , 2006, IEEE Transactions on Circuits and Systems II: Express Briefs.

[27]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[28]  Mu Zhou,et al.  Stable Subspace Tracking Algorithm Based on a Signed URV Decomposition , 2012, IEEE Transactions on Signal Processing.

[29]  V. Solo,et al.  High Dimensional Point Process System Identification: PCA and Dynamic Index Models , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[30]  Jar-Ferr Yang,et al.  Adaptive eigensubspace algorithms for direction or frequency estimation and tracking , 1988, IEEE Trans. Acoust. Speech Signal Process..

[31]  Steve Bartelmaos,et al.  Fast Principal Component Extraction Using Givens Rotations , 2008, IEEE Signal Processing Letters.

[32]  Karim Abed-Meraim,et al.  On a Class of Orthonormal Algorithms for Principal and Minor Subspace Tracking , 2002, J. VLSI Signal Process..

[33]  J. Príncipe,et al.  A fast, on-line algorithm for PCA and its convergence characteristics , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[34]  Benoît Champagne,et al.  Subspace tracking of fast time-varying channels in precoded MIMO-OFDM systems , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[35]  M. Barkat,et al.  PAST and OPAST algorithms for STAP in monostatic airborne radar , 2011, 2011 International Symposium on Innovations in Intelligent Systems and Applications.

[36]  Imran Ali,et al.  A New Subspace Tracking Algorithm Using Approximation of Gram-Schmidt Procedure , 2009, 2009 International Conference on Information and Multimedia Technology.

[37]  Masashi Hayakawa,et al.  Principal component analysis and singular spectrum analysis of ULF geomagnetic data associated with earthquakes , 2005 .

[38]  H. Zhou,et al.  Robust multiuser detection using Kalman filter and windowed projection approximation subspace tracking algorithm , 2011, IET Commun..

[39]  Yoram Bresler,et al.  On the number of signals resolvable by a uniform linear array , 1986, IEEE Trans. Acoust. Speech Signal Process..

[40]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[41]  Steve Bartelmaos,et al.  Fast adaptive algorithms for minor component analysis using Householder transformation , 2011, Digit. Signal Process..

[42]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[43]  G. Golub,et al.  Tracking a few extreme singular values and vectors in signal processing , 1990, Proc. IEEE.