Performance analysis of the FastICA algorithm and Crame/spl acute/r-rao bounds for linear independent component analysis

The FastICA or fixed-point algorithm is one of the most successful algorithms for linear independent component analysis (ICA) in terms of accuracy and computational complexity. Two versions of the algorithm are available in literature and software: a one-unit (deflation) algorithm and a symmetric algorithm. The main result of this paper are analytic closed-form expressions that characterize the separating ability of both versions of the algorithm in a local sense, assuming a "good" initialization of the algorithms and long data records. Based on the analysis, it is possible to combine the advantages of the symmetric and one-unit version algorithms and predict their performance. To validate the analysis, a simple check of saddle points of the cost function is proposed that allows to find a global minimum of the cost function in almost 100% simulation runs. Second, the Crame/spl acute/r-Rao lower bound for linear ICA is derived as an algorithm independent limit of the achievable separation quality. The FastICA algorithm is shown to approach this limit in certain scenarios. Extensive computer simulations supporting the theoretical findings are included.

[1]  Boaz Porat,et al.  Digital Processing of Random Signals: Theory and Methods , 2008 .

[2]  Erkki Oja,et al.  An Experimental Comparison of Neural Algorithms for Independent Component Analysis and Blind Separation , 1999, Int. J. Neural Syst..

[3]  Erkki Oja,et al.  Cramer-Rao lower bound for linear independent component analysis , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[4]  Zbynek Koldovský,et al.  Optimal pairing of signal components separated by blind techniques , 2004, IEEE Signal Processing Letters.

[5]  Arie Yeredor,et al.  A blind-ML scheme for blind source separation , 2004, IEEE Workshop on Statistical Signal Processing, 2003.

[6]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..

[7]  A. Hyvärinen,et al.  One-unit contrast functions for independent component analysis: a statistical analysis , 1997 .

[8]  Erkki Oja,et al.  Efficient Variant of Algorithm FastICA for Independent Component Analysis Attaining the CramÉr-Rao Lower Bound , 2006, IEEE Transactions on Neural Networks.

[9]  N. L. Johnson,et al.  Linear Statistical Inference and Its Applications , 1966 .

[10]  Yehoshua Y. Zeevi,et al.  QML Blind Deconvolution: Asymptotic Analysis , 2004, ICA.

[11]  Phillip A. Regalia,et al.  Monotonic convergence of fixed-point algorithms for ICA , 2003, IEEE Trans. Neural Networks.

[12]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis of Complex Valued Signals , 2000, Int. J. Neural Syst..

[13]  Constantinos B. Papadias,et al.  Globally convergent blind source separation based on a multiuser kurtosis maximization criterion , 2000, IEEE Trans. Signal Process..

[14]  Daniel Yellin,et al.  Multichannel system identification and deconvolution: performance bounds , 1999, IEEE Trans. Signal Process..

[15]  Vwani P. Roychowdhury,et al.  Independent component analysis based on nonparametric density estimation , 2004, IEEE Transactions on Neural Networks.

[16]  Jitendra K. Tugnait,et al.  Comments on 'New criteria for blind deconvolution of nonminimum phase systems (channels)' , 1992, IEEE Trans. Inf. Theory.

[17]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[18]  P. Comon,et al.  Contrasts for multichannel blind deconvolution , 1996, IEEE Signal Processing Letters.

[19]  Ehud Weinstein,et al.  Maximum likelihood and lower bounds in system identification with non-Gaussian inputs , 1994, IEEE Trans. Inf. Theory.

[20]  E. Oja,et al.  Convergence of the symmetrical FastICA algorithm , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[21]  Changyuan Fan,et al.  On the Convergence Behavior of the FastICA Algorithm with the Kurtosis Cost Function , 2007, Third International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP 2007).

[22]  J. Cardoso.  On the Performance of Orthogonal Source Separation Algorithms , 1994 .

[23]  T. Ens,et al.  Blind signal separation : statistical principles , 1998 .

[24]  M. Kupperman.  Linear Statistical Inference and Its Applications 2nd Edition (C. Radhakrishna Rao) , 1975 .

[25]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[26]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .

[27]  John W. Fisher,et al.  ICA Using Spacings Estimates of Entropy , 2003, J. Mach. Learn. Res..

[28]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[29]  P. Tichavský,et al.  Efficient variant of algorithm fastica for independent component analysis attaining the cramer-RAO lower bound , 2005, IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.

[30]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..

[31]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[32]  Fathi M. A. Salam,et al.  Blind source recovery using an adaptive generalized Gaussian score function , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[33]  Christian Jutten,et al.  Fisher Information in Source Separation Problems , 2004, ICA.

[34]  D. Donoho.  ON MINIMUM ENTROPY DECONVOLUTION , 1981 .