A Fast Fixed-Point Algorithm for Independent Component Analysis

We introduce a novel fast algorithm for independent component analysis, which can be used for blind source separation and feature extraction. We show how a neural network learning rule can be transformed into a fixedpoint iteration, which provides an algorithm that is very simple, does not depend on any user-defined parameters, and is fast to converge to the most accurate solution allowed by the data. The algorithm finds, one at a time, all nongaussian independent components, regardless of their probability distributions. The computations can be performed in either batch mode or a semiadaptive manner. The convergence of the algorithm is rigorously proved, and the convergence speed is shown to be cubic. Some comparisons to gradient-based algorithms are made, showing that the new algorithm is usually 10 to 100 times faster, sometimes giving the solution in just a few iterations.

[1]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[2]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[3]  Jean-Francois Cardoso,et al.  Eigen-structure of the fourth-order cumulant tensor with application to the blind source separation problem , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[4]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[5]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[6]  Charles W. Therrien,et al.  Discrete Random Signals and Statistical Signal Processing , 1992 .

[7]  Jean-Francois Cardoso,et al.  ITERATIVE TECHNIQUES FOR BLIND SOURCE SEPARATION USING ONLY FOURTH-ORDER CUMULANTS , 1992 .

[8]  Ehud Weinstein,et al.  Super-exponential methods for blind deconvolution , 1993, IEEE Trans. Inf. Theory.

[9]  Eric Moreau,et al.  New self-adaptative algorithms for source separation based on contrast functions , 1993, [1993 Proceedings] IEEE Signal Processing Workshop on Higher-Order Statistics.

[10]  Ah Chung Tsoi,et al.  Blind deconvolution of signals using a complex recurrent network , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[11]  Schuster,et al.  Separation of a mixture of independent signals using time delayed correlations. , 1994, Physical review letters.

[12]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[13]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[14]  Marimuthu Palaniswami,et al.  Computational Intelligence: A Dynamic System Perspective , 1995 .

[15]  Erkki Oja,et al.  Signal Separation by Nonlinear Hebbian Learning , 1995 .

[16]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..

[17]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[18]  Erkki Oja,et al.  Image Feature Extraction Using Independent Component Analysis , 1996 .

[19]  Slava M. Katz.  Distribution of content words and phrases in text and language modelling , 1996, Natural Language Engineering.

[20]  Aapo Hyvärinen,et al.  A neuron that learns to separate one signal from a mixture of independent sources , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[21]  T J Sejnowski,et al.  Learning the higher-order structure of a natural sound. , 1996, Network.

[22]  Aapo Hyvrinen.  Independent Component Analysis by Minimization of Mutual Information Independent Component Analysis by Minimization of Mutual Information Independent Component Analysis by Minimization of Mutual Information , 1997 .

[23]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[24]  A. Hyvarinen.  A family of fixed-point algorithms for independent component analysis , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[25]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[26]  Petteri Pajunen,et al.  Blind source separation using algorithmic information theory , 1998, Neurocomputing.

[27]  Santosh S. Vempala,et al.  Latent semantic indexing: a probabilistic analysis , 1998, PODS '98.

[28]  Jean-François Cardoso,et al.  Multidimensional independent component analysis , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[29]  Erkki Oja,et al.  Independent component analysis by general nonlinear Hebbian-like learning rules , 1998, Signal Process..

[30]  Paul A. Viola,et al.  Restructuring Sparse High Dimensional Data for Effective Retrieval , 1998, NIPS.

[31]  P. Philips,et al.  JADETD : COMBINING HIGHER-ORDER STATISTICS AND TEMPORALINFORMATION FOR BLIND SOURCE SEPARATION ( WITH NOISE ) , 1999 .

[32]  Petteri Pajunen,et al.  Blind Source Separation Of Natural Signals Based On Approximate Complexity Minimization , 1999 .

[33]  Thomas Hofmann,et al.  Probabilistic Latent Semantic Analysis , 1999, UAI.

[34]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.