Analysis of Feasible Solutions of the ICA Problem Under the One-Bit-Matching Condition

The one-bit-matching conjecture for independent component analysis (ICA) has been widely believed in the ICA community. Theoretically, it has been proved that under certain regular assumptions, the global maximum of a simplified objective function derived from the maximum likelihood or minimum mutual information criterion under the one-bit-matching condition corresponds to a feasible solution of the ICA problem, and also that all the local maxima of the objective function correspond to the feasible solutions of the ICA problem in the two-source square mixing setting. This paper further studies the one-bit-matching conjecture along this direction, and we prove that under the one-bit-matching condition there always exist many local maxima of the objective function that correspond to the stable feasible solutions of the ICA problem in the general case; moreover, in ceratin cases there also exist some local minima of the objective function that correspond to the stable feasible solutions of the ICA problem with mixed super- and sub-Gaussian sources.

[1]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[2]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[3]  Shun-ichi Amari,et al.  Independent component analysis by the information-theoretic approach with mixture of densities , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[4]  Lei Xu,et al.  One-Bit-Matching Conjecture for Independent Component Analysis , 2004, Neural Computation.

[5]  Lei Xu,et al.  Some global and local convergence analysis on the information-theoretic independent component analysis approach , 2000, Neurocomputing.

[6]  Mark A. Girolami,et al.  An Alternative Perspective on Adaptive Independent Component Analysis Algorithms , 1998, Neural Computation.

[7]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..

[8]  Andrzej Cichocki,et al.  New learning algorithm for blind separation of sources , 1992 .

[9]  Lang Tong,et al.  Waveform-preserving blind estimation of multiple independent sources , 1993, IEEE Trans. Signal Process..

[10]  Max Welling,et al.  A Constrained EM Algorithm for Independent Component Analysis , 2001, Neural Computation.

[11]  David Mautner Himmelblau,et al.  Applied Nonlinear Programming , 1972 .

[12]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[13]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[14]  Jinwen Ma,et al.  A Further Result on the ICA One-Bit-Matching Conjecture , 2005, Neural Computation.

[15]  Richard M. Everson,et al.  Independent Component Analysis: A Flexible Nonlinearity and Decorrelating Manifold Approach , 1999, Neural Computation.

[16]  Shun-ichi Amari,et al.  Learned parametric mixture based ICA algorithm , 1998, Neurocomputing.

[17]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[18]  Jinwen Ma,et al.  An alternative switching criterion for independent component analysis (ICA) , 2005, Neurocomputing.