A Class of Cost Functions for Independence

This paper addresses the problem of transforming a set of patterns into new patterns whose components are mutually statistically independent. This is a problem that, depending on the specific setting, is often designated as factorial coding, independent components analysis, source separation or nonlinear principal components analysis. The reasons for the growing interest on this problem are briefly examined. Some of the most important methods that have been proposed to solve the problem are overviewed, A new class of objective functions for solving the problem is presented. These new objective functions have the advantage of being continuous and differentiable even when they are computed from the empirical distribution of the training data. Some examples of the use of these objective functions are given.

[1]  L. B. Almeida,et al.  An Objective Function for Independence , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[2]  J. Karhunen,et al.  A bigradient optimization approach for robust PCA, MCA, and source separation , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[3]  A. Norman Redlich,et al.  Supervised Factorial Learning , 1993, Neural Computation.

[4]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[5]  Gustavo Deco,et al.  Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures , 1995, Neural Networks.

[6]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[7]  Jürgen Schmidhuber,et al.  Learning Factorial Codes by Predictability Minimization , 1992, Neural Computation.

[8]  Juha Karhunen,et al.  Nonlinear PCA type approaches for source separation and independent component analysis , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[9]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[10]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[11]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[12]  Geoffrey E. Hinton,et al.  Autoencoders, Minimum Description Length and Helmholtz Free Energy , 1993, NIPS.

[13]  P. Comon.  Independent Component Analysis , 1992 .