A common neural-network model for unsupervised exploratory data analysis and independent component analysis

This paper presents the derivation of an unsupervised learning algorithm, which enables the identification and visualization of latent structure within ensembles of high-dimensional data. This provides a linear projection of the data onto a lower dimensional subspace to identify the characteristic structure of the observations independent latent causes. The algorithm is shown to be a very promising tool for unsupervised exploratory data analysis and data visualization. Experimental results confirm the attractiveness of this technique for exploratory data analysis and an empirical comparison is made with the recently proposed generative topographic mapping (GTM) and standard principal component analysis (PCA). Based on standard probability density models a generic nonlinearity is developed which allows both 1) identification and visualization of dichotomised clusters inherent in the observed data and 2) separation of sources with arbitrary distributions from mixtures, whose dimensionality may be greater than that of number of sources. The resulting algorithm is therefore also a generalized neural approach to independent component analysis (ICA) and it is considered to be a promising method for analysis of real-world data that will consist of sub- and super-Gaussian components such as biomedical signals.

[1]  K. Pearson.  Contributions to the Mathematical Theory of Evolution , 1894 .

[2]  Brian Everitt,et al.  Cluster analysis , 1974 .

[3]  Brian Everitt,et al.  An Introduction to Latent Variable Models , 1984 .

[4]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[5]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .

[6]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[7]  Andrzej Cichocki,et al.  Robust learning algorithm for blind separation of signals , 1994 .

[8]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[9]  S. Klinke,et al.  Exploratory Projection Pursuit , 1995 .

[10]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[11]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[12]  Andrzej Cichocki,et al.  Robust neural networks with on-line learning for blind identification and blind separation of sources , 1996 .

[13]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[14]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[15]  Andrzej Cichocki,et al.  Stability Analysis of Learning Algorithms for Blind Source Separation , 1997, Neural Networks.

[16]  Mark Girolami,et al.  Extraction of independent signal sources using a deflationary exploratory projection pursuit network , 1997 .

[17]  S. Amari,et al.  Multichannel blind separation and deconvolution of sources with arbitrary distributions , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[18]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[19]  Allan Kardec Barros,et al.  A fixed-point algorithm for independent component analysis which uses a priori information , 1998, Proceedings 5th Brazilian Symposium on Neural Networks (Cat. No.98EX209).

[20]  Christopher M. Bishop,et al.  A Hierarchical Latent Variable Model for Data Visualization , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Mark A. Girolami,et al.  An Alternative Perspective on Adaptive Independent Component Analysis Algorithms , 1998, Neural Computation.