Variational Learning of Clusters of Undercomplete Nonsymmetric Independent Components

We apply a variational method to automatically determine the number of mixtures of independent components in high-dimensional datasets, in which the sources may be nonsymmetrically distributed. The data are modeled by clusters where each cluster is described as a linear mixture of independent factors. The variational Bayesian method yields an accurate density model for the observed data without overfitting problems. This allows the dimensionality of the data to be identified for each cluster. The new method was successfully applied to a difficult real-world medical dataset for diagnosing glaucoma.

[1]  R. Hitchings,et al.  The optic disc in glaucoma II: correlation of the appearance of the optic disc with the visual field. , 1977, The British journal of ophthalmology.

[2]  E. T. Jaynes,et al.  Papers on probability, statistics and statistical physics , 1983 .

[3]  B. Ripley,et al.  E. T. Jaynes: Papers on Probability, Statistics and Statistical Physics , 1983 .

[4]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[5]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[6]  D. Goss.  The Ocular Examination: Measurements and Findings , 1997 .

[7]  Eric Moulines,et al.  Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[8]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[9]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[10]  Harri Lappalainen,et al.  Ensemble learning for independent component analysis , 1999 .

[11]  Hagai Attias,et al.  Inferring Parameters and Structure of Latent Variable Models by Variational Bayes , 1999, UAI.

[12]  Zoubin Ghahramani,et al.  Variational Inference for Bayesian Mixtures of Factor Analysers , 1999, NIPS.

[13]  Shun-ichi Amari,et al.  Natural Gradient Learning for Over- and Under-Complete Bases in ICA , 1999, Neural Computation.

[14]  Visa Koivunen,et al.  Maximum likelihood estimation of ICA model for wide class of source distributions , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[15]  Sejnowski,et al.  ICA MIXTURE MODELS FOR UNSUPERVISED CLASSIFICATION AND AUTOMATIC CONTEXT SWITCHING , 2000 .

[16]  Max Welling,et al.  A Constrained EM Algorithm for Independent Component Analysis , 2001, Neural Computation.

[17]  E. Oja,et al.  Independent Component Analysis , 2013 .

[18]  Tzyy-Ping Jung,et al.  Imaging brain dynamics using independent component analysis , 2001, Proc. IEEE.

[19]  Ole Winther,et al.  Mean-Field Approaches to Independent Component Analysis , 2002, Neural Computation.

[20]  Terrence J. Sejnowski,et al.  Comparison of machine learning and traditional classifiers in glaucoma diagnosis , 2002, IEEE Transactions on Biomedical Engineering.

[21]  Stephen J. Roberts,et al.  Variational Mixture of Bayesian Independent Component Analyzers , 2003, Neural Computation.