A Hierarchical Latent Variable Model for Data Visualization

Visualization has proven to be a powerful and widely-applicable tool for the analysis and interpretation of multivariate data. Most visualization algorithms aim to find a projection from the data space down to a two-dimensional visualization space. However, for complex data sets living in a high-dimensional space, it is unlikely that a single two-dimensional projection can reveal all of the interesting structure. We therefore introduce a hierarchical visualization algorithm which allows the complete data set to be visualized at the top level, with clusters and subclusters of data points visualized at deeper levels. The algorithm is based on a hierarchical mixture of latent variable models, whose parameters are estimated using the expectation-maximization algorithm. We demonstrate the principle of the approach on a toy data set, and we then apply the algorithm to the visualization of a synthetic data set in 12 dimensions obtained from a simulation of multiphase flows in oil pipelines, and to data in 36 dimensions derived from satellite images.

[1]  James E. Dammann,et al.  A Technique for Determining and Coding Subclasses in Pattern Recognition Problems , 1965, IBM J. Res. Dev..

[2]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[3]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[4]  Dorothy T. Thayer,et al.  EM algorithms for ML factor analysis , 1982 .

[5]  Brian Everitt,et al.  An Introduction to Latent Variable Models , 1984 .

[6]  B. Everitt,et al.  An Introduction to Latent Variable Models , 1984 .

[7]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[8]  Risto Miikkulainen,et al.  Script Recognition with Hierarchical Feature Maps , 1992 .

[9]  John A. Nelder,et al.  Generalized linear models. 2nd ed. , 1993 .

[10]  C. Bishop,et al.  Analysis of multiphase flows using dual-energy gamma densitometry and neural networks , 1993 .

[11]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[12]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[13]  Michael I. Jordan,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.

[14]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[15]  Yeuvo Jphonen,et al.  Self-Organizing Maps , 1995 .

[16]  Andreas Buja,et al.  Interactive High-Dimensional Data Visualization , 1996 .

[17]  Luca Maria Gambardella,et al.  Learing Fine Motion by Using the Hierarchical Extended Kohonen Map , 1996, ICANN.

[18]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[19]  Michael E. Tipping,et al.  Mixtures of Principal Component Analysers , 1997 .

[20]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[21]  R. Shanmugam.  Multivariate Analysis: Part 2: Classification, Covariance Structures and Repeated Measurements , 1998 .

[22]  Ayoub Ghriss,et al.  Mixtures of Probabilistic Principal Component Analysers , 2018 .

[23]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.