Visualizing non-metric similarities in multiple maps

Techniques for multidimensional scaling visualize objects as points in a low-dimensional metric map. As a result, the visualizations are subject to the fundamental limitations of metric spaces. These limitations prevent multidimensional scaling from faithfully representing non-metric similarity data such as word associations or event co-occurrences. In particular, multidimensional scaling cannot faithfully represent intransitive pairwise similarities in a visualization, and it cannot faithfully visualize “central” objects. In this paper, we present an extension of a recently proposed multidimensional scaling technique called t-SNE. The extension aims to address the problems of traditional multidimensional scaling techniques when these techniques are used to visualize non-metric similarities. The new technique, called multiple maps t-SNE, alleviates these problems by constructing a collection of maps that reveal complementary structure in the similarity data. We apply multiple maps t-SNE to a large data set of word association data and to a data set of NIPS co-authorships, demonstrating its ability to successfully visualize non-metric similarities.

[1]  W. Torgerson Multidimensional scaling: I. Theory and method , 1952 .

[2]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[3]  J. W. Hutchinson,et al.  Nearest neighbor analysis of psychological spaces. , 1986 .

[4]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[5]  P. Groenen,et al.  Modern multidimensional scaling , 1996 .

[6]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[7]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[8]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[9]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[10]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[11]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[12]  Geoffrey E. Hinton,et al.  Stochastic Neighbor Embedding , 2002, NIPS.

[13]  M. Tyers,et al.  Osprey: a network visualization system , 2003, Genome Biology.

[14]  Thomas L. Griffiths,et al.  Hierarchical Topic Models and the Nested Chinese Restaurant Process , 2003, NIPS.

[15]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[16]  Gal Chechik,et al.  Euclidean Embedding of Co-occurrence Data , 2004, J. Mach. Learn. Res..

[17]  Catherine Plaisant,et al.  The challenge of information visualization evaluation , 2004, AVI.

[18]  Thomas L. Griffiths,et al.  The Author-Topic Model for Authors and Documents , 2004, UAI.

[19]  Yiming Yang,et al.  The Enron Corpus: A New Dataset for Email Classi(cid:12)cation Research , 2004 .

[20]  Thomas A. Schreiber,et al.  The University of South Florida free association, rhyme, and word fragment norms , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[21]  Klaus-Robert Müller,et al.  Feature Discovery in Non-Metric Pairwise Data , 2004, J. Mach. Learn. Res..

[22]  Amos Storkey,et al.  Advances in Neural Information Processing Systems 20 , 2007 .

[23]  Joydeep Ghosh,et al.  Model-based overlapping clustering , 2005, KDD '05.

[24]  Joshua B. Tenenbaum,et al.  The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth , 2001, Cogn. Sci..

[25]  Andrew McCallum,et al.  The author-recipient-topic model for topic and role discovery in social networks , 2005 .

[26]  Kilian Q. Weinberger,et al.  Nonlinear Dimensionality Reduction by Semidefinite Programming and Kernel Matrix Factorization , 2005, AISTATS.

[27]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[28]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[29]  Ann B. Lee,et al.  Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Klaus-Robert Müller,et al.  Inducing Metric Violations in Human Similarity Judgements , 2006, NIPS.

[31]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[32]  Sanjoy Dasgupta,et al.  Robust Euclidean embedding , 2006, ICML.

[33]  Sebastian Schmidtlein,et al.  Mapping the floristic continuum : Ordination space position estimated from imaging spectroscopy , 2007 .

[34]  Geoffrey E. Hinton,et al.  Visualizing Similarity Data with a Mixture of Maps , 2007, AISTATS.

[35]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[36]  Amir Globerson,et al.  Visualizing pairwise similarity via semidefinite programming , 2007, AISTATS.

[37]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[38]  Katherine A. Heller,et al.  A Nonparametric Bayesian Approach to Modeling Overlapping Clusters , 2007, AISTATS.

[39]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[40]  Michael I. Jordan,et al.  DiscLDA: Discriminative Learning for Dimensionality Reduction and Classification , 2008, NIPS.

[41]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[42]  A. Nobel,et al.  On the size and recovery of submatrices of ones in a random binary matrix , 2008 .

[43]  B. Schölkopf,et al.  Similarity, Kernels, and the Triangle Inequality , 2008 .

[44]  Geoffrey E. Hinton,et al.  A Scalable Hierarchical Distributed Language Model , 2008, NIPS.

[45]  Zenglin Xu,et al.  Heavy-Tailed Symmetric Stochastic Neighbor Embedding , 2009, NIPS.

[46]  Ulrike von Luxburg,et al.  Clustering Stability: An Overview , 2010, Found. Trends Mach. Learn..

[47]  Olivier Thonnard,et al.  An Experimental Study of Diversity with Off-the-Shelf AntiVirus Engines , 2009, 2009 Eighth IEEE International Symposium on Network Computing and Applications.

[48]  Laurens van der Maaten,et al.  Learning a Parametric Embedding by Preserving Local Structure , 2009, AISTATS.

[49]  Tony Jebara,et al.  Structure preserving embedding , 2009, ICML '09.

[50]  Marc Dacier,et al.  Addressing the attack attribution problem using knowledge discovery and multi-criteria fuzzy decision-making , 2009, CSI-KDD '09.

[51]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[52]  Eric O. Postma,et al.  Texton-based analysis of paintings , 2010, Optical Engineering + Applications.

[53]  Jarkko Venna,et al.  Information Retrieval Perspective to Nonlinear Dimensionality Reduction for Data Visualization , 2010, J. Mach. Learn. Res..

[54]  Miguel Á. Carreira-Perpiñán,et al.  The Elastic Embedding Algorithm for Dimensionality Reduction , 2010, ICML.

[55]  Daniel A. Keim,et al.  Mastering the Information Age - Solving Problems with Visual Analytics , 2010 .

[56]  Andrew R. Jamieson,et al.  Exploring nonlinear feature space dimension reduction and data representation in breast Cadx with Laplacian eigenmaps and t-SNE. , 2009, Medical physics.

[57]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[58]  Krishnakumar Balasubramanian,et al.  Dimensionality Reduction for Text using Domain Knowledge , 2010, COLING.

[59]  Jeffrey Heer,et al.  SpanningAspectRatioBank Easing FunctionS ArrayIn ColorIn Date Interpolator MatrixInterpola NumObjecPointI Rectang ISchedu Parallel Pause Scheduler Sequen Transition Transitioner Transiti Tween Co DelimGraphMLCon IData JSONCon DataField DataSc Dat DataSource Data DataUtil DirtySprite LineS RectSprite , 2011 .

[60]  Neil D. Lawrence,et al.  Spectral Dimensionality Reduction via Maximum Entropy , 2011, AISTATS.

[61]  Jeffrey Heer,et al.  D³ Data-Driven Documents , 2011, IEEE Transactions on Visualization and Computer Graphics.

[62]  Michael C. Hout,et al.  Multidimensional Scaling , 2003, Encyclopedic Dictionary of Archaeology.