Manifold Learning via Manifold Deflation

Nonlinear dimensionality reduction methods provide a valuable means to visualize and interpret high-dimensional data. However, many popular methods can fail dramatically, even on simple two-dimensional manifolds, due to problems such as vulnerability to noise, repeated eigendirections, holes in convex bodies, and boundary bias. We derive an embedding method for Riemannian manifolds that iteratively uses single-coordinate estimates to eliminate dimensions from an underlying differential operator, thus "deflating" it. These differential operators have been shown to characterize any local, spectral dimensionality reduction method. The key to our method is a novel, incremental tangent space estimator that incorporates global structure as coordinates are added. We prove its consistency when the coordinates converge to true coordinates. Empirically, we show our algorithm recovers novel and interesting embeddings on real-world and synthetic datasets.

[1]  Jianqing Fan,et al.  Local polynomial modelling and its applications , 1994 .

[2]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[3]  I. Hassan Embedded , 2005, The Cyber Security Handbook.

[4]  Andrew V. Knyazev,et al.  Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method , 2001, SIAM J. Sci. Comput..

[5]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[7]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[8]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[9]  J. Hannay,et al.  Fibonacci numerical integration on a sphere , 2004 .

[10]  Matthias Hein,et al.  Manifold Denoising , 2006, NIPS.

[11]  Haifeng Chen,et al.  Robust Nonlinear Dimensionality Reduction for Manifold Learning , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[12]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[13]  Ross T. Whitaker,et al.  Robust non-linear dimensionality reduction using successive 1-dimensional Laplacian Eigenmaps , 2007, ICML '07.

[14]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[15]  Alon Zakai,et al.  Manifold Learning: The Price of Normalization , 2008, J. Mach. Learn. Res..

[16]  M. Kosorok Introduction to Empirical Processes and Semiparametric Inference , 2008 .

[17]  Ronen Talmon,et al.  Nonlinear intrinsic variables and state reconstruction in multiscale simulations. , 2013, The Journal of chemical physics.

[18]  Ery Arias-Castro,et al.  On the convergence of maximum variance unfolding , 2012, J. Mach. Learn. Res..

[19]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[20]  Carmeline J. Dsilva,et al.  Parsimonious Representation of Nonlinear Dynamical Systems Through Manifold Learning: A Chemotaxis Case Study , 2015, 1505.06118.

[21]  Marina Meila,et al.  Megaman: Scalable Manifold Learning in Python , 2016, J. Mach. Learn. Res..

[22]  S. Teichmann,et al.  A practical guide to single-cell RNA-sequencing for biomedical research and clinical applications , 2017, Genome Medicine.

[23]  Sarah A. Teichmann,et al.  Computational approaches for interpreting scRNA‐seq data , 2017, FEBS letters.

[24]  Yochai Blau,et al.  Non-redundant Spectral Dimensionality Reduction , 2016, ECML/PKDD.

[25]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[26]  Michael I. Jordan,et al.  On Nonlinear Dimensionality Reduction, Linear Smoothing and Autoencoding , 2018, 1803.02432.

[27]  Zhengming Ma,et al.  On the Equivalence of HLLE and LTSA , 2018, IEEE Transactions on Cybernetics.

[28]  Lai Guan Ng,et al.  Dimensionality reduction for visualizing single-cell data using UMAP , 2018, Nature Biotechnology.

[29]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction , 2018, ArXiv.

[30]  Matti Lassas,et al.  Fitting a Putative Manifold to Noisy Data , 2018, COLT.

[31]  Marina Meila,et al.  Selecting the independent coordinates of manifolds with large aspect ratios , 2019, NeurIPS.