Similarity Learning of Manifold Data

Without constructing adjacency graph for neighborhood, we propose a method to learn similarity among sample points of manifold in Laplacian embedding (LE) based on adding constraints of linear reconstruction and least absolute shrinkage and selection operator type minimization. Two algorithms and corresponding analyses are presented to learn similarity for mix-signed and nonnegative data respectively. The similarity learning method is further extended to kernel spaces. The experiments on both synthetic and real world benchmark data sets demonstrate that the proposed LE with new similarity has better visualization and achieves higher accuracy in classification.

[1]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[2]  Lijun Jiang,et al.  Using locally estimated geodesic distance to optimize neighborhood graph for isometric data embedding , 2008, Pattern Recognit..

[3]  Mahdieh Soleymani Baghshah,et al.  Semi-Supervised Metric Learning Using Pairwise Constraints , 2009, IJCAI.

[4]  Longbing Cao,et al.  A Similarity-Based Classification Framework for Multiple-Instance Learning , 2014, IEEE Transactions on Cybernetics.

[5]  Marc Sebban,et al.  Good edit similarity learning by loss minimization , 2012, Machine Learning.

[6]  Gang Niu,et al.  Information-Theoretic Semi-Supervised Metric Learning via Entropy Regularization , 2012, Neural Computation.

[7]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[8]  Matti Pietikäinen,et al.  Selection of the Optimal Parameter value for the Locally Linear Embedding Algorithm , 2002, FSKD.

[9]  Paul L. Rosin,et al.  Selection of the optimal parameter value for the Isomap algorithm , 2006, Pattern Recognit. Lett..

[10]  Dacheng Tao,et al.  Constrained Empirical Risk Minimization Framework for Distance Metric Learning , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Marco Maggini,et al.  Learning from pairwise constraints by Similarity Neural Networks , 2012, Neural Networks.

[12]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[13]  Xiaofang Gao,et al.  The dynamical neighborhood selection based on the sampling density and manifold curvature for isometric data embedding , 2011, Pattern Recognit. Lett..

[14]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[15]  Vinod Nair,et al.  Learning hierarchical similarity metrics , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Stephen Tyree,et al.  Non-linear Metric Learning , 2012, NIPS.

[17]  Samy Bengio,et al.  Large Scale Online Learning of Image Similarity through Ranking , 2009, IbPRIA.

[18]  Rudolf Fleischer,et al.  Distance Approximating Dimension Reduction of Riemannian Manifolds , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  Gabriela Csurka,et al.  Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost , 2012, ECCV.

[20]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[21]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[22]  Feiping Nie,et al.  Cauchy Graph Embedding , 2011, ICML.

[23]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[24]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[25]  Guihua Wen.  Relative transformation-based neighborhood optimization for isometric embedding , 2009, Neurocomputing.

[26]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Hongyuan Zha,et al.  Adaptive Manifold Learning , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[29]  Bert Huang,et al.  Learning a Distance Metric from a Network , 2011, NIPS.

[30]  Chunhua Shen,et al.  Efficiently Learning a Distance Metric for Large Margin Nearest Neighbor Classification , 2011, AAAI.

[31]  Richard S. Zemel,et al.  Stochastic k-Neighborhood Selection for Supervised and Unsupervised Learning , 2013, ICML.

[32]  Frédéric Jurie,et al.  PCCA: A new approach for distance learning from sparse pairwise constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Zi Huang,et al.  Robust Hashing With Local Models for Approximate Similarity Search , 2014, IEEE Transactions on Cybernetics.

[34]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[35]  Qiang Yang,et al.  Distance Metric Learning under Covariate Shift , 2011, IJCAI.

[36]  I. Jolliffe.  Principal Component Analysis , 2002 .

[37]  L. Li,et al.  Learning Similarity With Multikernel Method , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[38]  Kaizhu Huang,et al.  Low Rank Metric Learning with Manifold Regularization , 2011, 2011 IEEE 11th International Conference on Data Mining.

[39]  Li Cheng,et al.  Riemannian Similarity Learning , 2013, ICML.

[40]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[41]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Marc Sebban,et al.  Similarity Learning for Provably Accurate Sparse Linear Classification , 2012, ICML.

[43]  R. Marshall.  5. Multidimensional Scaling. 2nd edn. Trevor F. Cox and Michael A. A. Cox, Chapman & Hall/CRC, Boca Raton, London, New York, Washington DC, 2000. No. of pages: xiv + 309. Price: $79.95. ISBN 1‐58488‐094‐5 , 2002 .

[44]  Jiayan Jiang,et al.  Learning a mixture of sparse distance metrics for classification and dimensionality reduction , 2011, 2011 International Conference on Computer Vision.