Robust Kernel Nonnegative Matrix Factorization

Kernel methods and Nonnegative matrix factorization (NMF) are both widely used in data mining and machine learning. The previous one is best known for its capability of transforming data into high dimension feature space, while the latter one is well known for its natural interpretations and good performance. In this paper, we propose a robust kernel NMF approach using L2, 1 norm loss function. Compared with the standard NMF algorithm, the new robust kernel NMF updating algorithm is as elegant and as simple, but with the newly added robustness to handle significantly corrupted datasets because of using L2, 1 norm. Experiments on normal and occluded datasets indicate that robust kernel NMF always perform better than k-means and standard NMF.

[1]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[2]  Cheng-Lin Liu,et al.  Classification and Learning for Character Recognition: Comparison of Methods and Remaining Problems , 2005 .

[3]  Tommi S. Jaakkola,et al.  Maximum-Margin Matrix Factorization , 2004, NIPS.

[4]  Zhaoshui He,et al.  Symmetric Nonnegative Matrix Factorization: Algorithms and Applications to Probabilistic Clustering , 2011, IEEE Transactions on Neural Networks.

[5]  Daoqiang Zhang,et al.  Non-negative Matrix Factorization on Kernels , 2006, PRICAI.

[6]  K. Kim,et al.  Face recognition using kernel principal component analysis , 2002, IEEE Signal Process. Lett..

[7]  Chris H. Q. Ding,et al.  Solving Consensus and Semi-supervised Clustering Problems Using Nonnegative Matrix Factorization , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[8]  Michael R. Lyu,et al.  Online learning for multi-task feature selection , 2010, CIKM '10.

[9]  Daniel D. Lee,et al.  Multiplicative Updates for Nonnegative Quadratic Programming in Support Vector Machines , 2002, NIPS.

[10]  Nanning Zheng,et al.  Nonnegative matrix factorization and its applications in pattern recognition , 2006 .

[11]  Andrzej Cichocki,et al.  Non-negative Matrix Factorization with Quasi-Newton Optimization , 2006, ICAISC.

[12]  Michael W. Berry,et al.  Text Mining Using Non-Negative Matrix Factorizations , 2004, SDM.

[13]  Quanquan Gu,et al.  Co-clustering on manifolds , 2009, KDD.

[14]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[15]  Fei Wang,et al.  Semi-Supervised Clustering via Matrix Factorization , 2008, SDM.

[16]  Jiawei Han,et al.  Non-negative Matrix Factorization on Manifold , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[17]  Pablo Tamayo,et al.  Metagenes and molecular pattern discovery using matrix factorization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[19]  Haesun Park,et al.  Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons , 2011, SIAM J. Sci. Comput..

[20]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2006 .

[21]  Andrzej Cichocki,et al.  Nonnegative matrix factorization with quadratic programming , 2008, Neurocomputing.

[22]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[24]  Haesun Park,et al.  Toward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[25]  Inderjit S. Dhillon,et al.  Generalized Nonnegative Matrix Approximations with Bregman Divergences , 2005, NIPS.

[26]  Feiping Nie,et al.  Improved MinMax Cut Graph Clustering with Nonnegative Relaxation , 2010, ECML/PKDD.

[27]  C. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and K-means - Spectral Clustering , 2005 .

[28]  Jonathan Foote,et al.  Summarizing video using non-negative similarity matrix factorization , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[29]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[30]  Dmitry Zelenko,et al.  Kernel Methods for Relation Extraction , 2002, J. Mach. Learn. Res..

[31]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[32]  Tamir Hazan,et al.  Non-negative tensor factorization with applications to statistics and computer vision , 2005, ICML.

[33]  Chris H. Q. Ding,et al.  Robust nonnegative matrix factorization using L21-norm , 2011, CIKM '11.

[34]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[35]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[36]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .