Robust Principal Component Analysis with Non-Greedy l1-Norm Maximization

Principal Component Analysis (PCA) is one of the most important methods to handle high-dimensional data. However, the high computational complexity makes it hard to apply to the large scale data with high dimensionality, and the used l2-norm makes it sensitive to outliers. A recent work proposed principal component analysis based on l1-normmaximization, which is efficient and robust to outliers. In that work, a greedy strategy was applied due to the difficulty of directly solving the l1-norm maximization problem, which is easy to get stuck in local solution. In this paper, we first propose an efficient optimization algorithmto solve a general l1-norm maximization problem, and then propose a robust principal component analysis with non-greedy l1-norm maximization. Experimental results on real world datasets show that the nongreedy method always obtains much better solution than that of the greedy method.

[1]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[2]  D. Hawkins,et al.  Methods of L1 estimation of a covariance matrix , 1987 .

[3]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[4]  P. Sabatier.  A L 1 -norm Pca and a Heuristic Approach , 1996 .

[5]  L. Lathauwer,et al.  Signal Processing based on Multilinear Algebra , 1997 .

[6]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[7]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[8]  Henrik Aanæs,et al.  Robust Factorization , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[10]  Takeo Kanade,et al.  Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[12]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[13]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[14]  Feiping Nie,et al.  Learning a Mahalanobis distance metric for data clustering and classification , 2008, Pattern Recognit..

[15]  Chris H. Q. Ding,et al.  Robust tensor factorization using R1 norm , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[18]  Yi Yang,et al.  Retrieval based interactive cartoon synthesis via unsupervised bi-distance metric learning , 2009, ACM Multimedia.

[19]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[20]  Yan Liu,et al.  Multilinear Maximum Distance Embedding Via L1-Norm Optimization , 2010, AAAI.

[21]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[22]  Xuelong Li,et al.  L1-Norm-Based 2DPCA , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Xuelong Li,et al.  Robust Tensor Analysis With L1-Norm , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Feiping Nie,et al.  A general kernelization framework for learning algorithms based on kernel PCA , 2010, Neurocomputing.