Recovery of Low-Rank Matrices Under Affine Constraints via a Smoothed Rank Function

In this paper, the problem of matrix rank minimization under affine constraints is addressed. The state-of-the-art algorithms can recover matrices with a rank much less than what is sufficient for the uniqueness of the solution of this optimization problem. We propose an algorithm based on a smooth approximation of the rank function, which practically improves recovery limits on the rank of the solution. This approximation leads to a non-convex program; thus, to avoid getting trapped in local solutions, we use the following scheme. Initially, a rough approximation of the rank function subject to the affine constraints is optimized. As the algorithm proceeds, finer approximations of the rank are optimized and the solver is initialized with the solution of the previous approximation until reaching the desired accuracy. On the theoretical side, benefiting from the spherical section property, we will show that the sequence of the solutions of the approximating programs converges to the minimum rank solution. On the experimental side, it will be shown that the proposed algorithm, termed SRF standing for smoothed rank function, can recover matrices, which are unique solutions of the rank minimization problem and yet not recoverable by nuclear norm minimization. Furthermore, it will be demonstrated that, in completing partially observed matrices, the accuracy of SRF is considerably and consistently better than some famous algorithms when the number of revealed entries is close to the minimum number of parameters that uniquely represent a low-rank matrix.

[1]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[2]  Rémi Gribonval,et al.  Crystal-MUSIC: Accurate Localization of Multiple Sources in Diffuse Noise Environments Using Crystal-Shaped Microphone Arrays , 2010, LVA/ICA.

[3]  Lieven Vandenberghe,et al.  Interior-Point Method for Nuclear Norm Approximation with Application to System Identification , 2009, SIAM J. Matrix Anal. Appl..

[4]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[5]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Math. Program. Comput..

[6]  Stephen Becker,et al.  Quantum state tomography via compressed sensing. , 2009, Physical review letters.

[7]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[8]  Shimon Ullman,et al.  Uncovering shared structures in multiclass classification , 2007, ICML '07.

[9]  Adrian S. Lewis,et al.  Convex Analysis on the Hermitian Matrices , 1996, SIAM J. Optim..

[10]  Mike E. Davies,et al.  Latent Variable Analysis and Signal Separation , 2010 .

[11]  Volkan Cevher,et al.  Bilinear Generalized Approximate Message Passing—Part I: Derivation , 2013, IEEE Transactions on Signal Processing.

[12]  Krishnamurthy Dvijotham,et al.  A nullspace analysis of the nuclear norm heuristic for rank minimization , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[14]  James M. Taylor Eigenvalues for Sums of Hermitian Matrices , 2015 .

[15]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[16]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[17]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[18]  Hristo S. Sendov,et al.  Nonsmooth Analysis of Singular Values. Part I: Theory , 2005 .

[19]  Babak Hassibi,et al.  New Null Space Results and Recovery Thresholds for Matrix Rank Minimization , 2010, ArXiv.

[20]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[21]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[22]  C. Jutten,et al.  SRF: Matrix completion based on smoothed rank function , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[23]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[24]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[25]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[26]  Yin Zhang,et al.  Theory of Compressive Sensing via ℓ1-Minimization: a Non-RIP Analysis and Extensions , 2013 .

[27]  Babak Hassibi,et al.  A simplified approach to recovery conditions for low rank matrices , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[28]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[29]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[30]  Naihua Xiu,et al.  Sufficient Conditions for Low-rank Matrix Recovery, Translated from Sparse Signal Recovery , 2011, ArXiv.

[31]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[32]  Dima Grigoriev,et al.  Complexity of Quantifier Elimination in the Theory of Algebraically Closed Fields , 1984, MFCS.

[33]  Yoram Bresler,et al.  ADMiRA: Atomic Decomposition for Minimum Rank Approximation , 2009, IEEE Transactions on Information Theory.

[34]  Christian Jutten,et al.  A Fast Approach for Overcomplete Sparse Decomposition Based on Smoothed $\ell ^{0}$ Norm , 2008, IEEE Transactions on Signal Processing.

[35]  R. Rockafellar Convex Analysis: (pms-28) , 1970 .

[36]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[37]  Zhu Han,et al.  Collaborative Spectrum Sensing from Sparse Observations in Cognitive Radio Networks , 2010, IEEE Journal on Selected Areas in Communications.

[38]  Weiyu Xu,et al.  Null space conditions and thresholds for rank minimization , 2011, Math. Program..

[39]  Volkan Cevher,et al.  Bilinear Generalized Approximate Message Passing , 2013, ArXiv.

[40]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[41]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[42]  Christian Jutten,et al.  Sparse Recovery using Smoothed ℓ0 (SL0): Convergence Analysis , 2010, ArXiv.

[43]  A. Lewis The Convex Analysis of Unitarily Invariant Matrix Functions , 1995 .

[44]  Christian Jutten,et al.  Sparse Recovery using Smoothed ℓ0 (SL0): Convergence Analysis , 2010, ArXiv.

[45]  Babak Hassibi,et al.  Improved thresholds for rank minimization , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[46]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .