Efficient Structured Matrix Rank Minimization

We study the problem of finding structured low-rank matrices using nuclear norm regularization where the structure is encoded by a linear map. In contrast to most known approaches for linearly structured rank minimization, we do not (a) use the full SVD, nor (b) resort to augmented Lagrangian techniques, nor (c) solve linear systems per iteration. Instead, we formulate the problem differently so that it is amenable to a generalized conditional gradient method, which results in a practical improvement with low per iteration computational cost. Numerical results show that our approach significantly outperforms state-of-the-art competitors in terms of running time, while effectively recovering low rank solutions in stochastic system realization and spectral compressed sensing problems.

[1]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[2]  J. Cullum,et al.  Lanczos algorithms for large symmetric eigenvalue computations , 1985 .

[3]  James A. Cadzow,et al.  Signal enhancement-a composite property mapping algorithm , 1988, IEEE Trans. Acoust. Speech Signal Process..

[4]  Yingbo Hua Estimating two-dimensional frequencies by matrix enhancement and matrix pencil , 1992, IEEE Trans. Signal Process..

[5]  Petre Stoica,et al.  Vector ARMA estimation: a reliable subspace approach , 2000, IEEE Trans. Signal Process..

[6]  Tommi S. Jaakkola,et al.  Maximum-Margin Matrix Factorization , 2004, NIPS.

[7]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.

[8]  Tao Ding,et al.  A Rank Minimization Approach to Video Inpainting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Ivan Markovsky,et al.  Structured low-rank approximation and its applications , 2008, Autom..

[10]  Dirk A. Lorenz,et al.  A generalized conditional gradient method and its connection to an iterative shrinkage method , 2009, Comput. Optim. Appl..

[11]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[12]  Lieven Vandenberghe,et al.  Semidefinite programming methods for system realization and identification , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[13]  Lieven Vandenberghe,et al.  Interior-Point Method for Nuclear Norm Approximation with Application to System Identification , 2009, SIAM J. Matrix Anal. Appl..

[14]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[15]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[16]  Mehryar Mohri,et al.  Spectral Learning of General Weighted Automata via Constrained Matrix Completion , 2012, NIPS.

[17]  Xiaofan Zhu,et al.  Graph Spectral Compressed Sensing , 2012 .

[18]  Yaoliang Yu,et al.  Accelerated Training for Matrix-norm Regularization: A Boosting Approach , 2012, NIPS.

[19]  Zhang Liu,et al.  Nuclear norm system identification with missing inputs and outputs , 2013, Syst. Control. Lett..

[20]  Marco F. Duarte,et al.  Spectral compressive sensing , 2013 .

[21]  Martin Jaggi,et al.  Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization , 2013, ICML.

[22]  Zaid Harchaoui,et al.  Conditional gradient algorithms for machine learning , 2013 .

[23]  Volkan Cevher,et al.  An SVD-free Approach to a Class of Structured Low Rank Matrix Optimization Problems with Application to System Identification , 2013, CDC 2013.

[24]  Yuxin Chen,et al.  Spectral Compressed Sensing via Structured Matrix Completion , 2013, ICML.

[25]  Paul Tseng,et al.  Hankel Matrix Rank Minimization with Applications to System Identification and Realization , 2013, SIAM J. Matrix Anal. Appl..

[26]  Michael Elad,et al.  Calibrationless parallel imaging reconstruction based on structured low‐rank matrix completion , 2013, Magnetic resonance in medicine.

[27]  Ivan Markovsky,et al.  Factorization Approach to Structured Low-Rank Approximation with Applications , 2014, SIAM J. Matrix Anal. Appl..