Probing the Pareto Frontier for Basis Pursuit Solutions

The basis pursuit problem seeks a minimum one-norm solution of an underdetermined least-squares problem. Basis pursuit denoise (BPDN) fits the least-squares problem only approximately, and a single parameter determines a curve that traces the optimal trade-off between the least-squares fit and the one-norm of the solution. We prove that this curve is convex and continuously differentiable over all points of interest, and show that it gives an explicit relationship to two other optimization problems closely related to BPDN. We describe a root-finding algorithm for finding arbitrary points on this curve; the algorithm is suitable for problems that are large scale and for those that are in the complex domain. At each iteration, a spectral gradient-projection method approximately minimizes a least-squares problem with an explicit one-norm constraint. Only matrix-vector operations are required. The primal-dual solution of this problem gives function and derivative information needed for the root-finding method. Numerical experiments on a comprehensive set of test problems demonstrate that the method scales well to large problems.

[1]  W. Hager,et al.  The cyclic Barzilai-–Borwein method for unconstrained optimization , 2006 .

[2]  I. Daubechies,et al.  Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints , 2007, 0706.4297.

[3]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[4]  F. Herrmann,et al.  Simply denoise: Wavefield reconstruction via jittered undersampling , 2008 .

[5]  Frédéric Lesage,et al.  The Application of Compressed Sensing for , 2009 .

[6]  A. Banerjee Convex Analysis and Optimization , 2006 .

[7]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[8]  Dimitri P. Bertsekas,et al.  Convex Analysis and Optimization , 2003 .

[9]  Yaakov Tsaig,et al.  Fast Solution of $\ell _{1}$ -Norm Minimization Problems When the Solution May Be Sparse , 2008, IEEE Transactions on Information Theory.

[10]  Stephen P. Boyd,et al.  Convex Optimization , 2004, IEEE Transactions on Automatic Control.

[11]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[12]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[13]  Michael P. Friedlander,et al.  Discussion: The Dantzig selector: Statistical estimation when p is much larger than n , 2007, 0803.3135.

[14]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[15]  J. M. Martínez,et al.  Inexact spectral projected gradient methods on convex sets , 2003 .

[16]  Achiya Dax On Regularized Least Norm Problems , 1992, SIAM J. Optim..

[17]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[18]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[19]  M. R. Osborne,et al.  On the LASSO and its Dual , 2000 .

[20]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[21]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[22]  J. Dennis,et al.  A closer look at drawbacks of minimizing weighted sums of objectives for Pareto set generation in multicriteria optimization problems , 1997 .

[23]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[24]  M. R. Osborne,et al.  A new approach to variable selection in least squares problems , 2000 .

[25]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[26]  Rayan Saab,et al.  Sparco: A Testing Framework for Sparse Reconstruction , 2007 .

[27]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[28]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[29]  Roger Fletcher,et al.  Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming , 2005, Numerische Mathematik.

[30]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[31]  S. Leyffer A Note on Multiobjective Optimization and Complementarity Constraints , 2005 .

[32]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[33]  Dianne P. O'Leary,et al.  The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems , 1993, SIAM J. Sci. Comput..

[34]  Antonin Chambolle,et al.  Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage , 1998, IEEE Trans. Image Process..

[35]  Åke Björck,et al.  Numerical methods for least square problems , 1996 .

[36]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[37]  Rayan Saab,et al.  Algorithm 890: Sparco: A Testing Framework for Sparse Reconstruction , 2009, TOMS.

[38]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[39]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[40]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[41]  F. Herrmann,et al.  Sparseness-constrained data continuation with frames: Applications to missing traces and aliased signals in 2/3-D , 2005 .

[42]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1995 .

[43]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.

[44]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.