Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints

Regularization of ill-posed linear inverse problems via ℓ1 penalization has been proposed for cases where the solution is known to be (almost) sparse. One way to obtain the minimizer of such an ℓ1 penalized functional is via an iterative soft-thresholding algorithm. We propose an alternative implementation to ℓ1-constraints, using a gradient method, with projection on ℓ1-balls. The corresponding algorithm uses again iterative soft-thresholding, now with a variable thresholding parameter. We also propose accelerated versions of this iterative method, using ingredients of the (linear) steepest descent method. We prove convergence in norm for one of these projected gradient methods, without and with acceleration.

[1]  D. Donoho,et al.  Uncertainty principles and signal recovery , 1989 .

[2]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[3]  D. Donoho.  Superresolution via sparsity constraints , 1992 .

[4]  B. Logan,et al.  Signal recovery and the large sieve , 1992 .

[5]  D. Donoho.  Nonlinear Solution of Linear Inverse Problems by Wavelet–Vaguelette Decomposition , 1995 .

[6]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[7]  Stephen Wolfram,et al.  The Mathematica Book , 1996 .

[8]  Alfredo N. Iusem,et al.  On the projected subgradient method for nonsmooth convex optimization in a Hilbert space , 1998, Math. Program..

[9]  Antonin Chambolle,et al.  Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage , 1998, IEEE Trans. Image Process..

[10]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[11]  Wolfgang Dahmen,et al.  Adaptive wavelet methods for elliptic operator equations: Convergence rates , 2001, Math. Comput..

[12]  O. Christensen.  An introduction to frames and Riesz bases , 2002 .

[13]  Wolfgang Dahmen,et al.  Adaptive Wavelet Methods II—Beyond the Elliptic Case , 2002, Found. Comput. Math..

[14]  S. Yau.  Mathematics and its applications , 2002 .

[15]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[16]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[17]  E. Candès,et al.  Astronomical image representation by the curvelet transform , 2003, Astronomy & Astrophysics.

[18]  P. Maass,et al.  AN OUTLINE OF ADAPTIVE WAVELET GALERKIN METHODS FOR TIKHONOV REGULARIZATION OF INVERSE PARABOLIC PROBLEMS , 2003 .

[19]  Fionn Murtagh,et al.  Fast communication , 2002 .

[20]  A. Cohen.  Numerical Analysis of Wavelet Methods , 2003 .

[21]  Recent Development in Theories and Numerics , 2003 .

[22]  Yuesheng Xu,et al.  Adaptive Wavelet Methods for Elliptic Operator Equations with Nonlinear Terms , 2003, Adv. Comput. Math..

[23]  Albert Cohen,et al.  Adaptive Wavelet Galerkin Methods for Linear Inverse Problems , 2004, SIAM J. Numer. Anal..

[24]  Ingrid Daubechies,et al.  Variational image restoration by means of wavelets: simultaneous decomposition , 2005 .

[25]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[26]  E. Candès,et al.  New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities , 2004 .

[27]  I. Daubechiesa,et al.  Variational image restoration by means of wavelets : Simultaneous decomposition , deblurring , and denoising , 2005 .

[28]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[29]  S. Anthoine.  Different Wavelet-based Approaches for the Separation of Noisy and Blurred Mixtures of Components. Application to Astrophysical Data. , 2005 .

[30]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[31]  H. Rauhut.  Random Sampling of Sparse Trigonometric Polynomials , 2005, math/0512642.

[32]  G. Teschke,et al.  Tikhonov replacement functionals for iteratively solving nonlinear operator equations , 2005 .

[33]  Justin Romberg,et al.  Practical Signal Recovery from Random Projections , 2005 .

[34]  Claudio Canuto,et al.  Adaptive Optimization of Convex Functionals in Banach Spaces , 2004, SIAM J. Numer. Anal..

[35]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[36]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[37]  E. Berg,et al.  In Pursuit of a Root , 2007 .

[38]  Wotao Yin,et al.  TR 0707 A Fixed-Point Continuation Method for ` 1-Regularized Minimization with Applications to Compressed Sensing , 2007 .

[39]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[40]  I. Daubechies,et al.  Tomographic inversion using L1-norm regularization of wavelet coefficients , 2006, physics/0608094.

[41]  M. Fornasier.  Domain decomposition methods for linear inverse problems with sparsity constraints , 2007 .

[42]  G. Teschke.  Multi-frame representations in linear inverse problems with mixed multi-constraints , 2007 .

[43]  M. Fornasier,et al.  Adaptive Frame Methods for Elliptic Operator Equations: The Steepest Descent Approach , 2007 .

[44]  Massimo Fornasier,et al.  Adaptive frame methods for elliptic operator equations , 2007, Adv. Comput. Math..

[45]  Massimo Fornasier,et al.  Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints , 2008, SIAM J. Numer. Anal..

[46]  M. Fornasier,et al.  Adaptive iterative thresholding algorithms for magnetoencephalography (MEG) , 2008 .

[47]  M. Fornasier,et al.  Iterative thresholding algorithms , 2008 .