Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints

Vector-valued data appearing in concrete applications often possess sparse expansions with respect to a preassigned frame for each vector component individually. Additionally, different components may also exhibit common sparsity patterns. Recently, there were introduced sparsity measures that take into account such joint sparsity patterns, promoting coupling of nonvanishing components. These measures are typically constructed as weighted $\ell_1$ norms of componentwise $\ell_q$ norms of frame coefficients. We show how to compute solutions of linear inverse problems with such joint sparsity regularization constraints by fast thresholded Landweber algorithms. Next we discuss the adaptive choice of suitable weights appearing in the definition of sparsity measures. The weights are interpreted as indicators of the sparsity pattern and are iteratively updated after each new application of the thresholded Landweber algorithm. The resulting two-step algorithm is interpreted as a double-minimization scheme for a suitable target functional. We show its $\ell_2$-norm convergence. An implementable version of the algorithm is also formulated, and its norm convergence is proven. Numerical experiments in color image restoration are presented.

[1]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[2]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[3]  M. Fornasier,et al.  Adaptive Frame Methods for Elliptic Operator Equations: The Steepest Descent Approach , 2007 .

[4]  M. Fornasier,et al.  Iterative thresholding algorithms , 2008 .

[5]  H. Keller,et al.  Continuation-Conjugate Gradient Methods for the Least Squares Solution of Nonlinear Boundary Value Problems , 1985 .

[6]  H. Walker Implementation of the GMRES method using householder transformations , 1988 .

[7]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[8]  Yousef Saad,et al.  Hybrid Krylov Methods for Nonlinear Systems of Equations , 1990, SIAM J. Sci. Comput..

[9]  Holger Rauhut,et al.  Random Sampling of Sparse Trigonometric Polynomials, II. Orthogonal Matching Pursuit versus Basis Pursuit , 2008, Found. Comput. Math..

[10]  S. Mallat A wavelet tour of signal processing , 1998 .

[11]  I. Daubechies,et al.  Iteratively solving linear inverse problems under general convex constraints , 2007 .

[12]  O. Christensen An introduction to frames and Riesz bases , 2002 .

[13]  H. Rauhut Random Sampling of Sparse Trigonometric Polynomials , 2005, math/0512642.

[14]  D. Donoho Superresolution via sparsity constraints , 1992 .

[15]  Massimo Fornasier,et al.  Fast, robust and efficient 2D pattern recognition for re-assembling fragmented images , 2005, Pattern Recognit..

[16]  Richard G. Baraniuk,et al.  Distributed Compressed Sensing Dror , 2005 .

[17]  S. Anthoine Different Wavelet-based Approaches for the Separation of Noisy and Blurred Mixtures of Components. Application to Astrophysical Data. , 2005 .

[18]  J. Tropp Algorithms for simultaneous sparse approximation. Part II: Convex relaxation , 2006, Signal Process..

[19]  D. Donoho Nonlinear Solution of Linear Inverse Problems by Wavelet–Vaguelette Decomposition , 1995 .

[20]  Stéphane Jaffard,et al.  Beyond Besov Spaces, Part 2: Oscillation Spaces , 2003 .

[21]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[22]  Y. Saad Krylov subspace methods for solving large unsymmetric linear systems , 1981 .

[23]  P. Maass,et al.  AN OUTLINE OF ADAPTIVE WAVELET GALERKIN METHODS FOR TIKHONOV REGULARIZATION OF INVERSE PARABOLIC PROBLEMS , 2003 .

[24]  Kazuo Murota LU-Decomposition of a Matrix with Entries of Different Kinds (線型計算の標準算法と実現) , 1982 .

[25]  L. Ambrosio,et al.  Approximation of functional depending on jumps by elliptic functional via t-convergence , 1990 .

[26]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[27]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[28]  Rob P. Stevenson,et al.  Adaptive Solution of Operator Equations Using Wavelet Frames , 2003, SIAM J. Numer. Anal..

[29]  Albert Cohen,et al.  Adaptive Wavelet Galerkin Methods for Linear Inverse Problems , 2004, SIAM J. Numer. Anal..

[30]  H. Engl,et al.  Regularization of Inverse Problems , 1996 .

[31]  G. Teschke,et al.  Tikhonov replacement functionals for iteratively solving nonlinear operator equations , 2005 .

[32]  Richard A. Brualdi,et al.  On Sign-Nonsingular Matrices and the Conversion of the Permanent into the Determinant , 1990, Applied Geometry And Discrete Mathematics.

[33]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[34]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[35]  Stéphane Jaffard Beyond Besov Spaces Part 1: Distributions of Wavelet Coefficients , 2004 .

[36]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[37]  O. Axelsson Conjugate gradient type methods for unsymmetric and inconsistent systems of linear equations , 1980 .

[38]  Claudio Canuto,et al.  Adaptive Optimization of Convex Functionals in Banach Spaces , 2004, SIAM J. Numer. Anal..

[39]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[40]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[41]  I. Daubechiesa,et al.  Variational image restoration by means of wavelets : Simultaneous decomposition , deblurring , and denoising , 2005 .

[42]  A. Cohen Numerical Analysis of Wavelet Methods , 2003 .

[43]  G. Teschke Multi-Frames in Thresholding Iterations for Nonlinear Operator Equations with Mixed Sparsity Constraints , 2005 .

[44]  Massimo Fornasier,et al.  Adaptive frame methods for elliptic operator equations , 2007, Adv. Comput. Math..

[45]  Roland A. Sweet,et al.  Algorithm 541: Efficient Fortran Subprograms for the Solution of Separable Elliptic Partial Differential Equations [D3] , 1979, TOMS.

[46]  H. Elman Iterative methods for large, sparse, nonsymmetric systems of linear equations , 1982 .

[47]  S. Eisenstat,et al.  Variational Iterative Methods for Nonsymmetric Systems of Linear Equations , 1983 .

[48]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[49]  Frank H. Clarks Convex Analysis and Variational Problems (Ivar Ekeland and Roger Temam) , 1978 .

[50]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[51]  Ron Kimmel,et al.  Variational Restoration and Edge Detection for Color Images , 2003, Journal of Mathematical Imaging and Vision.

[52]  E. Candès,et al.  New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities , 2004 .

[53]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[54]  P. M. Gibson,et al.  Conversion of the permanent into the determinant , 1971 .

[55]  Ingrid Daubechies,et al.  Variational image restoration by means of wavelets: simultaneous decomposition , 2005 .