Signal Recovery by Proximal Forward-Backward Splitting

We show that various inverse problems in signal recovery can be formulated as the generic problem of minimizing the sum of two convex functions with certain regularity properties. This formulation makes it possible to derive existence, uniqueness, characterization, and stability results in a unified and standardized fashion for a large class of apparently disparate problems. Recent results on monotone operator splitting methods are applied to establish the convergence of a forward-backward algorithm to solve the generic problem. In turn, we recover, extend, and provide a simplified analysis for a variety of existing iterative methods. Applications to geometry/texture image decomposition schemes are also discussed. A novelty of our framework is to use extensively the notion of a proximity operator, which was introduced by Moreau in the 1960s.

[1]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[2]  A. Papoulis,et al.  The Fourier Integral and Its Applications , 1963 .

[3]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[4]  S. Twomey.  The application of numerical filtering to the solution of integral equations encountered in indirect sensing measurements , 1965 .

[5]  Boris Polyak,et al.  Constrained minimization methods , 1966 .

[6]  B. Hunt.  The inverse problem of radiography , 1970 .

[7]  B. R. Hunt,et al.  The Application of Constrained Least Squares Estimation to Image Restoration by Digital Computer , 1973, IEEE Transactions on Computers.

[8]  R. Gerchberg.  Super-resolution through Error Energy Reduction , 1974 .

[9]  A. Papoulis.  A new algorithm in spectral analysis and band-limited extrapolation. , 1975 .

[10]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[11]  B. R. Hunt,et al.  Digital Image Restoration , 1977 .

[12]  J. Baillon,et al.  Quelques propriétés des opérateurs angle-bornés etn-cycliquement monotones , 1977 .

[13]  Dante C. Youla,et al.  Generalized Image Restoration by the Method of Alternating Orthogonal Projections , 1978 .

[14]  Felipe Acker,et al.  Convergence d'un schéma de minimisation alternée , 1980 .

[15]  Gabor T. Herman,et al.  Image reconstruction from projections : the fundamentals of computerized tomography , 1980 .

[16]  D. Youla,et al.  Image Restoration by the Method of Convex Projections: Part 1ߞTheory , 1982, IEEE Transactions on Medical Imaging.

[17]  A. Levy.  A fast quadratic programming algorithm for positive signal restoration , 1983 .

[18]  C. Zălinescu.  On uniformly convex functions , 1983 .

[19]  R. Marks,et al.  Signal synthesis in the presence of an inconsistent set of constraints , 1985 .

[20]  D. Titterington.  General structure of regularization procedures in image reconstruction , 1985 .

[21]  J. Cadzow,et al.  Sequences with positive semidefinite Fourier transforms , 1986, IEEE Trans. Acoust. Speech Signal Process..

[22]  A. Lannes,et al.  Stabilized Reconstruction in Signal and Image Processing , 1987 .

[23]  Henry Stark,et al.  Image recovery: Theory and application , 1987 .

[24]  E. Zeidler.  Nonlinear functional analysis and its applications , 1988 .

[25]  Norman E. Hurt,et al.  Phase Retrieval and Zero Crossings , 1989 .

[26]  J. Aubin.  Set-valued analysis , 1990 .

[27]  W. F. Ames,et al.  Phase retrieval and zero crossings (Mathematical methods in image reconstruction) , 1990 .

[28]  Bertolt Eicke.  Iteration methods for convexly constrained ill-posed problems in hilbert space , 1992 .

[29]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[30]  J. Borwein,et al.  Krasnoselski-Mann Iterations in Normed Spaces , 1992, Canadian Mathematical Bulletin.

[31]  P. L. Combettes.  The foundations of set theoretic estimation , 1993 .

[32]  A. M. Thompson,et al.  On some Bayesian choices of regularization parameter in image restoration , 1993 .

[33]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[34]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[35]  Patrick L. Combettes,et al.  Inconsistent signal feasibility problems: least-squares solutions in a product space , 1994, IEEE Trans. Signal Process..

[36]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[37]  Gabor T. Herman,et al.  Image Reconstruction From Projections , 1975, Real Time Imaging.

[38]  P. L. Combettes,et al.  Convex multiresolution analysis , 1996, Proceedings of Third International Symposium on Time-Frequency and Time-Scale Analysis (TFTS-96).

[39]  Patrick L. Combettes,et al.  Wavelet synthesis by alternating projections , 1996, IEEE Trans. Signal Process..

[40]  D. Dobson,et al.  Analysis of regularized total variation penalty methods for denoising , 1996 .

[41]  P. L. Combettes.  The Convex Feasibility Problem in Image Recovery , 1996 .

[42]  Y. Censor,et al.  Parallel Optimization: Theory, Algorithms, and Applications , 1997 .

[43]  L. Vese,et al.  A Variational Method in Image Recovery , 1997 .

[44]  P. Lions,et al.  Image recovery via total variation minimization and related problems , 1997 .

[45]  Y. Censor,et al.  Parallel Optimization:theory , 1997 .

[46]  Aggelos K. Katsaggelos,et al.  Digital image restoration , 2012, IEEE Signal Process. Mag..

[47]  Antonin Chambolle,et al.  Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage , 1998, IEEE Trans. Image Process..

[48]  Ashutosh Sabharwal,et al.  Convexly constrained linear inverse problems: iterative least-squares and regularization , 1998, IEEE Trans. Signal Process..

[49]  Dominikus Noll,et al.  Reconstruction with Noisy Data: An Approach via Eigenvalue Optimization , 1998, SIAM J. Optim..

[50]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[51]  Yongyi Yang,et al.  Vector Space Projections: A Numerical Approach to Signal and Image Processing, Neural Nets, and Optics , 1998 .

[52]  P. Marchal,et al.  A new reconstruction methodology for computerized tomography: FRECT (Fourier regularized computed tomography) , 1999 .

[53]  Patrick L. Combettes,et al.  Hard-constrained inconsistent signal feasibility problems , 1999, IEEE Trans. Signal Process..

[54]  P. Maréchal,et al.  A new reconstruction methodology for computerized tomography: FRECT (Fourier regularized computed tomography) , 1999, 1999 IEEE Nuclear Science Symposium. Conference Record. 1999 Nuclear Science Symposium and Medical Imaging Conference (Cat. No.99CH37019).

[55]  Dominikus Noll,et al.  Adaptive Image Reconstruction Using Information Measures , 2000, SIAM J. Control. Optim..

[56]  Yves Meyer,et al.  Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures , 2001 .

[57]  C. Zălinescu.  Convex analysis in general vector spaces , 2002 .

[58]  C. Byrne.  Iterative oblique projection onto convex sets and the split feasibility problem , 2002 .

[59]  Michel Defrise,et al.  A note on wavelet-based inversion algorithms , 2002 .

[60]  Stanley Osher,et al.  Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing , 2003, J. Sci. Comput..

[61]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[62]  C. Zălinescu,et al.  On Uniform Convexity, Total Convexity and Convergence of the Proximal Point and Outer Bregman Projection Algorithms in Banach Spaces , 2003 .

[63]  Stephen L. Keeling,et al.  Total variation based convex filters for medical imaging , 2003, Appl. Math. Comput..

[64]  Stanley Osher,et al.  Image Decomposition and Restoration Using Total Variation Minimization and the H1 , 2003, Multiscale Model. Simul..

[65]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[66]  E. Candès,et al.  Astronomical image representation by the curvelet transform , 2003, Astronomy & Astrophysics.

[67]  Fionn Murtagh,et al.  Fast communication , 2002 .

[68]  A. Cohen.  Numerical Analysis of Wavelet Methods , 2003 .

[69]  Antonin Chambolle,et al.  A l1-Unified Variational Framework for Image Restoration , 2004, ECCV.

[70]  P. L. Combettes.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[71]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[72]  Hein Hundal.  An alternating projection that does not converge in norm , 2004 .

[73]  Patrick L. Combettes,et al.  WAVELET-CONSTRAINED IMAGE RESTORATION , 2004 .

[74]  Stanley Osher,et al.  Image Denoising and Decomposition with Total Variation Minimization and Oscillatory Functions , 2004, Journal of Mathematical Imaging and Vision.

[75]  Heinz H. Bauschke,et al.  The asymptotic behavior of the composition of two resolvents , 2005, Nonlinear Analysis: Theory, Methods & Applications.

[76]  Antonin Chambolle,et al.  Dual Norms and Image Decomposition Models , 2005, International Journal of Computer Vision.

[77]  Yair Censor,et al.  A multiprojection algorithm using Bregman projections in a product space , 1994, Numerical Algorithms.

[78]  P. L. Combettes,et al.  Equilibrium programming in Hilbert spaces , 2005 .

[79]  Antonin Chambolle,et al.  Image Decomposition into a Bounded Variation Component and an Oscillating Component , 2005, Journal of Mathematical Imaging and Vision.

[80]  Tony F. Chan,et al.  Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection , 2006, International Journal of Computer Vision.

[81]  H. Hanche-Olsen.  On the uniform convexity of L^p , 2005, math/0502021.