Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit

This paper demonstrates theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal. This is a massive improvement over previous results, which require O(m2) measurements. The new results for OMP are comparable with recent results for another approach called basis pursuit (BP). In some settings, the OMP algorithm is faster and easier to implement, so it is an attractive alternative to BP for signal recovery problems.

[1]  X. Fernique Regularite des trajectoires des fonctions aleatoires gaussiennes , 1975 .

[2]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[3]  Gene H. Golub,et al.  Matrix computations , 1983 .

[4]  G. Pisier The volume of convex bodies and Banach space geometry , 1989 .

[5]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[6]  A. Atkinson Subset Selection in Regression , 1992 .

[7]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[8]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[9]  D. Du,et al.  Combinatorial Group Testing and Its Applications , 1993 .

[10]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[11]  E. Szemerédi,et al.  On the probability that a random ±1-matrix is singular , 1995 .

[12]  J. Navarro-Pedreño Numerical Methods for Least Squares Problems , 1996 .

[13]  Ronald A. DeVore,et al.  Some remarks on greedy algorithms , 1996, Adv. Comput. Math..

[14]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[15]  G. Lugosi,et al.  On Concentration-of-Measure Inequalities , 1998 .

[16]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[17]  R. DeVore,et al.  Nonlinear approximation , 1998, Acta Numerica.

[18]  Bhaskar D. Rao,et al.  An affine scaling methodology for best basis selection , 1999, IEEE Trans. Signal Process..

[19]  M. Ledoux The concentration of measure phenomenon , 2001 .

[20]  S. Szarek,et al.  Chapter 8 - Local Operator Theory, Random Matrices and Banach Spaces , 2001 .

[21]  K. Ball Chapter 4 – Convex Geometry and Functional Analysis , 2001 .

[22]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[23]  V. Temlyakov Nonlinear Methods of Approximation , 2003, Found. Comput. Math..

[24]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[25]  Dimitris Achlioptas,et al.  Database-friendly random projections: Johnson-Lindenstrauss with binary coins , 2003, J. Comput. Syst. Sci..

[26]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[27]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[28]  G. Lugosi Concentration-of-measure inequalities Lecture notes by Gábor Lugosi , 2005 .

[29]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[30]  Dmitry M. Malioutov,et al.  Homotopy continuation for sparse signal representation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[31]  M. Rudelson,et al.  Geometric approach to error-correcting codes and reconstruction of signals , 2005, math/0502299.

[32]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[33]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[34]  R. DeVore,et al.  The Johnson-Lindenstrauss Lemma Meets Compressed Sensing , 2006 .

[35]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[36]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[37]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[38]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[39]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[40]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[41]  Golub Gene H. Et.Al Matrix Computations, 3rd Edition , 2007 .

[42]  Holger Rauhut,et al.  Random Sampling of Sparse Trigonometric Polynomials, II. Orthogonal Matching Pursuit versus Basis Pursuit , 2008, Found. Comput. Math..

[43]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[44]  Holger Rauhut Stability Results for Random Sampling of Sparse Trigonometric Polynomials , 2008, IEEE Transactions on Information Theory.

[45]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..