For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution

We consider linear equations y = Φx where y is a given vector in ℝn and Φ is a given n × m matrix with n < m ≤ τn, and we wish to solve for x ∈ ℝm. We suppose that the columns of Φ are normalized to the unit 𝓁2‐norm, and we place uniform measure on such Φ. We prove the existence of ρ = ρ(τ) > 0 so that for large n and for all Φ's except a negligible fraction, the following property holds: For every y having a representation y = Φx0 by a coefficient vector x0 ∈ ℝm with fewer than ρ · n nonzeros, the solution x1 of the 𝓁1‐minimization problem $${\rm min} \|x\|_{1} \;\;{subject \; to}\;\; \Phi x = y$$ is unique and equal to x0. In contrast, heuristic attempts to sparsely solve such systems—greedy algorithms and thresholding—perform poorly in this challenging setting. The techniques include the use of random proportional embeddings and almost‐spherical sections in Banach space theory, and deviation bounds for the eigenvalues of random Wishart matrices. © 2006 Wiley Periodicals, Inc.

[1]  T. Figiel,et al.  The dimension of almost spherical sections of convex bodies , 1976 .

[2]  G. Schechtman Random embeddings of Euclidean spaces in sequence spaces , 1981 .

[3]  William B. Johnson,et al.  Embeddinglpm intol1n , 1982 .

[4]  M. R. Leadbetter,et al.  Extremes and Related Properties of Random Sequences and Processes: Springer Series in Statistics , 1983 .

[5]  V. Milman,et al.  Asymptotic Theory Of Finite Dimensional Normed Spaces , 1986 .

[6]  A. Edelman Eigenvalues and condition numbers of random matrices , 1988 .

[7]  G. Pisier The volume of convex bodies and Banach space geometry , 1989 .

[8]  D. Pollard Empirical Processes: Theory and Applications , 1990 .

[9]  S. Szarek Spaces with large distance to l∞n and random matrices , 1990 .

[10]  Ronald R. Coifman,et al.  Entropy-based algorithms for best basis selection , 1992, IEEE Trans. Inf. Theory.

[11]  B. M. Fulk MATH , 1992 .

[12]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[13]  Ronald R. Coifman,et al.  Signal processing and compression with wavelet packets , 1994 .

[14]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[15]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[16]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[17]  M. Ledoux The concentration of measure phenomenon , 2001 .

[18]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[19]  S. Szarek,et al.  Chapter 8 - Local Operator Theory, Random Matrices and Banach Spaces , 2001 .

[20]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[21]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[23]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[24]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[25]  J. Tropp JUST RELAX: CONVEX PROGRAMMING METHODS FOR SUBSET SELECTION AND SPARSE APPROXIMATION , 2004 .

[26]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[27]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.