Uncertainty principles and ideal atomic decomposition

Suppose a discrete-time signal S(t), 0/spl les/t<N, is a superposition of atoms taken from a combined time-frequency dictionary made of spike sequences 1/sub {t=/spl tau/}/ and sinusoids exp{2/spl pi/iwt/N}//spl radic/N. Can one recover, from knowledge of S alone, the precise collection of atoms going to make up S? Because every discrete-time signal can be represented as a superposition of spikes alone, or as a superposition of sinusoids alone, there is no unique way of writing S as a sum of spikes and sinusoids in general. We prove that if S is representable as a highly sparse superposition of atoms from this time-frequency dictionary, then there is only one such highly sparse representation of S, and it can be obtained by solving the convex optimization problem of minimizing the l/sup 1/ norm of the coefficients among all decompositions. Here "highly sparse" means that N/sub t/+N/sub w/</spl radic/N/2 where N/sub t/ is the number of time atoms, N/sub w/ is the number of frequency atoms, and N is the length of the discrete-time signal. Underlying this result is a general l/sup 1/ uncertainty principle which says that if two bases are mutually incoherent, no nonzero signal can have a sparse representation in both bases simultaneously. For the above setting, the bases are sinusoids and spikes, and mutual incoherence is measured in terms of the largest inner product between different basis elements. The uncertainty principle holds for a variety of interesting basis pairs, not just sinusoids and spikes. The results have idealized applications to band-limited approximation with gross errors, to error-correcting encryption, and to separation of uncoordinated sources. Related phenomena hold for functions of a real variable, with basis pairs such as sinusoids and wavelets, and for functions of two variables, with basis pairs such as wavelets and ridgelets. In these settings, if a function f is representable by a sufficiently sparse superposition of terms taken from both bases, then there is only one such sparse representation; it may be obtained by minimum l/sup 1/ norm atomic decomposition. The condition "sufficiently sparse" becomes a multiscale condition; for example, that the number of wavelets at level j plus the number of sinusoids in the jth dyadic frequency band are together less than a constant times 2/sup j/2/.

[1]  AARON D. WYNER An analog scrambling scheme which does not expand bandwidth, Part II: Continuous time , 1979, IEEE Trans. Inf. Theory.

[2]  David N. Williams New mathematical proof of the uncertainty relation , 1979 .

[3]  Aaron D. Wyner An analog scrambling scheme which does not expand bandwidth, Part I: Discrete time , 1979, IEEE Trans. Inf. Theory.

[4]  N. J. A. Sloane,et al.  Encrypting by Random Rotations , 1982, EUROCRYPT.

[5]  A. Spanias,et al.  Accurate representation of time-varying signals using mixed transforms with applications to speech , 1989 .

[6]  D. Donoho,et al.  Uncertainty principles and signal recovery , 1989 .

[7]  P. Yip,et al.  Discrete Cosine Transform: Algorithms, Advantages, Applications , 1990 .

[8]  I. Daubechies,et al.  A simple Wilson orthonormal basis with exponential decay , 1991 .

[9]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[10]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[11]  Ronald R. Coifman,et al.  Adapted wave form analysis, wavelet-packets and applications , 1992 .

[12]  B. Logan,et al.  Signal recovery and the large sieve , 1992 .

[13]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[14]  G. Weiss,et al.  Local sine and cosine bases of Coifman and Meyer and the construction of smooth wavelets , 1993 .

[15]  Yves Meyer Wavelets - algorithms & applications , 1993 .

[16]  D. Donoho,et al.  Basis pursuit , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[17]  E. Kolaczyk Wavelet Methods For The Inversion Of Certain Homogeneous Linear Operators In The Presence Of Noisy D , 1994 .

[18]  Marjorie G. Hahn Review: Michel Ledoux, Michel Talagrand, Probability in Banach Spaces: Isoperimetry and Processes , 1994 .

[19]  M. Victor Wickerhauser,et al.  Adapted wavelet analysis from theory to software , 1994 .

[20]  Ronald A. DeVore,et al.  Some remarks on greedy algorithms , 1996, Adv. Comput. Math..

[21]  Alan S. Stern,et al.  NMR Data Processing , 1996 .

[22]  Victor E. DeBrunner,et al.  Lapped multiple bases algorithms for still image compression without blocking effect , 1997, IEEE Trans. Image Process..

[23]  V. DeBrunner,et al.  Multiple transform algorithms for time-varying signal representation , 1997 .

[24]  G. Folland,et al.  The uncertainty principle: A mathematical survey , 1997 .

[25]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[26]  Margaret H. Wright,et al.  The interior-point revolution in constrained optimization , 1998 .

[27]  S. Mallat A wavelet tour of signal processing , 1998 .

[28]  Wasfy B. Mikhael,et al.  A survey of mixed transform techniques for speech and image coding , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[29]  Xiaoming Huo,et al.  Sparse image representation via combined transforms , 1999 .

[30]  Victor E. DeBrunner,et al.  Using a new uncertainty measure to determine optimal bases for signal representations , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[31]  E. Candès,et al.  Ridgelets: a key to higher-dimensional intermittency? , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[32]  David L. Donoho,et al.  Orthonormal Ridgelets and Linear Singularities , 2000, SIAM J. Math. Anal..

[33]  C. Chui Adapted Wavelet Analysis from Theory to Software , 2022 .