Sparse Recovery of Nonnegative Signals With Minimal Expansion

We investigate the problem of reconstructing a high-dimensional nonnegative sparse vector from lower-dimensional linear measurements. While much work has focused on dense measurement matrices, sparse measurement schemes can be more efficient both with respect to signal sensing as well as reconstruction complexity. Known constructions use the adjacency matrices of expander graphs, which often lead to recovery algorithms which are much more efficient than minimization. However, prior constructions of sparse measurement matrices rely on expander graphs with very high expansion coefficients which make the construction of such graphs difficult and the size of the recoverable sets very small. In this paper, we introduce sparse measurement matrices for the recovery of nonnegative vectors, using perturbations of the adjacency matrices of expander graphs requiring much smaller expansion coefficients, hereby referred to as minimal expanders. We show that when minimization is used as the reconstruction method, these constructions allow the recovery of signals that are almost three orders of magnitude larger compared to the existing theoretical results for sparse measurement matrices. We provide for the first time tight upper bounds for the so called weak and strong recovery thresholds when minimization is used. We further show that the success of optimization is equivalent to the existence of a “unique” vector in the set of solutions to the linear equations, which enables alternative algorithms for minimization. We further show that the defined minimal expansion property is necessary for all measurement matrices for compressive sensing, (even when the non-negativity assumption is removed) therefore implying that our construction is tight. We finally present a novel recovery algorithm that exploits expansion and is much more computationally efficient compared to minimization.

[1]  Venkatesan Guruswami,et al.  Euclidean Sections of with Sublinear Randomness and Error-Correction over the Reals , 2008, APPROX-RANDOM.

[2]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[3]  David L. Donoho,et al.  High-Dimensional Centrally Symmetric Polytopes with Neighborliness Proportional to Dimension , 2006, Discret. Comput. Geom..

[4]  James R. Lee,et al.  Almost Euclidean subspaces of e N 1 via expander codes , 2008, SODA 2008.

[5]  Richard G. Baraniuk,et al.  Compressive Sensing DNA Microarrays , 2008, EURASIP J. Bioinform. Syst. Biol..

[6]  Piotr Indyk Explicit constructions for compressed sensing of sparse signals , 2008, SODA '08.

[7]  Richard G. Baraniuk,et al.  Probe Design for Compressive Sensing DNA Microarrays , 2008, 2008 IEEE International Conference on Bioinformatics and Biomedicine.

[8]  Eli Upfal,et al.  Probability and Computing: Randomized Algorithms and Probabilistic Analysis , 2005 .

[9]  B. Hassibi,et al.  Further Results on Performance Analysis for Compressive Sensing Using Expander Graphs , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[10]  Piotr Indyk,et al.  Sparse Recovery Using Sparse Matrices , 2010, Proceedings of the IEEE.

[11]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[12]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[13]  A. Robert Calderbank,et al.  Efficient and Robust Compressed Sensing using High-Quality Expander Graphs , 2008, ArXiv.

[14]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Michael Elad,et al.  On the Uniqueness of Nonnegative Sparse Solutions to Underdetermined Systems of Equations , 2008, IEEE Transactions on Information Theory.

[16]  Yin Zhang Caam When is missing data recoverable ? , 2006 .

[17]  Yi Lu,et al.  Counter Braids: Asymptotic optimality of the message passing decoding algorithm , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[18]  David Burshtein,et al.  Expander graph arguments for message-passing algorithms , 2001, IEEE Trans. Inf. Theory.

[19]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[20]  Venkatesan Guruswami,et al.  Almost Euclidean subspaces of ℓ1N VIA expander codes , 2007, SODA '08.

[21]  Weiyu Xu,et al.  Compressed sensing - probabilistic analysis of a null-space characterization , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[22]  Max Buot Probability and Computing: Randomized Algorithms and Probabilistic Analysis , 2006 .

[23]  Fan Zhang,et al.  On the Iterative Decoding of High-Rate LDPC Codes With Applications in Compressed Sensing , 2009, ArXiv.

[24]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[25]  A. Robert Calderbank,et al.  Efficient and Robust Compressed Sensing Using Optimized Expander Graphs , 2009, IEEE Transactions on Information Theory.

[26]  G. Stewart Perturbation theory for the singular value decomposition , 1990 .

[27]  B. Hassibi,et al.  Compressed sensing over the Grassmann manifold: A unified analytical framework , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[28]  Babak Hassibi,et al.  Recovering Sparse Signals Using Sparse Measurement Matrices in Compressed DNA Microarrays , 2008, IEEE Journal of Selected Topics in Signal Processing.

[29]  P. Indyk,et al.  Near-Optimal Sparse Recovery in the L1 Norm , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[30]  Weiyu Xu,et al.  Efficient Compressive Sensing with Deterministic Guarantees Using Expander Graphs , 2007, 2007 IEEE Information Theory Workshop.

[31]  Piotr Indyk,et al.  Combining geometry and combinatorics: A unified approach to sparse signal recovery , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[32]  A. Bruckstein,et al.  A Non-Negative and Sparse Enough Solution of an Underdetermined Linear System of Equations is Unique , 2007 .

[33]  Daniel A. Spielman,et al.  Expander codes , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[34]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[35]  Nathan Linial,et al.  How Neighborly Can a Centrally Symmetric Polytope Be? , 2006, Discret. Comput. Geom..

[36]  Arkadi Nemirovski,et al.  On sparse representation in pairs of bases , 2003, IEEE Trans. Inf. Theory.

[37]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[38]  Jack K. Wolf,et al.  Redundancy, the Discrete Fourier Transform, and Impulse Noise Cancellation , 1983, IEEE Trans. Commun..