Multidimensional compressed sensing and their applications

Compressed sensing (CS) comprises a set of relatively new techniques that exploit the underlying structure of data sets allowing their reconstruction from compressed versions or incomplete information. CS reconstruction algorithms are essentially nonlinear, demanding heavy computation overhead and large storage memory, especially in the case of multidimensional signals. Excellent review papers discussing CS state‐of‐the‐art theory and algorithms already exist in the literature, which mostly consider data sets in vector forms. In this paper, we give an overview of existing techniques with special focus on the treatment of multidimensional signals (tensors). We discuss recent trends that exploit the natural multidimensional structure of signals (tensors) achieving simple and efficient CS algorithms. The Kronecker structure of dictionaries is emphasized and its equivalence to the Tucker tensor decomposition is exploited allowing us to use tensor tools and models for CS. Several examples based on real world multidimensional signals are presented, illustrating common problems in signal processing such as the recovery of signals from compressed measurements for magnetic resonance imaging (MRI) signals or for hyper‐spectral imaging, and the tensor completion problem (multidimensional inpainting). WIREs Data Mining Knowl Discov 2013, 3:355–380. doi: 10.1002/widm.1108

[1]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[2]  Yonina C. Eldar,et al.  Structured Compressed Sensing: From Theory to Applications , 2011, IEEE Transactions on Signal Processing.

[3]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[4]  Robert D. Nowak,et al.  Majorization–Minimization Algorithms for Wavelet-Based Image Restoration , 2007, IEEE Transactions on Image Processing.

[5]  Morten Mørup,et al.  Applications of tensor (multiway array) factorizations and decompositions in data mining , 2011, WIREs Data Mining Knowl. Discov..

[6]  Peter Boesiger,et al.  Compressed sensing in dynamic MRI , 2008, Magnetic resonance in medicine.

[7]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Akram Aldroubi,et al.  Optimal Non-Linear Models for Sparsity and Sampling , 2007, 0707.2008.

[9]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[10]  V. Mehrmann,et al.  Sparse solutions to underdetermined Kronecker product systems , 2009 .

[11]  Stéphane Mallat,et al.  Sparse geometric image representations with bandelets , 2005, IEEE Transactions on Image Processing.

[12]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[14]  Robert D. Nowak,et al.  Toeplitz Compressed Sensing Matrices With Applications to Sparse Channel Estimation , 2010, IEEE Transactions on Information Theory.

[15]  Minh N. Do,et al.  A New Contourlet Transform with Sharp Frequency Localization , 2006, 2006 International Conference on Image Processing.

[16]  Lasse Borup,et al.  Nonlinear approximation with redundant dictionaries , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[17]  Mohamed-Jalal Fadili,et al.  Inpainting and Zooming Using Sparse Representations , 2009, Comput. J..

[18]  Justin K. Romberg,et al.  Restricted Isometries for Partial Random Circulant Matrices , 2010, ArXiv.

[19]  J. Landsberg Tensors: Geometry and Applications , 2011 .

[20]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[21]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[22]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[23]  Bernhard Schölkopf,et al.  Optimization of k‐space trajectories for compressed sensing by Bayesian experimental design , 2010, Magnetic resonance in medicine.

[24]  Justin Dauwels,et al.  Handling missing data in medical questionnaires using tensor decompositions , 2011, 2011 8th International Conference on Information, Communications & Signal Processing.

[25]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[26]  F. Rybicki,et al.  Fast, exact k‐space sample density compensation for trajectories composed of rotationally symmetric segments, and the SNR‐optimized image reconstruction from non‐Cartesian samples , 2008, Magnetic resonance in medicine.

[27]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[28]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[29]  A. Stern,et al.  Random Projections Imaging With Extended Space-Bandwidth Product , 2007, Journal of Display Technology.

[30]  T. Blumensath,et al.  Theory and Applications , 2011 .

[31]  Yoram Bresler,et al.  Learning Sparsifying Transforms , 2013, IEEE Transactions on Signal Processing.

[32]  Andrzej Cichocki,et al.  Block sparse representations of tensors using Kronecker bases , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[34]  Yonina C Eldar,et al.  Super-resolution and reconstruction of sparse sub-wavelength images. , 2009, Optics express.

[35]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[36]  Florian Roemer,et al.  Higher-Order SVD-Based Subspace Estimation to Improve the Parameter Estimation Accuracy in Multidimensional Harmonic Retrieval Problems , 2008, IEEE Transactions on Signal Processing.

[37]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[38]  Stéphane Mallat,et al.  Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity , 2010, IEEE Transactions on Image Processing.

[39]  Guangdong Feng,et al.  A Tensor Based Method for Missing Traffic Data Completion , 2013 .

[40]  Erik G. Larsson,et al.  Lecture Notes: Floating-Point Numbers , 2010 .

[41]  Michael Elad,et al.  Learning Multiscale Sparse Representations for Image and Video Restoration , 2007, Multiscale Model. Simul..

[42]  Fumikazu Miwakeichi,et al.  Decomposing EEG data into space–time–frequency components using Parallel Factor Analysis , 2004, NeuroImage.

[43]  Trac D. Tran,et al.  Fast compressive sampling with structurally random matrices , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  Robert D. Nowak,et al.  Wavelet-based statistical signal processing using hidden Markov models , 1998, IEEE Trans. Signal Process..

[45]  Adrian Stern,et al.  Compressed Imaging With a Separable Sensing Operator , 2009, IEEE Signal Processing Letters.

[46]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part I: theory , 2006, IEEE Transactions on Image Processing.

[47]  E. Candès,et al.  Ridgelets: a key to higher-dimensional intermittency? , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[48]  Andrzej Cichocki,et al.  Computing Sparse Representations of Multidimensional Signals Using Kronecker Bases , 2013, Neural Computation.

[49]  S. Mallat A wavelet tour of signal processing , 1998 .

[50]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[51]  Michael Elad,et al.  Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit , 2008 .

[52]  Adrian Stern,et al.  Compressed imaging system with linear sensors. , 2007, Optics letters.

[53]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[54]  R. Fergus,et al.  Random Lens Imaging , 2006 .

[55]  José M. Bioucas-Dias,et al.  Bayesian wavelet-based image deconvolution: a GEM algorithm exploiting a class of heavy-tailed priors , 2006, IEEE Transactions on Image Processing.

[56]  Pierre Comon,et al.  Multiarray Signal Processing: Tensor decomposition meets compressed sensing , 2010, ArXiv.

[57]  Muhammad Tayyab Asif,et al.  Low-dimensional models for missing data imputation in road networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[58]  Emmanuel J. Candès,et al.  The curvelet transform for image denoising , 2002, IEEE Trans. Image Process..

[59]  Nikos D. Sidiropoulos,et al.  Tensor Algebra and Multidimensional Harmonic Retrieval in Signal Processing for MIMO Radar , 2010, IEEE Transactions on Signal Processing.

[60]  Karin Schnass,et al.  Dictionary Identification—Sparse Matrix-Factorization via $\ell_1$ -Minimization , 2009, IEEE Transactions on Information Theory.

[61]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[62]  Justin Dauwels,et al.  Tensor factorization for missing data imputation in medical questionnaires , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[63]  Tamara G. Kolda,et al.  Multilinear Algebra for Analyzing Data with Multiple Linkages , 2006, Graph Algorithms in the Language of Linear Algebra.

[64]  Mathews Jacob,et al.  Accelerated Dynamic MRI Exploiting Sparsity and Low-Rank Structure: k-t SLR , 2011, IEEE Transactions on Medical Imaging.

[65]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[66]  Xuelong Li,et al.  Greedy regression in sparse coding space for single-image super-resolution , 2013, J. Vis. Commun. Image Represent..

[67]  Tamara G. Kolda,et al.  Scalable Tensor Factorizations for Incomplete Data , 2010, ArXiv.

[68]  Stephen J. Wright,et al.  Computational Methods for Sparse Solution of Linear Inverse Problems , 2010, Proceedings of the IEEE.

[69]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  Deanna Needell,et al.  Signal Recovery From Incomplete and Inaccurate Measurements Via Regularized Orthogonal Matching Pursuit , 2007, IEEE Journal of Selected Topics in Signal Processing.

[71]  Jesús Grajal,et al.  Adaptive-FRESH Filters for Compensation of Cycle-Frequency Errors , 2010, IEEE Transactions on Signal Processing.

[72]  David V. Anderson,et al.  Compressive Sensing on a CMOS Separable-Transform Image Sensor , 2010, Proceedings of the IEEE.

[73]  David L Wilson,et al.  A simple application of compressed sensing to further accelerate partially parallel imaging. , 2013, Magnetic resonance imaging.

[74]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions , 1994 .

[75]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[76]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[77]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[78]  Wai Lam Chan,et al.  A single-pixel terahertz imaging system based on compressed sensing , 2008 .

[79]  Rebecca Willett,et al.  Compressive coded aperture superresolution image reconstruction , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[80]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[81]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[82]  Høgskolen i Stavanger FRAME DESIGN USING FOCUSS WITH METHOD OF OPTIMAL DIRECTIONS (MOD) , 2000 .

[83]  Armando Manduca,et al.  Highly Undersampled Magnetic Resonance Image Reconstruction via Homotopic $\ell_{0}$ -Minimization , 2009, IEEE Transactions on Medical Imaging.

[84]  Babak Hossein Khalaj,et al.  A unified approach to sparse signal processing , 2009, EURASIP Journal on Advances in Signal Processing.

[85]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.

[86]  Michael Elad,et al.  Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation , 2010, IEEE Transactions on Signal Processing.

[87]  D. Donoho Wedgelets: nearly minimax estimation of edges , 1999 .

[88]  A. Majumdar Improved dynamic MRI reconstruction by exploiting sparsity and rank-deficiency. , 2013, Magnetic resonance imaging.

[89]  Albert P. Chen,et al.  Compressed sensing for resolution enhancement of hyperpolarized 13C flyback 3D-MRSI. , 2008, Journal of magnetic resonance.

[90]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[91]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[92]  Anastasios Kyrillidis,et al.  Multi-Way Compressed Sensing for Sparse Low-Rank Tensors , 2012, IEEE Signal Processing Letters.

[93]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[94]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[95]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[97]  Florian Roemer,et al.  Iterative Sequential GSVD (I-S-GSVD) based prewhitening for multidimensional HOSVD based subspace estimation without knowledge of the noise covariance information , 2010, 2010 International ITG Workshop on Smart Antennas (WSA).

[98]  Onur G. Guleryuz Nonlinear approximation based image recovery using adaptive sparse reconstructions , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[99]  Michael Elad,et al.  The Cosparse Analysis Model and Algorithms , 2011, ArXiv.

[100]  D. Foster,et al.  Frequency of metamerism in natural scenes. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[101]  Holger Rauhut,et al.  Circulant and Toeplitz matrices in compressed sensing , 2009, ArXiv.

[102]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[103]  Richard G. Baraniuk,et al.  Kronecker Compressive Sensing , 2012, IEEE Transactions on Image Processing.

[104]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[105]  Michael P. Friedlander,et al.  Sparse Optimization with Least-Squares Constraints , 2011, SIAM J. Optim..

[106]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.