Sparse Covariance Estimation from Quadratic Measurements: A Precise Analysis

We study the problem of estimating a high-dimensional sparse covariance matrix, Σ0, from a finite number of quadratic measurements, i.e., measurements ${\text{a}}_i^T{\Sigma _0}{{\text{a}}_i}$ which are quadratic forms in the measurement vectors ai resulting from the covariance matrix, Σ0. Such a problem arises in applications where we can only make energy measurements of the underlying random variables. We study a simple LASSO-like convex recovery algorithm which involves a squared 2-norm (to match the covariance estimate to the measurements), plus a regularization term (that penalizes the ℓ1−norm of the non-diagonal entries of Σ0 to enforce sparsity). When the measurement vectors are i.i.d. Gaussian, we obtain the precise error performance of the algorithm (accurately determining the estimation error in any metric, e.g., 2-norm, operator norm, etc.) as a function of the number of measurements and the underlying distribution of Σ0. In particular, in the noiseless case we determine the necessary and sufficient number of measurements required to perfectly recover Σ0 as a function of its sparsity. Our results rely on a novel comparison lemma which relates a convex optimization problem with "quadratic Gaussian" measurements to one which has i.i.d. Gaussian measurements.

[1]  Andrea J. Goldsmith,et al.  Exact and Stable Covariance Estimation From Quadratic Sampling via Convex Programming , 2013, IEEE Transactions on Information Theory.

[2]  Christos Thrampoulidis,et al.  Ber analysis of the box relaxation for BPSK signal recovery , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[4]  Mustafa Khammash,et al.  Parameter Estimation and Model Selection in Computational Biology , 2010, PLoS Comput. Biol..

[5]  S. Muthukrishnan,et al.  Data streams: algorithms and applications , 2005, SODA '03.

[6]  Babak Hassibi,et al.  Performance Analysis of Convex Data Detection in MIMO , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Babak Hassibi,et al.  A Universal Analysis of Large-Scale Regularized Least Squares Solutions , 2017, NIPS.

[8]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[9]  Parikshit Shah,et al.  Sketching Sparse Matrices, Covariances, and Graphs via Tensor Products , 2015, IEEE Transactions on Information Theory.

[10]  Alexander Jung,et al.  Compressive Spectral Estimation for Nonstationary Random Processes , 2009, IEEE Transactions on Information Theory.

[11]  Christos Thrampoulidis,et al.  General performance metrics for the LASSO , 2016, 2016 IEEE Information Theory Workshop (ITW).

[12]  Jianqing Fan,et al.  High-Frequency Covariance Estimates With Noisy and Asynchronous Financial Data , 2010 .

[13]  Christos Thrampoulidis,et al.  Precise Error Analysis of Regularized $M$ -Estimators in High Dimensions , 2016, IEEE Transactions on Information Theory.

[14]  Louis L. Scharf,et al.  Matched subspace detectors , 1994, IEEE Trans. Signal Process..

[15]  Christos Thrampoulidis,et al.  Phase Retrieval via Polytope Optimization: Geometry, Phase Transitions, and New Algorithms , 2018, ArXiv.

[16]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[17]  Christos Thrampoulidis,et al.  Symbol Error Rate Performance of Box-Relaxation Decoders in Massive MIMO , 2018, IEEE Transactions on Signal Processing.

[18]  Joel A. Tropp,et al.  Universality laws for randomized dimension reduction, with applications , 2015, ArXiv.

[19]  Babak Hassibi,et al.  A Precise Analysis of PhaseMax in Phase Retrieval , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).