The incredible shrinking covariance estimator

Covariance estimation is a key step in many target detection algorithms. To distinguish target from background requires that the background be well-characterized. This applies to targets ranging from the precisely known chemical signatures of gaseous plumes to the wholly unspecified signals that are sought by anomaly detectors. When the background is modelled by a (global or local) Gaussian or other elliptically contoured distribution (such as Laplacian or multivariate-t), a covariance matrix must be estimated. The standard sample covariance overfits the data, and when the training sample size is small, the target detection performance suffers. Shrinkage addresses the problem of overfitting that inevitably arises when a high-dimensional model is fit from a small dataset. In place of the (overfit) sample covariance matrix, a linear combination of that covariance with a fixed matrix is employed. The fixed matrix might be the identity, the diagonal elements of the sample covariance, or some other underfit estimator. The idea is that the combination of an overfit with an underfit estimator can lead to a well-fit estimator. The coefficient that does this combining, called the shrinkage parameter, is generally estimated by some kind of cross-validation approach, but direct cross-validation can be computationally expensive. This paper extends an approach suggested by Hoffbeck and Landgrebe, and presents efficient approximations of the leave-one-out cross-validation (LOOC) estimate of the shrinkage parameter used in estimating the covariance matrix from a limited sample of data.

[1]  Anne H. Schistad Solberg,et al.  Regression Approaches to Small Sample Inverse Covariance Matrix Estimation for Hyperspectral Image Classification , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Charles A. Bouman,et al.  Covariance Estimation for High Dimensional Data Vectors Using the Sparse Matrix Transform , 2008, NIPS.

[3]  Charles A. Bouman,et al.  Sparse Matrix Transform for Hyperspectral Image Processing , 2011, IEEE Journal of Selected Topics in Signal Processing.

[4]  S.R. Rotman,et al.  Improved covariance matrices for point target detection in hyperspectral data , 2008, 2009 IEEE International Conference on Microwaves, Communications, Antennas and Electronics Systems.

[5]  Don R. Hush,et al.  Statistics for characterizing data on the periphery , 2010, 2010 IEEE International Geoscience and Remote Sensing Symposium.

[6]  David A. Landgrebe,et al.  Covariance estimation with limited training samples , 1999, IEEE Trans. Geosci. Remote. Sens..

[7]  Charles E. Davidson,et al.  Estimation of hyperspectral covariance matrices , 2011, 2011 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[8]  Anne H. Schistad Solberg,et al.  Sparse Inverse Covariance Estimates for Hyperspectral Image Classification , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Olivier Ledoit,et al.  Improved estimation of the covariance matrix of stock returns with an application to portfolio selection , 2003 .

[10]  Charles A. Bouman,et al.  Weak signal detection in hyperspectral imagery using sparse matrix transform (smt) covariance estimation , 2009, 2009 First Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[11]  N. Campbell Robust Procedures in Multivariate Analysis I: Robust Covariance Estimation , 1980 .

[12]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[13]  R. Kass,et al.  Shrinkage Estimators for Covariance Matrices , 2001, Biometrics.

[14]  Wallace M. Porter,et al.  The airborne visible/infrared imaging spectrometer (AVIRIS) , 1993 .

[15]  Stefania Matteoli,et al.  Improved estimation of local background covariance matrix for anomaly detection in hyperspectral images , 2010 .

[16]  David I. Warton,et al.  Penalized Normal Likelihood and Ridge Regularization of Correlation and Covariance Matrices , 2008 .

[17]  Avishai Ben-David,et al.  Performance loss of multivariate detection algorithms due to covariance estimation , 2009, Remote Sensing.

[18]  K. Strimmer,et al.  Statistical Applications in Genetics and Molecular Biology A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics , 2011 .

[19]  David A. Landgrebe,et al.  Covariance Matrix Estimation and Classification With Limited Training Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  J. Friedman Regularized Discriminant Analysis , 1989 .

[21]  Bor-Chen Kuo,et al.  Localized shrinkage covariance estimation of hyperspectral image classification , 2009, 2009 IEEE International Geoscience and Remote Sensing Symposium.

[22]  Nasser M. Nasrabadi,et al.  Regularization for spectral matched filter and RX anomaly detector , 2008, SPIE Defense + Commercial Sensing.