Geodesically-convex optimization for averaging partially observed covariance matrices

Symmetric positive definite (SPD) matrices permeates numerous scientific disciplines, including machine learning, optimization, and signal processing. Equipped with a Riemannian geometry, the space of SPD matrices benefits from compelling properties and its derived Riemannian mean is now the gold standard in some applications, e.g. brain-computer interfaces (BCI). This paper addresses the problem of averaging covariance matrices with missing variables. This situation often occurs with inexpensive or unreliable sensors, or when artifact-suppression techniques remove corrupted sensors leading to rank deficient matrices, hindering the use of the Riemannian geometry in covariance-based approaches. An alternate but questionable method consists in removing the matrices with missing variables, thus reducing the training set size. We address those limitations and propose a new formulation grounded in geodesic convexity. Our approach is evaluated on generated datasets with a controlled number of missing variables and a known baseline, demonstrating the robustness of the proposed estimator. The practical interest of this approach is assessed on real BCI datasets. Our results show that the proposed average is more robust and better suited for classification than classical data imputation methods.

[1]  Yuanming Shi,et al.  Geometric mean of partial positive definite matrices with missing entries , 2018, Linear and Multilinear Algebra.

[2]  Kiyoshi Asai,et al.  The em Algorithm for Kernel Matrix Completion with Auxiliary Data , 2003, J. Mach. Learn. Res..

[3]  Masashi Sugiyama,et al.  Averaging covariance matrices for EEG signal classification based on the CSP: An empirical study , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[4]  Christian Jutten,et al.  Single-trial classification of multi-user P300-based Brain-Computer Interface using riemannian geometry , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[5]  Xavier Pennec,et al.  A Riemannian Framework for Tensor Computing , 2005, International Journal of Computer Vision.

[6]  John L.P. Thompson,et al.  Missing data , 2004, Amyotrophic lateral sclerosis and other motor neuron disorders : official publication of the World Federation of Neurology, Research Group on Motor Neuron Diseases.

[7]  P. Thomas Fletcher,et al.  Principal geodesic analysis for the study of nonlinear statistics of shape , 2004, IEEE Transactions on Medical Imaging.

[8]  N. H. Bingham,et al.  Modelling and Prediction of Financial Time Series , 2014 .

[9]  R. Little Robust Estimation of the Mean and Covariance Matrix from Data with Missing Values , 1988 .

[10]  Kazuo Murota,et al.  Exploiting Sparsity in Semidefinite Programming via Matrix Completion I: General Framework , 2000, SIAM J. Optim..

[11]  Anoop Cherian,et al.  Riemannian Dictionary Learning and Sparse Coding for Positive Definite Matrices , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[13]  Maher Moakher,et al.  A Differential Geometric Approach to the Geometric Mean of Symmetric Positive-Definite Matrices , 2005, SIAM J. Matrix Anal. Appl..

[14]  Florian Yger,et al.  Riemannian Classification for SSVEP-Based BCI : Offline versus Online Implementations , 2018 .

[15]  Charles R. Johnson,et al.  Connections between the real positive semidefinite and distance matrix completion problems , 1995 .

[16]  Masashi Sugiyama,et al.  Geometry-aware principal component analysis for symmetric positive definite matrices , 2017, Machine Learning.

[17]  W. Förstner,et al.  A Metric for Covariance Matrices , 2003 .

[18]  Alan Olinsky,et al.  The comparative efficacy of imputation methods for missing data in structural equation modeling , 2003, Eur. J. Oper. Res..

[19]  Marc Arnaudon,et al.  Riemannian Medians and Means With Applications to Radar Signal Processing , 2013, IEEE Journal of Selected Topics in Signal Processing.

[20]  F. Yger,et al.  Riemannian Approaches in Brain-Computer Interfaces: A Review , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Yuxin Chen,et al.  Spectral Compressed Sensing via Structured Matrix Completion , 2013, ICML.

[22]  Masashi Sugiyama,et al.  Supervised LogEuclidean Metric Learning for Symmetric Positive Definite Matrices , 2015, ArXiv.

[23]  Suvrit Sra,et al.  Conic Geometric Optimization on the Manifold of Positive Definite Matrices , 2013, SIAM J. Optim..

[24]  Byron M. Yu,et al.  Deterministic Symmetric Positive Semidefinite Matrix Completion , 2014, NIPS.

[25]  Matthieu Cord,et al.  Riemannian batch normalization for SPD neural networks , 2019, NeurIPS.

[26]  Emmanuel K. Kalunga,et al.  From Euclidean to Riemannian Means: Information Geometry for SSVEP Classification , 2015, GSI.

[27]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[28]  C. Richard Johnson,et al.  Matrix Completion Problems: A Survey , 1990 .

[29]  Vinay Jayaram,et al.  MOABB: trustworthy algorithm benchmarking for BCIs , 2018, Journal of neural engineering.

[30]  Alexandre Barachant,et al.  Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review , 2017 .

[31]  Christian Jutten,et al.  Multiclass Brain–Computer Interface Classification by Riemannian Geometry , 2012, IEEE Transactions on Biomedical Engineering.

[32]  T. Schneider Analysis of Incomplete Climate Data: Estimation of Mean Values and Covariance Matrices and Imputation of Missing Values. , 2001 .

[33]  Monique Laurent,et al.  Polynomial Instances of the Positive Semidefinite and Euclidean Distance Matrix Completion Problems , 2000, SIAM J. Matrix Anal. Appl..

[34]  Karim Lounici High-dimensional covariance matrix estimation with missing observations , 2012, 1201.2577.

[35]  Dong Ming,et al.  Evaluation of EEG Oscillatory Patterns and Cognitive Process during Simple and Compound Limb Motor Imagery , 2014, PloS one.

[36]  Suvrit Sra,et al.  Geometric Mean Metric Learning , 2016, ICML.

[37]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[38]  Niklas Koep,et al.  Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation , 2016, J. Mach. Learn. Res..

[39]  R. Kass,et al.  Shrinkage Estimators for Covariance Matrices , 2001, Biometrics.

[40]  Luc Van Gool,et al.  A Riemannian Network for SPD Matrix Learning , 2016, AAAI.

[41]  Ben Taskar,et al.  Joint covariate selection and joint subspace selection for multiple classification problems , 2010, Stat. Comput..

[42]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[43]  Sunil K. Narang,et al.  Signal processing techniques for interpolation in graph structured data , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  Genevera I. Allen,et al.  Graph quilting: graphical model selection from partially observed covariances. , 2019 .

[45]  Mehrtash Harandi,et al.  Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Emmanuel K. Kalunga,et al.  Review of Riemannian Distances and Divergences, Applied to SSVEP-based BCI , 2020, Neuroinformatics.

[47]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.