Conic Geometric Optimization on the Manifold of Positive Definite Matrices

We develop geometric optimization on the manifold of Hermitian positive definite (HPD) matrices. In particular, we consider optimizing two types of cost functions: (i) geodesically convex (g-convex) and (ii) log-nonexpansive (LN). G-convex functions are nonconvex in the usual Euclidean sense but convex along the manifold and thus allow global optimization. LN functions may fail to be even g-convex but still remain globally optimizable due to their special structure. We develop theoretical tools to recognize and generate g-convex functions as well as cone theoretic fixed-point optimization algorithms. We illustrate our techniques by applying them to maximum-likelihood parameter estimation for elliptically contoured distributions (a rich class that substantially generalizes the multivariate normal distribution). We compare our fixed-point algorithms with sophisticated manifold optimization methods and obtain notable speedups.

[1]  Rajendra Bhatia,et al.  The matrix geometric mean , 2011 .

[2]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[3]  H. Vincent Poor,et al.  Complex Elliptically Symmetric Distributions: Survey, New Results and Applications , 2012, IEEE Transactions on Signal Processing.

[4]  F. Hiai,et al.  Riemannian metrics on positive definite matrices related to means , 2008, 0809.4974.

[5]  R. Bhatia Positive Definite Matrices , 2007 .

[6]  Bas Lemmens,et al.  Nonlinear Perron-Frobenius Theory , 2012 .

[7]  K. Kraus General state changes in quantum theory , 1971 .

[8]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[9]  Hidegoro Nakano,et al.  Invariant metrics , 1965 .

[10]  lawa Kanas,et al.  Metric Spaces , 2020, An Introduction to Functional Analysis.

[11]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[12]  Constantin P. Niculescu,et al.  Convex Functions and Their Applications: A Contemporary Approach , 2005 .

[13]  Marie Faerber,et al.  Recent Advances In Optimization And Its Applications In Engineering , 2016 .

[14]  S. Sra Positive definite matrices and the Symmetric Stein Divergence , 2011 .

[15]  S. Kotz,et al.  Symmetric Multivariate and Related Distributions , 1989 .

[16]  Stephen P. Boyd,et al.  A tutorial on geometric programming , 2007, Optimization and Engineering.

[17]  Bamdev Mishra,et al.  Manopt, a matlab toolbox for optimization on manifolds , 2013, J. Mach. Learn. Res..

[18]  Jaspal Singh Aujla,et al.  Some inequalities for unitarily invariant norm , 2012 .

[19]  Marc Arnaudon,et al.  Riemannian Medians and Means With Applications to Radar Signal Processing , 2013, IEEE Journal of Selected Topics in Signal Processing.

[20]  Ben Jeuris,et al.  A survey and comparison of contemporary algorithms for computing the matrix geometric mean , 2012 .

[21]  Alfred O. Hero,et al.  Robust Shrinkage Estimation of High-Dimensional Covariance Matrices , 2010, IEEE Transactions on Signal Processing.

[22]  S. Sra Positive definite matrices and the S-divergence , 2011, 1110.1773.

[23]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[24]  R. Karandikar,et al.  Monotonicity of the matrix geometric mean , 2012 .

[25]  S. Sra,et al.  Jensen-Bregman LogDet Divergence for Efficient Similarity Computations on Positive Definite Tensors , 2012 .

[26]  Constantin P. Niculescu Convexity according to the geometric mean , 2000 .

[27]  R. Muirhead Aspects of Multivariate Statistical Theory , 1982, Wiley Series in Probability and Statistics.

[28]  J. Ibrahim,et al.  Statistical Analysis of Diffusion Tensors in Diffusion-Weighted Magnetic Resonance Imaging Data , 2007 .

[29]  Baba C. Vemuri,et al.  A Novel Dynamic System in the Space of SPD Matrices with Applications to Appearance Tracking , 2013, SIAM J. Imaging Sci..

[30]  C. Loan The ubiquitous Kronecker product , 2000 .

[31]  Baba C. Vemuri,et al.  Efficient Recursive Algorithms for Computing the Mean Diffusion Tensor and Applications to DTI Segmentation , 2012, ECCV.

[32]  A. Rukhin Matrix Variate Distributions , 1999, The Multivariate Normal Distribution.

[33]  M. Bridson,et al.  Metric Spaces of Non-Positive Curvature , 1999 .

[34]  Anoop Cherian,et al.  Riemannian Dictionary Learning and Sparse Coding for Positive Definite Matrices , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Maher Moakher,et al.  A Differential Geometric Approach to the Geometric Mean of Symmetric Positive-Definite Matrices , 2005, SIAM J. Matrix Anal. Appl..

[36]  Xingkai Hu,et al.  SOME INEQUALITIES FOR UNITARILY INVARIANT NORMS , 2012 .

[37]  N. L. Johnson,et al.  SERIES REPRESENTATIONS OF DISTRIBUTIONS OF QUADRATIC FORMS IN NORMAL VARIABLES, I. CENTRAL CASE, , 1967 .

[38]  A. Papadopoulos Metric Spaces, Convexity and Nonpositive Curvature , 2004 .

[39]  Chi-Kwong Li Geometric Means , 2003 .

[40]  T. Andô,et al.  Means of positive linear operators , 1980 .

[41]  Dario Bini,et al.  Computing the Karcher mean of symmetric positive definite matrices , 2013 .

[42]  Pierre-Antoine Absil,et al.  Riemannian BFGS Algorithm with Applications , 2010 .

[43]  Man-Duen Choi Completely positive linear maps on complex matrices , 1975 .

[44]  G. Simons,et al.  On the theory of elliptically contoured distributions , 1981 .

[45]  P. Krishnaiah,et al.  Complex elliptically symmetric distributions , 1986 .

[46]  M. Edelstein On Fixed and Periodic Points Under Contractive Mappings , 1962 .

[47]  Alex Samorodnitsky,et al.  A Deterministic Algorithm for Approximating the Mixed Discriminant and Mixed Volume, and a Combinatorial Corollary , 2002, Discret. Comput. Geom..

[48]  Y. Lim,et al.  Matrix power means and the Karcher mean , 2012 .

[49]  Y. Lim,et al.  Invariant metrics, contractions and nonlinear matrix equations , 2008 .

[50]  Ami Wiesel,et al.  Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models , 2013, IEEE Transactions on Signal Processing.

[51]  Frank Nielsen,et al.  Matrix Information Geometry , 2012 .

[52]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[53]  T. Andô Concavity of certain maps on positive definite matrices and applications to Hadamard products , 1979 .

[54]  David E. Tyler,et al.  Redescending $M$-Estimates of Multivariate Location and Scatter , 1991 .

[55]  Anoop Cherian,et al.  Generalized Dictionary Learning for Symmetric Positive Definite Matrices with Application to Nearest Neighbor Retrieval , 2011, ECML/PKDD.

[56]  Maher Moakher,et al.  Means of Hermitian positive-definite matrices based on the log-determinant α-divergence function , 2012 .

[57]  Ami Wiesel,et al.  Geodesic Convexity and Covariance Estimation , 2012, IEEE Transactions on Signal Processing.

[58]  T. Rapcsák Geodesic convexity in nonlinear optimization , 1991 .

[59]  Benedikt Wirth,et al.  Optimization Methods on Riemannian Manifolds and Their Application to Shape Space , 2012, SIAM J. Optim..

[60]  Rekha R. Thomas,et al.  Semidefinite Optimization and Convex Algebraic Geometry , 2012 .