Low-Rank Riemannian Optimization on Positive Semidefinite Stochastic Matrices with Applications to Graph Clustering

Ahmed Douik 1 Babak Hassibi 1 Abstract This paper develops a Riemannian optimization framework for solving optimization problems on the set of symmetric positive semidefinite stochastic matrices. The paper first reformulates the problem by factorizing the optimization variable as X = YY and deriving conditions on p, i.e., the number of columns of Y, under which the factorization yields a satisfactory solution. The reparameterization of the problem allows its formulation as an optimization over either an embedded or quotient Riemannian manifold whose geometries are investigated. In particular, the paper explicitly derives the tangent space, Riemannian gradients and retraction operator that allow the design of efficient optimization methods on the proposed manifolds. The numerical results reveal that, when the optimal solution has a known low-rank, the resulting algorithms present a clear complexity advantage when compared with stateof-the-art Euclidean and Riemannian approaches for graph clustering applications.

[1]  Maya R. Gupta,et al.  Clustering by Left-Stochastic Matrix Factorization , 2011, ICML.

[2]  J. Csima,et al.  The DAD Theorem for Symmetric Non-negative Matrices , 1972, J. Comb. Theory, Ser. A.

[3]  Francis R. Bach,et al.  Low-Rank Optimization on the Cone of Positive Semidefinite Matrices , 2008, SIAM J. Optim..

[4]  Silvere Bonnabel,et al.  Riemannian Metric and Geometric Mean for Positive Semidefinite Matrices of Fixed Rank , 2008, SIAM J. Matrix Anal. Appl..

[5]  Marcus Peinado,et al.  Design and Performance of Parallel and Distributed Approximation Algorithms for Maxcut , 1997, J. Parallel Distributed Comput..

[6]  Amnon Shashua,et al.  Doubly Stochastic Normalization for Spectral Clustering , 2006, NIPS.

[7]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[8]  Marina Meila,et al.  Comparing Clusterings by the Variation of Information , 2003, COLT.

[9]  Erkki Oja,et al.  Clustering by Low-Rank Doubly Stochastic Matrix Decomposition , 2012, ICML.

[10]  Igor Grubisic,et al.  Efficient Rank Reduction of Correlation Matrices , 2007 .

[11]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[12]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[13]  Pierre-Antoine Absil,et al.  Robust Low-Rank Matrix Completion by Riemannian Optimization , 2016, SIAM J. Sci. Comput..

[14]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[15]  Renato D. C. Monteiro,et al.  A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization , 2003, Math. Program..

[16]  Bart Vandereycken,et al.  Low-Rank Matrix Completion by Riemannian Optimization , 2013, SIAM J. Optim..

[17]  Richard Sinkhorn A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices , 1964 .

[18]  Fei-Fei Li,et al.  Novel Dataset for Fine-Grained Image Categorization : Stanford Dogs , 2012 .

[19]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Franz Rendl,et al.  A Spectral Bundle Method for Semidefinite Programming , 1999, SIAM J. Optim..

[21]  Babak Hassibi,et al.  Manifold Optimization Over the Set of Doubly Stochastic Matrices: A Second-Order Geometry , 2018, IEEE Transactions on Signal Processing.

[22]  Babak Hassibi,et al.  Similarity clustering in the presence of outliers: Exact recovery via convex program , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[23]  Pierre-Antoine Absil,et al.  RTRMC: A Riemannian trust-region method for low-rank matrix completion , 2011, NIPS.

[24]  Erkki Oja,et al.  Unified Development of Multiplicative Algorithms for Linear and Quadratic Nonnegative Matrix Factorization , 2011, IEEE Transactions on Neural Networks.

[25]  Bamdev Mishra,et al.  Manopt, a matlab toolbox for optimization on manifolds , 2013, J. Mach. Learn. Res..