Correlation matrix nearness and completion under observation uncertainty

This paper introduces the paradigm of optimization under uncertainty for modelling and solving matrix nearness problems. In particular, it considers the concrete problem of recovering correlation matrices from uncertain observations by introducing two different approaches to tackling uncertainty. The first approach invokes the framework of robust optimization to construct low error solutions that are immune to worst-case uncertainty in the input. The second approach takes a less pessimistic view on uncertainty, and considers a situation where instead of the worst one, it suffices to use any matrix in the uncertainty set. We formulate both our approaches as convex (possibly nonsmooth) optimization problems. Thereafter, we show how to exploit problem structure to obtain efficient iterative first-order algorithms. We present several numerical results on both nearness and completion versions of the optimization problem; our results highlight the effectiveness of our proposed algorithmic techniques.

[1]  D. Bertsimas,et al.  Robust and Data-Driven Optimization: Modern Decision-Making Under Uncertainty , 2006 .

[2]  Jean-Philippe Vial,et al.  Robust Optimization , 2021, ICORES.

[3]  H. Qi,et al.  An augmented Lagrangian dual approach for the H-weighted nearest correlation matrix problem , 2011 .

[4]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[5]  R. Rockafellar Monotone Operators and the Proximal Point Algorithm , 1976 .

[6]  Nicholas J. Higham,et al.  A Preconditioned Newton Algorithm for the Nearest Correlation Matrix , 2010 .

[7]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[8]  Guoyin Li,et al.  Robust least square semidefinite programming with applications , 2014, Comput. Optim. Appl..

[9]  Defeng Sun,et al.  Correlation stress testing for value-at-risk: an unconstrained convex optimization approach , 2010, Comput. Optim. Appl..

[10]  J. Moreau Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[11]  James E. Gentle,et al.  Matrix Algebra: Theory, Computations, and Applications in Statistics , 2007 .

[12]  N. Higham Computing the nearest correlation matrix—a problem from finance , 2002 .

[13]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[14]  George B. Dantzig,et al.  Linear Programming Under Uncertainty , 2004, Manag. Sci..

[15]  N. Higham MATRIX NEARNESS PROBLEMS AND APPLICATIONS , 1989 .

[16]  Vineer Bhansali,et al.  Forecasting Portfolio Risk in Normal and Stressed Markets , 2001, nlin/0108022.

[17]  Defeng Sun,et al.  A Quadratically Convergent Newton Method for Computing the Nearest Correlation Matrix , 2006, SIAM J. Matrix Anal. Appl..

[18]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .