Optimization and estimation on manifolds

How to make the best decision? This general concern, pervasive in both research and industry, is what optimization is all about. Optimization is a field of applied mathematics concerned with making the best use—according to some quantitative criterion called the cost function—of our degrees of freedom called the variables, possibly under some constraints. Optimization problems come in various forms. We consider continuous variables with differentiable cost functions. Furthermore, and this is central to our investigation, we assume that the variables are constrained to belong to a Riemannian manifold, that is, to a smooth space. Building upon prior theory, we develop Manopt, a toolbox which considerably simplifies the use of Riemannian optimization. We apply this tool to two applications. First, we study low-rank matrix completion, which appears in recommender systems. Such systems aim at predicting which movies, books, etc. different users might appreciate, based on partial knowledge of their preferences. Second, we study synchronization of rotations. This is a central player in the reconstruction of 3D computer models of physical objects based on scans of their surface. In both cases, Riemannian optimization provides competitive, scalable and accurate algorithms. Both applications constitute estimation problems. In estimation, one wishes to determine the value of unknown parameters based on noisy measurements. We address the following fundamental question: given a noise level on the measurements, how accurately can one hope to estimate the parameters? This prompts us to further develop Cramer-Rao bounds when the parameter space is a manifold. Applied to synchronization, these bounds bring about practical implications. First, they suggest that in many nontrivial scenarios, our estimation algorithm could be optimal. Second, they reveal the defining features that make a synchronization task more or less difficult, hinting at which measurements should be acquired.