ROI: An Extensible R Optimization Infrastructure

Optimization plays an important role in many methods routinely used in statistics, machine learning and data science. Often, implementations of these methods rely on highly specialized optimization algorithms, designed to be only applicable within a specific application. However, in many instances recent advances, in particular in the field of convex optimization, make it possible to conveniently and straightforwardly use modern solvers instead with the advantage of enabling broader usage scenarios and thus promoting reusability. This paper introduces the R Optimization Infrastructure which provides an extensible infrastructure to model linear, quadratic, conic and general nonlinear optimization problems in a consistent way. Furthermore, the infrastructure administers many different solvers, reformulations, problem collections and functions to read and write optimization problems in various formats.

[1]  H. M. Wagner Linear Programming Techniques for Regression Analysis , 1959 .

[2]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..

[3]  R. Gomory AN ALGORITHM FOR THE MIXED INTEGER PROBLEM , 1960 .

[4]  J. Meeraus A. Bisschop,et al.  ON THE DEVELOPMENT OF A GENERAL ALGEBRAIC MODELING SYSTEM IN A STRATEGIC PLANNING ENVIRONMENT , 1982 .

[5]  Donald Goldfarb,et al.  A numerically stable dual method for solving strictly convex quadratic programs , 1983, Math. Program..

[6]  R. Dykstra An Algorithm for Restricted Least Squares Regression , 1983 .

[7]  Aharon Ben-Tal,et al.  Lectures on modern convex optimization , 1987 .

[8]  Brian W. Kernighan,et al.  AMPL: a mathematical programming language , 1989 .

[9]  Stein W. Wallace Algorithms and model formulations in mathematical programming , 1989 .

[10]  A. Atkinson Subset Selection in Regression , 1992 .

[11]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[12]  Michel X. Goemans,et al.  Semidefinite programming in combinatorial optimization , 1997, Math. Program..

[13]  William Gropp,et al.  Optimization environments and the NEOS server , 1997 .

[14]  Jorge J. Moré,et al.  The NEOS Server , 1998 .

[15]  B. Borchers CSDP, A C library for semidefinite programming , 1999 .

[16]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[17]  Katya Scheinberg,et al.  Efficient SVM Training Using Low-Rank Kernel Representations , 2002, J. Mach. Learn. Res..

[18]  Elizabeth D. Dolan,et al.  NEOS Server 4.0 Administrative Guide , 2001, ArXiv.

[19]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[20]  Endre Boros,et al.  Pseudo-Boolean optimization , 2002, Discret. Appl. Math..

[21]  Donald Goldfarb,et al.  Second-order cone programming , 2003, Math. Program..

[22]  Josef Kallrath Mathematical Optimization and the Role of Modeling Languages , 2004 .

[23]  R. Freund Introduction to Semidefinite Programming ( SDP ) , 2004 .

[24]  Alan J. Miller,et al.  leaps: Regression Subset Selection. , 2004 .

[25]  Kurt Hornik,et al.  kernlab - An S4 Package for Kernel Methods in R , 2004 .

[26]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[27]  Jeff Linderoth,et al.  Noncommercial Software for Mixed-Integer Linear Programming , 2005 .

[28]  Kurt Hornik,et al.  A CLUE for CLUster Ensembles , 2005 .

[29]  J. Suykens,et al.  Convex Clustering Shrinkage , 2005 .

[30]  Ted K. Ralphs,et al.  The Symphony Callable Library for Mixed Integer Programming , 2005 .

[31]  A. Nemirovski Advances in convex optimization : conic programming , 2005 .

[32]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[33]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[34]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[35]  Shuangge Ma,et al.  Relative Risk Regression in Medical Research: Models, Contrasts, Estimators, and Algorithms , 2006 .

[36]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[37]  Yinyu Ye,et al.  Algorithm 875: DSDP5—software for semidefinite programming , 2008, TOMS.

[38]  R. Hankin Special functions in R: introducing the gsl package , 2009 .

[39]  Sebastian Bassi,et al.  Python Language Reference , 2009 .

[40]  Ravi Varadhan,et al.  BB: An R Package for Solving a Large System of Nonlinear Equations and for Optimizing a High-Dimensional Nonlinear Objective Function , 2009 .

[41]  Achim Zeileis,et al.  Collaborative Software Development Using R-Forge , 2009, R J..

[42]  David Ardia,et al.  DEoptim: An R Package for Global Optimization by Differential Evolution , 2009 .

[43]  Leo Liberti,et al.  Branching and bounds tighteningtechniques for non-convex MINLP , 2009, Optim. Methods Softw..

[44]  Robert Chares Cones and interior-point algorithms for structured convex optimization involving powers andexponentials , 2009 .

[45]  Technik CMA-ES (Covariance Matrix Adaptation Evolution Strategy) , 2010 .

[46]  Ailsa H. Land,et al.  An Automatic Method of Solving Discrete Programming Problems , 1960 .

[47]  Matteo Fischetti,et al.  Pruning Moves , 2010, INFORMS J. Comput..

[48]  John C. Nash,et al.  Unifying Optimization Algorithms to Aid Software System Users: optimx for R , 2011 .

[49]  Thierry Benoist,et al.  LocalSolver 1.x: a black-box local-search solver for 0-1 programming , 2011, 4OR.

[50]  L. Ljung,et al.  Just Relax and Come Clustering! : A Convexification of k-Means Clustering , 2011 .

[51]  Nikolaos V. Sahinidis,et al.  Semidefinite relaxations for quadratically constrained quadratic programming: A review and comparisons , 2011, Math. Program..

[52]  Andrea Lodi,et al.  Mixed Integer Programming Library version 5 , 2011 .

[53]  Jasjeet S. Sekhon,et al.  Genetic Optimization Using Derivatives , 2011, Political Analysis.

[54]  Enrico Schumann,et al.  Numerical Methods and Optimization in Finance , 2011 .

[55]  Zhang Liu,et al.  Interior-point methods for large-scale cone programming , 2011 .

[56]  Francis R. Bach,et al.  Clusterpath: an Algorithm for Clustering using Convex Fusion Penalties , 2011, ICML.

[57]  Guoping He,et al.  A new one-step smoothing Newton method for second-order cone programming , 2012 .

[58]  Joaquim R. R. A. Martins,et al.  pyOpt: a Python-based object-oriented framework for nonlinear constrained optimization , 2011, Structural and Multidisciplinary Optimization.

[59]  R. Dennis Cook,et al.  GrassmannOptim: An R Package for Grassmann Manifold Optimization , 2012 .

[60]  下田 吉之,et al.  PSO(Particle Swarm Optimization)手法による最適熱源探索 , 2012 .

[61]  Lisa Turner,et al.  Applications of Second Order Cone Programming , 2012 .

[62]  Robert E. Bixby,et al.  A Brief History of Linear and Mixed-Integer Programming Computation , 2012 .

[63]  Mauricio Zambrano-Bigiarini,et al.  A model-independent Particle Swarm Optimisation software for model calibration , 2013, Environ. Model. Softw..

[64]  Stephen P. Boyd,et al.  ECOS: An SOCP solver for embedded systems , 2013, 2013 European Control Conference (ECC).

[65]  Gabriel Gelius-Dietrich R Interface to C API of COIN-OR Clp , 2013 .

[66]  Mehmet Hakan Satman,et al.  Machine Coded Genetic Algorithms For Real Parameter Optimization Problems , 2013 .

[67]  Luca Scrucca,et al.  GA: A Package for Genetic Algorithms in R , 2013 .

[68]  Yang Xiang,et al.  Generalized Simulated Annealing for Global Optimization: The GenSA Package , 2013, R J..

[69]  José H. Dulá,et al.  The L1-norm best-fit hyperplane problem , 2013, Appl. Math. Lett..

[70]  Ji Luo,et al.  Estimation of relative risk using a log-binomial model with constraints , 2014, Comput. Stat..

[71]  Bernhard Pfaff rneos: XML-RPC Interface to NEOS , 2014 .

[72]  Katharine M. Mullen Continuous Global Optimization in R , 2014 .

[73]  Michael Braun trustOptim: An R Package for Trust Region Optimization with Sparse Hessians , 2014 .

[74]  R. Koenker,et al.  Convex Optimization in R , 2014 .

[75]  Mary C. Meyer,et al.  coneproj: An R Package for the Primal or Dual Cone Projections with Routines for Constrained Regression , 2014 .

[76]  Stephen P. Boyd,et al.  Convex Optimization in Julia , 2014, 2014 First Workshop for High Performance Technical Computing in Dynamic Languages.

[77]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[78]  Anton van den Hengel,et al.  Semidefinite Programming , 2014, Computer Vision, A Reference Guide.

[79]  John C. Nash,et al.  On Best Practice Optimization Methods in R , 2014 .

[80]  Matlab Matlab (the language of technical computing): using matlab graphics ver.5 , 2014 .

[81]  Christian Kirches,et al.  qpOASES: a parametric active-set algorithm for quadratic programming , 2014, Math. Program. Comput..

[82]  Shuicheng Yan,et al.  Convex Optimization Procedure for Clustering: Theoretical Revisit , 2014, NIPS.

[83]  Kurt Hornik,et al.  R/GNU Linear Programming Kit Interface , 2015 .

[84]  Kean Ming Tan,et al.  Statistical properties of convex clustering. , 2015, Electronic journal of statistics.

[85]  Eric C. Chi,et al.  Splitting Methods for Convex Clustering , 2013, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[86]  J. Nocedal,et al.  Limited Memory BFGS Minimizer with Bounds on Parameters , 2015 .

[87]  Iain Dunning,et al.  Computing in Operations Research Using Julia , 2013, INFORMS J. Comput..

[88]  D. Bertsimas,et al.  Best Subset Selection via a Modern Optimization Lens , 2015, 1507.03133.

[89]  Stephen P. Boyd,et al.  Convex Optimization with Abstract Linear Operators , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[90]  Kurt Hornik,et al.  Data Structures and Algorithms for Relations , 2015 .

[91]  Angelo J. Canty,et al.  Bootstrap Functions (Originally by Angelo Canty for S) , 2015 .

[92]  Gabriel Gelius-Dietrich R Interface to C API of GLPK , 2015 .

[93]  Daniel Molina,et al.  Memetic Algorithms with Local Search Chains in R: The Rmalschains Package , 2016 .

[94]  Stephen P. Boyd,et al.  Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding , 2013, Journal of Optimization Theory and Applications.

[95]  Stephen P. Boyd,et al.  CVXPY: A Python-Embedded Modeling Language for Convex Optimization , 2016, J. Mach. Learn. Res..

[96]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[97]  Stephen P. Boyd,et al.  CVXR: An R Package for Disciplined Convex Optimization , 2017, Journal of Statistical Software.

[98]  Stephen P. Boyd,et al.  OSQP: an operator splitting solver for quadratic programs , 2017, 2018 UKACC 12th International Conference on Control (CONTROL).

[99]  Andrea Lodi,et al.  QPLIB: a library of quadratic programming instances , 2018, Math. Program. Comput..