A distributional interpretation of robust optimization

Motivated by data-driven decision making and sampling problems, we investigate probabilistic interpretations of Robust Optimization (RO). We establish a connection between RO and Distributionally Robust Stochastic Programming (DRSP), showing that the solution to any RO problem is also a solution to a DRSP problem. Specifically, we consider the case where multiple uncertain parameters belong to the same fixed dimensional space, and find the set of distributions of the equivalent DRSP. The equivalence we derive enables us to construct RO formulations that are statistically consistent, and in the process, provides a systematic approach for tuning the uncertainty set.

[1]  J Figueira,et al.  Stochastic Programming , 1998, J. Oper. Res. Soc..

[2]  J. Dupacová.  The minimax approach to stochastic programming and an illustrative application , 1987 .

[3]  Alexander J. Smola,et al.  Second Order Cone Programming Approaches for Handling Missing and Uncertain Data , 2006, J. Mach. Learn. Res..

[4]  Shie Mannor,et al.  The Robustness-Performance Tradeoff in Markov Decision Processes , 2006, NIPS.

[5]  Stephen P. Boyd,et al.  Digital Circuit Optimization via Geometric Programming , 2005, Oper. Res..

[6]  Peter L. Bartlett,et al.  Neural Network Learning - Theoretical Foundations , 1999 .

[7]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[8]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[9]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[10]  John R. Birge,et al.  Introduction to Stochastic Programming , 1997 .

[11]  Yinyu Ye,et al.  Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems , 2010, Oper. Res..

[12]  Arkadi Nemirovski,et al.  Robust solutions of uncertain linear programs , 1999, Oper. Res. Lett..

[13]  R. Wets,et al.  Epi‐consistency of convex stochastic programs , 1991 .

[14]  Alexander Shapiro,et al.  Asymptotic analysis of stochastic programs , 1991, Ann. Oper. Res..

[15]  Arkadi Nemirovski,et al.  Robust solutions of Linear Programming problems contaminated with uncertain data , 2000, Math. Program..

[16]  Constantine Caramanis,et al.  Theory and Applications of Robust Optimization , 2010, SIAM Rev..

[17]  Allen L. Soyster,et al.  Technical Note - Convex Programming with Set-Inclusive Constraints and Applications to Inexact Linear Programming , 1973, Oper. Res..

[18]  Melvyn Sim,et al.  Distributionally Robust Optimization and Its Tractable Approximations , 2010, Oper. Res..

[19]  K. Isii.  On sharpness of tchebycheff-type inequalities , 1962 .

[20]  Melvyn Sim,et al.  The Price of Robustness , 2004, Oper. Res..

[21]  Shie Mannor,et al.  Percentile Optimization for Markov Decision Processes with Parameter Uncertainty , 2010, Oper. Res..

[22]  Shie Mannor,et al.  Robustness and Regularization of Support Vector Machines , 2008, J. Mach. Learn. Res..

[23]  Ingo Steinwart,et al.  Consistency of support vector machines and other regularized kernel classifiers , 2005, IEEE Transactions on Information Theory.

[24]  Alexander Shapiro,et al.  Worst-case distribution analysis of stochastic programs , 2006, Math. Program..

[25]  Ioana Popescu,et al.  Robust Mean-Covariance Solutions for Stochastic Optimization , 2007, Oper. Res..

[26]  Boaz Golany,et al.  Retailer-Supplier Flexible Commitments Contracts: A Robust Optimization Approach , 2005, Manuf. Serv. Oper. Manag..

[27]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[28]  Arkadi Nemirovski,et al.  Robust Modeling of Multi-Stage Portfolio Problems , 2000 .

[29]  Shie Mannor,et al.  Robust Regression and Lasso , 2008, IEEE Transactions on Information Theory.

[30]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[31]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[32]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[33]  Herbert E. Scarf,et al.  A Min-Max Solution of an Inventory Problem , 1957 .

[34]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[35]  A. Ben-Tal,et al.  Adjustable robust solutions of uncertain linear programs , 2004, Math. Program..

[36]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[37]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[38]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[39]  P. Kall,et al.  Stochastric programming with recourse: upper bounds and moment problems: a review , 1988 .

[40]  Michael I. Jordan,et al.  A Robust Minimax Approach to Classification , 2003, J. Mach. Learn. Res..

[41]  A. Shapiro,et al.  On rate of convergence of Monte Carlo approximations of stochastic programs , 1998 .

[42]  Peter Kall,et al.  Stochastic Programming , 1995 .