Uncertain convex programs: randomized solutions and confidence levels

Abstract.Many engineering problems can be cast as optimization problems subject to convex constraints that are parameterized by an uncertainty or ‘instance’ parameter. Two main approaches are generally available to tackle constrained optimization problems in presence of uncertainty: robust optimization and chance-constrained optimization. Robust optimization is a deterministic paradigm where one seeks a solution which simultaneously satisfies all possible constraint instances. In chance-constrained optimization a probability distribution is instead assumed on the uncertain parameters, and the constraints are enforced up to a pre-specified level of probability. Unfortunately however, both approaches lead to computationally intractable problem formulations.In this paper, we consider an alternative ‘randomized’ or ‘scenario’ approach for dealing with uncertainty in optimization, based on constraint sampling. In particular, we study the constrained optimization problem resulting by taking into account only a finite set of N constraints, chosen at random among the possible constraint instances of the uncertain problem. We show that the resulting randomized solution fails to satisfy only a small portion of the original constraints, provided that a sufficient number of samples is drawn. Our key result is to provide an efficient and explicit bound on the measure (probability or volume) of the original constraints that are possibly violated by the randomized solution. This volume rapidly decreases to zero as N is increased.

[1]  J. E. Kelley,et al.  The Cutting-Plane Method for Solving Convex Programs , 1960 .

[2]  W. Hoeffding Probability Inequalities for sums of Bounded Random Variables , 1963 .

[3]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[4]  David P. Williamson,et al.  .879-approximation algorithms for MAX CUT and MAX 2SAT , 1994, STOC '94.

[5]  Robert J. Vanderbei,et al.  Robust Optimization of Large-Scale Systems , 1995, Oper. Res..

[6]  Arkadi Nemirovski,et al.  Robust Truss Topology Design via Semidefinite Programming , 1997, SIAM J. Optim..

[7]  Elijah Polak,et al.  Semi-Infinite Optimization , 1997 .

[8]  Mathukumalli Vidyasagar,et al.  A Theory of Learning and Generalization , 1997 .

[9]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[10]  A. Shapiro,et al.  On rate of convergence of Monte Carlo approximations of stochastic programs , 1998 .

[11]  J Figueira,et al.  Stochastic Programming , 1998, J. Oper. Res. Soc..

[12]  Laurent El Ghaoui,et al.  Robust Solutions to Uncertain Semidefinite Programs , 1998, SIAM J. Optim..

[13]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[14]  M. A. López-Cerdá,et al.  Linear Semi-Infinite Optimization , 1998 .

[15]  Arkadi Nemirovski,et al.  Robust solutions of uncertain linear programs , 1999, Oper. Res. Lett..

[16]  J. R. Morrison,et al.  New Linear Program Performance Bounds for Queueing Networks , 1999 .

[17]  Russ Bubley,et al.  Randomized algorithms , 1995, CSUR.

[18]  Alexander Shapiro,et al.  On the Rate of Convergence of Optimal Solutions of Monte Carlo Approximations of Stochastic Programs , 2000, SIAM J. Optim..

[19]  Pierre Apkarian,et al.  Parameterized LMIs in Control Theory , 2000, SIAM J. Control. Optim..

[20]  Giuseppe Carlo Calafiore,et al.  Robust filtering for discrete-time systems with bounded noise and parametric uncertainty , 2001, IEEE Trans. Autom. Control..

[21]  Giuseppe Carlo Calafiore,et al.  Stochastic algorithms for exact and approximate feasibility of robust LMIs , 2001, IEEE Trans. Autom. Control..

[22]  Dale Schuurmans,et al.  Direct value-approximation for factored MDPs , 2001, NIPS.

[23]  Georg Still,et al.  Discretization in semi-infinite programming: the rate of convergence , 2001, Math. Program..

[24]  Giuseppe Carlo Calafiore,et al.  Identification of Reliable Predictor Models for Unknown Systems: a Data-Consistency Approach based on Learning Theory , 2002 .

[25]  On avoiding vertexization of robustness problems: the approximate feasibility concept , 2002, IEEE Trans. Autom. Control..

[26]  G. Calafiore,et al.  Interval predictors for unknown dynamical systems: an assessment of reliability , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[27]  Giuseppe Carlo Calafiore,et al.  A probabilistic framework for problems with real structured uncertainty in systems and control , 2002, Autom..

[28]  Arkadi Nemirovski,et al.  On Tractable Approximations of Uncertain Linear Matrix Inequalities Affected by Interval Uncertainty , 2002, SIAM J. Optim..

[29]  Kartik Krishnan LINEAR PROGRAMMING (LP) APPROACHES TO SEMIDEFINITE PROGRAMMING (SDP) PROBLEMS , 2002 .

[30]  Shobha Venkataraman,et al.  Efficient Solution Algorithms for Factored MDPs , 2003, J. Artif. Intell. Res..

[31]  D. D. Farias,et al.  On constraint sampling in the linear programming approach to approximate linear programming , 2003, CDC.

[32]  George B. Dantzig,et al.  Linear Programming Under Uncertainty , 2004, Manag. Sci..

[33]  Benjamin Van Roy,et al.  On Constraint Sampling in the Linear Programming Approach to Approximate Dynamic Programming , 2004, Math. Oper. Res..

[34]  Jacques Desrosiers,et al.  Selected Topics in Column Generation , 2002, Oper. Res..