A comparison of cross-entropy and variance minimization strategies

The variance minimization (VM) and cross-entropy (CE) methods are two versatile adaptive importance sampling procedures that have been successfully applied to a wide variety of difficult rare-event estimation problems. We compare these two methods via various examples where the optimal VM and CE importance densities can be obtained analytically. We find that in the cases studied both VM and CE methods prescribe the same importance sampling parameters, suggesting that the criterion of minimizing the CE distance is very close, if not asymptotically identical, to minimizing the variance of the associated importance sampling estimator.

[1]  Dirk P. Kroese,et al.  HEAVY TAILS, IMPORTANCE SAMPLING AND CROSS–ENTROPY , 2005 .

[2]  Peter W. Glynn,et al.  How to Deal with the Curse of Dimensionality of Likelihood Ratios in Monte Carlo Simulation , 2009 .

[3]  Dirk P. Kroese,et al.  Cross‐Entropy Method , 2011 .

[4]  R. Rubinstein.  The Gibbs Cloner for Combinatorial Optimization, Counting and Sampling , 2009 .

[5]  P. Glynn,et al.  Efficient rare-event simulation for the maximum of heavy-tailed random walks , 2008, 0808.2731.

[6]  Dirk P. Kroese,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning , 2004 .

[7]  J. Hammersley.  SIMULATION AND THE MONTE CARLO METHOD , 1982 .

[8]  Enver Yücesan,et al.  Guest editors' introduction to special issue on the first INFORMS simulation society research workshop , 2010, TOMC.

[9]  Dirk P. Kroese,et al.  Improved cross-entropy method for estimation , 2011, Statistics and Computing.

[10]  P. Shahabuddin,et al.  Simulating heavy tailed processes using delayed hazard rate twisting , 1999, WSC'99. 1999 Winter Simulation Conference Proceedings. 'Simulation - A Bridge to the Future' (Cat. No.99CH37038).

[11]  Peter W. Glynn,et al.  Asymptotic robustness of estimators in rare-event simulation , 2007, TOMC.

[12]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[13]  Dirk P. Kroese,et al.  Rare-event probability estimation with conditional Monte Carlo , 2011, Ann. Oper. Res..

[14]  Peter W. Glynn,et al.  Stochastic Simulation: Algorithms and Analysis , 2007 .

[15]  Dirk P. Kroese,et al.  The Transform Likelihood Ratio Method for Rare Event Simulation with Heavy Tails , 2004, Queueing Syst. Theory Appl..

[16]  Dirk P. Kroese,et al.  Improved algorithms for rare event simulation with heavy tails , 2006, Advances in Applied Probability.

[17]  Jose H. Blanchet,et al.  Efficient rare event simulation for heavy-tailed compound sums , 2011, TOMC.

[18]  Dirk P. Kroese,et al.  The Cross-Entropy Method , 2011, Information Science and Statistics.

[19]  S. Asmussen,et al.  Rare events simulation for heavy-tailed distributions , 2000 .

[20]  Dirk P. Kroese,et al.  Simulation and the Monte Carlo Method: Solutions Manual to Accompany , 2007 .