Global likelihood optimization via the cross-entropy method with an application to mixture models

Global likelihood maximization is an important aspect of many statistical analyses. Often the likelihood function is highly multiextremal. This presents a significant challenge to standard search procedures, which often settle too quickly into an inferior local maximum. We present a new approach based on the cross-entropy (CE) method, and illustrate its use for the analysis of mixture models.

[1]  David G. Stork,et al.  Pattern Classification , 1973 .

[2]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[3]  R. Rubinstein.  The Cross-Entropy Method for Combinatorial and Continuous Optimization , 1999 .

[4]  Dirk P. Kroese,et al.  Combinatorial Optimization via Cross-Entropy , 2004 .

[5]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  Reuven Y. Rubinstein,et al.  Cross-entropy and rare events for maximal cut and partition problems , 2002, TOMC.

[8]  Dirk P. Kroese,et al.  The Cross Entropy Method: A Unified Approach To Combinatorial Optimization, Monte-carlo Simulation (Information Science and Statistics) , 2004 .

[9]  Arnaud Doucet,et al.  The cross-entropy method for blind multiuser detection , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[10]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[11]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[12]  David G. Stork,et al.  Computer Manual in MATLAB to Accompany Pattern Classification, Second Edition , 2004 .

[13]  Lih-Yuan Deng,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning , 2006, Technometrics.

[14]  Dirk P. Kroese,et al.  Application of the Cross-Entropy Method to the Buffer Allocation Problem in a Simulation-Based Environment , 2005, Ann. Oper. Res..

[15]  Bjarne E. Helvik,et al.  Using the Cross-Entropy Method to Guide/Govern Mobile Agent's Path Finding in Networks , 2001, MATA.

[16]  Yan-Bin Jia.  The Simplex Method , 2012 .

[17]  Bjarne E. Helvik,et al.  Cross entropy guided ant-like agents finding dependable primary/backup path patterns in networks , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[18]  L. Margolin,et al.  On the Convergence of the Cross-Entropy Method , 2005, Ann. Oper. Res..

[19]  R. A. Boyles.  On the Convergence of the EM Algorithm , 1983 .

[20]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[21]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..

[22]  Shie Mannor,et al.  The Cross Entropy Method for Fast Policy Search , 2003, ICML.

[23]  Tito Homem-de-Mello,et al.  Solving the Vehicle Routing Problem with Stochastic Demands using the Cross-Entropy Method , 2005, Ann. Oper. Res..