Benefits of Data Clustering in Multimodal Function Optimization via EDAs

This chapter shows how Estimation of Distribution Algorithms (EDAs) can benefit from data clustering in order to optimize both discrete and continuous multimodal functions. To be exact, the advantage of incorporating clustering into EDAs is two-fold: to obtain all the best solutions rather than only one of them, and to alleviate the difficulties that affect many evolutionary algorithms when more than one global optimum exists. We propose the use of Bayesian networks and conditional Gaussian networks to perform such a data clustering when EDAs are applied to optimization in discrete and continuous multimodal domains, respectively. The dynamics and performance of our approach are shown by evaluating it on a number of symmetrical functions, some of them highly multimodal.

[1]  E. Forgy.  Cluster analysis of multivariate data : efficiency versus interpretability of classifications , 1965 .

[2]  Michael R. Anderberg,et al.  Cluster Analysis for Applications , 1973 .

[3]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[4]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[5]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[6]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[7]  A. Sanderson,et al.  Model inference and pattern discovery by minimal representation method , 1981 .

[8]  N. Wermuth,et al.  Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative , 1989 .

[9]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1991 .

[10]  S. Lauritzen.  Propagation of Probabilities, Means, and Variances in Mixed Graphical Association Models , 1992 .

[11]  Philippe Collard,et al.  DGA: An Efficient Genetic Algorithm , 1994, ECAI.

[12]  David Heckerman,et al.  Learning Gaussian Networks , 1994, UAI.

[13]  Arthur C. Sanderson,et al.  Evolutionary Speciation Using Minimal Representation Size Clustering , 1995, Evolutionary Programming.

[14]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[15]  Nir Friedman,et al.  Building Classifiers Using Bayesian Networks , 1996, AAAI/IAAI, Vol. 2.

[16]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[17]  Michael I. Jordan,et al.  Estimating Dependency Structure as a Hidden Variable , 1997, NIPS.

[18]  Arthur C. Sanderson,et al.  Multimodal Function Optimization Using Minimal Representation Size Clustering and Its Application to Planning Multipaths , 1997, Evolutionary Computation.

[19]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[20]  Bo Thiesson,et al.  Learning Mixtures of DAG Models , 1998, UAI.

[21]  Jan Naudts,et al.  The Effect of Spin-Flip Symmetry on the Performance of the Simple GA , 1998, PPSN.

[22]  Pedro Larrañaga,et al.  Learning Bayesian networks for clustering by means of constructive induction , 1999, Pattern Recognit. Lett..

[23]  Marcus Gallagher,et al.  Real-valued Evolutionary Optimization using a Flexible Probability Density Estimator , 1999, GECCO.

[24]  Pedro Larrañaga,et al.  Combinatonal Optimization by Learning and Simulation of Bayesian Networks , 2000, UAI.

[25]  Pedro Larrañaga,et al.  An improved Bayesian structural EM algorithm for learning Bayesian networks for clustering , 2000, Pattern Recognit. Lett..

[26]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[27]  Pedro Larrañaga,et al.  Optimization in Continuous Domains by Learning and Simulation of Gaussian Networks , 2000 .

[28]  C. Van Hoyweghen,et al.  Symmetry in the search space , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[29]  David E. Goldberg,et al.  Genetic Algorithms, Clustering, and the Breaking of Symmetry , 2000, PPSN.

[30]  C. V. Hoyweghen.  Detecting spin-flip symmetry in optimization problems , 2001 .

[31]  Pedro Larrañaga,et al.  Geographical clustering of cancer incidence by means of Bayesian networks and conditional Gaussian networks , 2001, AISTATS.

[32]  L. Kallel,et al.  Theoretical Aspects of Evolutionary Computing , 2001, Natural Computing Series.

[33]  David E. Goldberg,et al.  Bayesian optimization algorithm, decision graphs, and Occam's razor , 2001 .

[34]  Pedro Larrañaga,et al.  Performance evaluation of compromise conditional Gaussian networks for data clustering , 2001, Int. J. Approx. Reason..

[35]  Jiri Ocenasek,et al.  EXPERIMENTAL STUDY: HYPERGRAPH PARTITIONING BASED ON THE SIMPLE AND ADVANCED GENETIC ALGORITHM BMDA AND BOA , 2002 .

[36]  Pedro Larrañaga,et al.  Learning Recursive Bayesian Multinets for Data Clustering by Means of Constructive Induction , 2002, Machine Learning.