Using Maximum Entropy and Generalized Belief Propagation in Estimation of Distribution Algorithms

EDAs work by sampling a population from a factorized distribution, like the Boltzmann distribution of an additively decomposable fitness function (ADF). In the Factorized Distribution Algorithm (FDA), a factorization is built from an ADF by choosing a subset of the factors. I present a new algorithm to merge factors into larger sets, allowing to account for all dependencies between the variables. Estimating larger subset distributions is more prone to sample noise, so the larger distribution can be estimated from the smaller ones with the Maximum Entropy method. Building an exact graphical model for sampling is often infeasible. E.g. in a 2-D grid the triangulated Markov network has linear clique size, thus exponentially large distributions. I explore ways to use loopy models and Generalized Belief Propagation in the context of EDA and optimization. The merging algorithm mentioned above can be combined with this.