Differential evolution based on reinforcement learning with fitness ranking for solving multimodal multiobjective problems

Abstract In multimodal multiobjective optimization problems (MMOOPs), there is more than one Pareto-optimal Set (PS) in the decision space corresponding to the same Pareto Front(PF). How to dynamically adjust the evolution direction of the population adaptively is a key problem, to ensure approaching the PF in the global sense with good convergence while finding out more PSs. In this paper, a novel Differential Evolution algorithm based on Reinforcement Learning with Fitness Ranking (DE-RLFR) is proposed. The DE-RLFR is based on the Q-learning framework, and each individual in the population is considered an agent. The fitness ranking values of each agent are used to encode hierarchical state variables. Three typical DE mutation operations are employed as optional actions for the agent. Based on the analysis of the distribution characteristics of the population in objective space, decision space and fitness-ranking space, we design a reward function of the 〉state, action〈 pairs to guide the population to move to the PF asymptotically. According to its reinforcement learning experience represented by the corresponding Q table value, each agent could adaptively select a mutation strategy to generate offspring individuals. The evaluation results on eleven MMOOP test functions show that DE-RLFR could quickly and effectively find multiple PSs in the decision space, and approach PF in the global sense.