Ensemble Learning Based on Multimodal Multiobjective Optimization

In ensemble learning, the accuracy and diversity are two conflicting objectives. As the number of base learners increases, the prediction speed of ensemble learning machines drops significantly and the required storage space also increases rapidly. How to balance these two goals for selective ensemble learning is an extremely essential problem. In this paper, ensemble learning based on multimodal multiobjective optimization is studied in detail. The great significance and importance of multimodal multiobjective optimization algorithm is to find these different classifiers ensemble by considering the balance between accuracy and diversity, and different classifiers ensemble correspond to the same accuracy and diversity. Experimental results show that multimodal multiobjective optimization algorithm can find more ensemble combinations than unimodal optimization algorithms.

[1]  Matthew A. Kupinski,et al.  Multiobjective Genetic Optimization of Diagnostic Classifiers with Implications for Generating ROC Curves , 1999, IEEE Trans. Medical Imaging.

[2]  Ye Tian,et al.  A Classification-Based Surrogate-Assisted Evolutionary Algorithm for Expensive Many-Objective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[3]  Yianni Attikiouzel,et al.  A novel multicriteria optimization algorithm for the structure determination of multilayer feedforward neural networks , 1996 .

[4]  Okan Duru,et al.  Predictive analytics of crude oil prices by utilizing the intelligent model search engine , 2018, Applied Energy.

[5]  Wentao Mao,et al.  Model selection of extreme learning machine based on multi-objective optimization , 2011, Neural Computing and Applications.

[6]  Hamido Fujita,et al.  Incremental fuzzy cluster ensemble learning based on rough set theory , 2017, Knowl. Based Syst..

[7]  Fernando Nogueira,et al.  Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning , 2016, J. Mach. Learn. Res..

[8]  Aliaksandr Barushka,et al.  Spam Filtering in Social Networks Using Regularized Deep Neural Networks with Ensemble Learning , 2018, AIAI.

[9]  Byoung-Tak Zhang,et al.  Ensemble Learning with Active Example Selection for Imbalanced Biomedical Data Classification , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[10]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[11]  Jing J. Liang,et al.  A Multiobjective Particle Swarm Optimizer Using Ring Topology for Solving Multimodal Multiobjective Problems , 2018, IEEE Transactions on Evolutionary Computation.

[12]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[13]  Xuejun Li,et al.  Gear fault diagnosis based on support vector machine optimized by artificial bee colony algorithm , 2015 .

[14]  Yurong Liu,et al.  A niching evolutionary algorithm with adaptive negative correlation learning for neural network ensemble , 2017, Neurocomputing.

[15]  Xin Yao,et al.  Evolving artificial neural network ensembles , 2008 .

[16]  Peter Stone,et al.  Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science , 2017, Nature Communications.

[17]  Cha Zhang,et al.  Ensemble Machine Learning , 2012 .

[18]  Cha Zhang,et al.  Ensemble Machine Learning: Methods and Applications , 2012 .

[19]  Wei Tang,et al.  Selective Ensemble of Decision Trees , 2003, RSFDGrC.

[20]  Xin Yao,et al.  Ensemble Learning Using Multi-Objective Evolutionary Algorithms , 2006, J. Math. Model. Algorithms.

[21]  Simon G. Thompson.  Pruning boosted classifiers with a real valued genetic algorithm , 1999, Knowl. Based Syst..

[22]  Kay Chen Tan,et al.  A Subregion Division-Based Evolutionary Algorithm With Effective Mating Selection for Many-Objective Optimization , 2020, IEEE Transactions on Cybernetics.

[23]  Lam Thu Bui,et al.  A novel evolutionary multi-objective ensemble learning approach for forecasting currency exchange rates , 2017, Data Knowl. Eng..

[24]  Ye Tian,et al.  A region division based diversity maintaining approach for many-objective optimization , 2017, Integr. Comput. Aided Eng..

[25]  Wei Tang,et al.  Ensembling neural networks: Many could be better than all , 2002, Artif. Intell..

[26]  Jing J. Liang,et al.  A novel scalable test problem suite for multimodal multiobjective optimization , 2019, Swarm Evol. Comput..

[27]  Yaochu Jin,et al.  A radial space division based evolutionary algorithm for many-objective optimization , 2017, Appl. Soft Comput..

[28]  Jing J. Liang,et al.  Multimodal multi-objective optimization: A preliminary study , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[29]  Kalyanmoy Deb,et al.  Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems , 1999, Evolutionary Computation.

[30]  Václav Snásel,et al.  Ensemble of heterogeneous flexible neural trees using multiobjective genetic programming , 2017, Appl. Soft Comput..

[31]  Yi Chen,et al.  Heterogeneous ensemble learning with feature engineering for default prediction in peer-to-peer lending in China , 2019, World Wide Web.

[32]  Jing J. Liang,et al.  A Self-organizing Multi-objective Particle Swarm Optimization Algorithm for Multimodal Multi-objective Problems , 2018, ICSI.

[33]  Carolina P. de Almeida,et al.  Extreme Learning Surrogate Models in Multi-objective Optimization based on Decomposition , 2016, Neurocomputing.

[34]  Yong Zhang,et al.  Differential Evolution Based Selective Ensemble of Extreme Learning Machine , 2016, 2016 IEEE Trustcom/BigDataSE/ISPA.