When Neural Network Computation Meets Evolutionary Computation: A Survey

Neural network (NN) and evolutionary computation (EC) are two of the most popular and important techniques in computational intelligence, which can be combined together to solve the complex real world problems. This paper represents a review of the researches that combined NN and EC. There are 3 main research focuses as follows. In the first research focus, EC algorithms have been widely used to optimize the structure and parameter of the NN, including weight, structure, learning rates, and others. In the second research focus, lots literatures have witnessed that EC based NNs are widespread in the applications such as classification, automatic control, prediction, and many other fields. These two kinds of researches into combining NN and EC are mainly focuses on using EC algorithms to optimize NN, to enhance the NN performance and the NN application ability. Our survey finds that particle swarm optimization is the most popular EC algorithm that the researchers choice to optimize NN in recent year, while genetic algorithm and differential evolution are also widely used. In the third research focus, there are also researches adopted NN as a tool to enhance the performance of EC algorithms. Although the literatures in this focus are not as many as the above two focuses, the existing results show that NN has great potential in enhancing EC algorithms. The survey shows that when NN and EC meet, combining them would result in an effective way to deal with the real world application. This interesting research topic has become more and more significant in the field of computer science and still has much room for development.

[1]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[2]  Maoguo Gong,et al.  A Multiobjective Sparse Feature Learning Model for Deep Neural Networks , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Faa-Jeng Lin,et al.  Wavelet Fuzzy Neural Network With Asymmetric Membership Function Controller for Electric Power Steering System via Improved Differential Evolution , 2015, IEEE Transactions on Power Electronics.

[4]  Shinji Watanabe,et al.  Structure discovery of deep neural network based on evolutionary algorithms , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  David B. Fogel,et al.  Evolutionary Computation: The Fossil Record , 1998 .

[6]  Kit Po Wong,et al.  A Self-Adaptive RBF Neural Network Classifier for Transformer Fault Analysis , 2010, IEEE Transactions on Power Systems.

[7]  Pedro Antonio Gutiérrez,et al.  Sensitivity Versus Accuracy in Multiclass Problems Using Memetic Pareto Evolutionary Neural Networks , 2010, IEEE Transactions on Neural Networks.

[8]  Witold Pedrycz,et al.  Granular Neural Networks: Concepts and Development Schemes , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[9]  H Zareipour,et al.  Wind Power Prediction by a New Forecast Engine Composed of Modified Hybrid Neural Network and Enhanced Particle Swarm Optimization , 2011, IEEE Transactions on Sustainable Energy.

[10]  Chia-Feng Juang,et al.  Evolving Gaits of a Hexapod Robot by Recurrent Neural Networks With Symbiotic Species-Based Particle Swarm Optimization , 2011, IEEE Transactions on Industrial Electronics.

[11]  R. Ruthen.  The Frustrations of a Quark Hunter , 1992 .

[12]  Zhi-hui Zhan,et al.  Evolutionary Neural Network Based Energy Consumption Forecast for Cloud Computing , 2015, 2015 International Conference on Cloud Computing Research and Innovation (ICCCRI).

[13]  Stephen A. Billings,et al.  Lattice Dynamical Wavelet Neural Networks Implemented Using Particle Swarm Optimization for Spatio–Temporal System Identification , 2009, IEEE Transactions on Neural Networks.

[14]  Yu Gong,et al.  Online Modeling With Tunable RBF Network , 2013, IEEE Transactions on Cybernetics.

[15]  Pedro Antonio Gutiérrez,et al.  Logistic Regression by Means of Evolutionary Radial Basis Function Neural Networks , 2011, IEEE Transactions on Neural Networks.

[16]  Jun Zhang,et al.  Deadline Constrained Cloud Computing Resources Scheduling through an Ant Colony System Approach , 2015, 2015 International Conference on Cloud Computing Research and Innovation (ICCCRI).

[17]  R.P. Maheshwari,et al.  Power Transformer Differential Protection Based On Optimal Probabilistic Neural Network , 2010, IEEE Transactions on Power Delivery.

[18]  Chin-Teng Lin,et al.  Nonlinear System Control Using Adaptive Neural Fuzzy Networks Based on a Modified Differential Evolution , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[19]  Rifai Chai,et al.  Brain–Computer Interface Classifier for Wheelchair Commands Using Neural Network With Fuzzy Particle Swarm Optimization , 2014, IEEE Journal of Biomedical and Health Informatics.

[20]  Chin-Teng Lin,et al.  A Hybrid of Cooperative Particle Swarm Optimization and Cultural Algorithm for Neural Fuzzy Networks and Its Prediction Applications , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[21]  Marzuki Khalid,et al.  Evolutionary Fuzzy ARTMAP Neural Networks for Classification of Semiconductor Defects , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Jun Wang,et al.  A Dynamic Feedforward Neural Network Based on Gaussian Particle Swarm Optimization and its Application for Predictive Control , 2011, IEEE Transactions on Neural Networks.

[23]  Hung T. Nguyen,et al.  Industrial Application of Evolvable Block-Based Neural Network to Hypoglycemia Monitoring System , 2013, IEEE Transactions on Industrial Electronics.

[24]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[25]  Farshid Keynia,et al.  Short-Term Load Forecast of Microgrids by a New Bilevel Prediction Strategy , 2010, IEEE Transactions on Smart Grid.

[26]  Yigang He,et al.  A Novel Method for Analog Fault Diagnosis Based on Neural Networks and Genetic Algorithms , 2008, IEEE Transactions on Instrumentation and Measurement.

[27]  Mohamed E. El-Hawary,et al.  A Survey of Particle Swarm Optimization Applications in Electric Power Systems , 2009, IEEE Transactions on Evolutionary Computation.

[28]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[29]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[30]  Y. Volkan Pehlivanoglu,et al.  A New Particle Swarm Optimization Method Enhanced With a Periodic Mutation Strategy and Neural Networks , 2013, IEEE Transactions on Evolutionary Computation.

[31]  Meie Shen,et al.  Bi-Velocity Discrete Particle Swarm Optimization and Its Application to Multicast Routing Problem in Communication Networks , 2014, IEEE Transactions on Industrial Electronics.

[32]  Po-Li Chen,et al.  Thermal Optimal Design for Plain Plate-Fin Heat Sinks by Using Neuro-Genetic Method , 2008, IEEE Transactions on Components and Packaging Technologies.

[33]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[34]  Chin-Teng Lin,et al.  An Improved Polynomial Neural Network Classifier Using Real-Coded Genetic Algorithm , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[35]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[36]  Jiannong Cao,et al.  Multiple Populations for Multiple Objectives: A Coevolutionary Technique for Solving Multiobjective Optimization Problems , 2013, IEEE Transactions on Cybernetics.

[37]  Vladimiro Miranda,et al.  Optimizing Large Scale Problems With Metaheuristics in a Reduced Space Mapped by Autoencoders—Application to the Wind-Hydro Coordination , 2014, IEEE Transactions on Power Systems.

[38]  Kyriakos Kentzoglanakis,et al.  A Swarm Intelligence Framework for Reconstructing Gene Networks: Searching for Biologically Plausible Architectures , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[39]  Syuan-Yi Chen,et al.  Recurrent Functional-Link-Based Fuzzy Neural Network Controller With Improved Particle Swarm Optimization for a Linear Synchronous Motor Drive , 2009, IEEE Transactions on Magnetics.

[40]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[41]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[42]  Jun Zhang,et al.  Competitive and cooperative particle swarm optimization with information sharing mechanism for global optimization problems , 2015, Inf. Sci..

[43]  Thomas Stützle,et al.  Ant colony optimization: artificial ants as a computational intelligence technique , 2006 .

[44]  Jun Zhang,et al.  Deadline constrained cloud computing resources scheduling for cost optimization based on dynamic objective genetic algorithm , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[45]  Georges G. E. Gielen,et al.  Synthesis of Integrated Passive Components for High-Frequency RF ICs Based on Evolutionary Computation and Machine Learning Techniques , 2011, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[46]  A. Patnaik,et al.  Design of Custom-Made Fractal Multi-Band Antennas Using ANN-PSO [Antenna Designer's Notebook] , 2011, IEEE Antennas and Propagation Magazine.

[47]  Adam Slowik,et al.  Application of an Adaptive Differential Evolution Algorithm With Multiple Trial Vectors to Artificial Neural Network Training , 2011, IEEE Transactions on Industrial Electronics.