A multi-objective feature selection method based on bacterial foraging optimization

Feature selection plays an important role in data preprocessing. The aim of feature selection is to recognize and remove redundant or irrelevant features. The key issue is to use as few features as possible to achieve the lowest classification error rate. This paper formulates feature selection as a multi-objective problem. In order to address feature selection problem, this paper uses the multi-objective bacterial foraging optimization algorithm to select the feature subsets and k-nearest neighbor algorithm as the evaluation algorithm. The wheel roulette mechanism is further introduced to remove duplicated features. Four information exchange mechanisms are integrated into the bacteria-inspired algorithm to avoid the individuals getting trapped into the local optima so as to achieve better results in solving high-dimensional feature selection problem. On six small datasets and ten high-dimensional datasets, comparative experiments with different conventional wrapper methods and several evolutionary algorithms demonstrate the superiority of the proposed bacteria-inspired based feature selection method.

[1]  Dervis Karaboga,et al.  AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION , 2005 .

[2]  Xin Yao,et al.  A Survey on Evolutionary Computation Approaches to Feature Selection , 2016, IEEE Transactions on Evolutionary Computation.

[3]  Rossitza Setchi,et al.  Feature selection using Joint Mutual Information Maximisation , 2015, Expert Syst. Appl..

[4]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms , 2014, Appl. Soft Comput..

[5]  Hossein Nezamabadi-pour,et al.  An advanced ACO algorithm for feature subset selection , 2015, Neurocomputing.

[6]  Rich Caruana,et al.  Greedy Attribute Selection , 1994, ICML.

[7]  Ben Niu,et al.  A novel bacterial algorithm with randomness control for feature selection in classification , 2017, Neurocomputing.

[8]  Mengjie Zhang,et al.  New fitness functions in binary particle swarm optimisation for feature selection , 2012, 2012 IEEE Congress on Evolutionary Computation.

[9]  Li-Yeh Chuang,et al.  Improved binary PSO for feature selection using gene expression data , 2008, Comput. Biol. Chem..

[10]  Fakhri Karray,et al.  Multi-objective Feature Selection with NSGA II , 2007, ICANNGA.

[11]  Li-Yeh Chuang,et al.  IG-GA: A Hybrid Filter/Wrapper Method for Feature Selection of Microarray Data , 2010 .

[12]  Chulhee Lee,et al.  Feature extraction based on the Bhattacharyya distance , 2003, Pattern Recognit..

[13]  Adel Al-Jumaily,et al.  Feature subset selection using differential evolution and a statistical repair mechanism , 2011, Expert Syst. Appl..

[14]  Qun Dai,et al.  A hierarchical and parallel branch-and-bound ensemble selection algorithm , 2016, Applied Intelligence.

[15]  Ben Niu,et al.  A discrete bacterial algorithm for feature selection in classification of microarray gene expression cancer data , 2017, Knowl. Based Syst..

[16]  Zne-Jung Lee,et al.  Parameter determination of support vector machine and feature selection using simulated annealing approach , 2008, Appl. Soft Comput..

[17]  Kevin M. Passino,et al.  Biomimicry of bacterial foraging for distributed optimization and control , 2002 .

[18]  Bin Li,et al.  Link prediction based on sampling in complex networks , 2017, Applied Intelligence.

[19]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Randy J. Pell,et al.  Genetic algorithms combined with discriminant analysis for key variable identification , 2004 .

[21]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[22]  Jianyu Yang,et al.  Object-oriented feature selection of high spatial resolution images using an improved Relief algorithm , 2013, Math. Comput. Model..

[23]  Xuefeng Yan,et al.  Optimizing the echo state network with a binary particle swarm optimization algorithm , 2015, Knowl. Based Syst..

[24]  Huan Liu,et al.  Searching for interacting features in subset selection , 2009, Intell. Data Anal..

[25]  Michael G. Pecht,et al.  Health Monitoring of Cooling Fans Based on Mahalanobis Distance With mRMR Feature Selection , 2012, IEEE Transactions on Instrumentation and Measurement.

[26]  Tansel Özyer,et al.  A Consistency-Based Feature Selection Method Allied with Linear SVMs for HIV-1 Protease Cleavage Site Prediction , 2013, PloS one.

[27]  Jian Ma,et al.  Igf-bagging: Information gain based feature selection for bagging , 2011 .

[28]  Seoung Bum Kim,et al.  Sequential random k-nearest neighbor feature selection for high-dimensional data , 2015, Expert Syst. Appl..

[29]  Mengjie Zhang,et al.  Multi-objective particle swarm optimisation (PSO) for feature selection , 2012, GECCO '12.

[30]  Li-Yeh Chuang,et al.  Improved binary particle swarm optimization using catfish effect for feature selection , 2011, Expert Syst. Appl..

[31]  Kevin D. Seppi,et al.  An exploration of topologies and communication in large particle swarms , 2009, 2009 IEEE Congress on Evolutionary Computation.

[32]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[33]  Huan Liu,et al.  Consistency Based Feature Selection , 2000, PAKDD.

[34]  Ben Niu,et al.  Multi-objective bacterial foraging optimization , 2013, Neurocomputing.

[35]  William H. Hsu,et al.  Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning , 2004, Inf. Sci..

[36]  Bin Ran,et al.  Feature selection with redundancy-complementariness dispersion , 2015, Knowl. Based Syst..

[37]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[38]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[39]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[40]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[41]  Le Hoang Son,et al.  Some novel hybrid forecast methods based on picture fuzzy clustering for weather nowcasting from satellite image sequences , 2016, Applied Intelligence.

[42]  Gang Wang,et al.  A novel bacterial foraging optimization algorithm for feature selection , 2017, Expert Syst. Appl..

[43]  Zexuan Zhu,et al.  Markov blanket-embedded genetic algorithm for gene selection , 2007, Pattern Recognit..

[44]  Nikola Bogunovic,et al.  A review of feature selection methods with applications , 2015, 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[45]  Eibe Frank,et al.  Large-scale attribute selection using wrappers , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.