Filter versus wrapper feature selection based on problem landscape features

Feature selection is a complex problem used across many fields, such as computer vision and data mining. Feature selection algorithms extract a subset of features from a greater feature set which can improve algorithm accuracy by discarding features that are less significant in achieving the goal function. Current approaches are often computationally expensive, provide insignificant increases in predictor performance, and can lead to overfitting. This paper investigates the binary feature selection problem and the applicability of using filter and wrapper techniques guided by fitness landscape characteristics. It is shown that using filter methods are more appropriate for problems where the fitness does not provide sufficient information to guide search as needed by wrapper techniques.

[1]  Michael Affenzeller,et al.  A Comprehensive Survey on Fitness Landscape Analysis , 2012, Recent Advances in Intelligent Engineering Systems.

[2]  Edoardo Amaldi,et al.  On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems , 1998, Theor. Comput. Sci..

[3]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[4]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[5]  Anna Sergeevna Bosman,et al.  Characterising neutrality in neural network error landscapes , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[6]  Guoqing Wang,et al.  McTwo: a two-step feature selection algorithm based on maximal information coefficient , 2016, BMC Bioinformatics.

[7]  Jin-Kao Hao,et al.  Landscapes and the Maximal Constraint Satisfaction Problem , 1999, Artificial Evolution.

[8]  D. Kibler,et al.  Instance-based learning algorithms , 2004, Machine Learning.

[9]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  M. Kimura,et al.  The neutral theory of molecular evolution. , 1983, Scientific American.

[11]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[12]  Christian M. Reidys,et al.  Neutrality in fitness landscapes , 2001, Appl. Math. Comput..

[13]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[14]  Arie Ben-David,et al.  Comparison of classification accuracy using Cohen's Weighted Kappa , 2008, Expert Syst. Appl..

[15]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[16]  Yun-Chi Yeh,et al.  Feature Selection Algorithm for Motor Quality Types Using Weighted Principal Component Analysis , 2016 .

[17]  W Ebeling,et al.  Smoothing representation of fitness landscapes--the genotype-phenotype map of evolution. , 1995, Bio Systems.

[18]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[19]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[20]  Dinggang Shen,et al.  Machine Learning in Medical Imaging , 2012, Lecture Notes in Computer Science.

[21]  Grenville J. Armitage,et al.  A survey of techniques for internet traffic classification using machine learning , 2008, IEEE Communications Surveys & Tutorials.

[22]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[23]  T. Jukes,et al.  The neutral theory of molecular evolution. , 2000, Genetics.

[24]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[25]  Larry A. Rendell,et al.  The Feature Selection Problem: Traditional Methods and a New Algorithm , 1992, AAAI.

[26]  Andries Petrus Engelbrecht,et al.  A survey of techniques for characterising fitness landscapes and some possible ways forward , 2013, Inf. Sci..

[27]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..