This study presents a novel performance metric for feature selection algorithms that is unbiased and can be used for comparative analysis across feature selection problems. The baseline fitness improvement (BFI) measure quantifies the potential value gained by applying feature selection. The BFI measure can be used to compare the performance of feature selection algorithms across datasets by measuring the change in classifier performance as a result of feature selection, with respect to the baseline where all features are included. Empirical results are presented to show that there is performance complementarity for a suite of feature selection algorithms on a variety of real world datasets. The BFI measure is a normalised performance metric that can be used to correlate problem characteristics with feature selection algorithm performance, across multiple datasets. This ability paves the way towards describing the performance space of the per-instance algorithm selection problem for feature selection algorithms.
[1]
David M. W. Powers,et al.
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
,
2011,
ArXiv.
[2]
Lars Kotthoff,et al.
Algorithm Selection for Combinatorial Search Problems: A Survey
,
2012,
AI Mag..
[3]
David W. Aha,et al.
A Comparative Evaluation of Sequential Feature Selection Algorithms
,
1995,
AISTATS.
[4]
Huan Liu,et al.
Advancing feature selection research
,
2010
.
[5]
John R. Rice,et al.
The Algorithm Selection Problem
,
1976,
Adv. Comput..
[6]
Andries Petrus Engelbrecht,et al.
Insights into the Feature Selection Problem Using Local Optima Networks
,
2019,
EvoCOP.
[7]
Isabelle Guyon,et al.
An Introduction to Variable and Feature Selection
,
2003,
J. Mach. Learn. Res..