Extended adaptive Lasso for multi-class and multi-label feature selection

Abstract Feature selection is a basic step and important task in applications of pattern recognition and machine learning. In this paper, we propose a new Extended Adaptive Least absolute shrinkage and selection operator (EALasso) feature selection method for multi-class and multi-label learning problems. It preserves the oracle properties of identifying the correct subset model and having the optimal estimation accuracy. An iterative optimization algorithm, with theoretical proof of convergence, is proposed to solve the optimization problem. Experimental results on several real multi-class and multi-label data sets show that the proposed EALasso method outperforms many state-of-the-art feature selection methods for multi-class single-label classification as well as multi-class multi-label classification.

[1]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[2]  E. Lander,et al.  Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclasses , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Marcel Worring,et al.  The challenge problem for automated detection of 101 semantic concepts in multimedia , 2006, MM '06.

[4]  Rami N. Khushaba,et al.  Feature subset selection using differential evolution and a wheel based search strategy , 2013, Swarm Evol. Comput..

[5]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[6]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[7]  Xindong Wu,et al.  Online feature selection for high-dimensional class-imbalanced data , 2017, Knowl. Based Syst..

[8]  Pan Su,et al.  Feature Selection Ensemble , 2012, Turing-100.

[9]  T. Golub,et al.  Gene expression-based classification of malignant gliomas correlates better with survival than histological classification. , 2003, Cancer research.

[10]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Dae-Won Kim,et al.  Feature selection for multi-label classification using multivariate mutual information , 2013, Pattern Recognit. Lett..

[12]  Nicu Sebe,et al.  Feature Selection for Multimedia Analysis by Sharing Information Among Multiple Tasks , 2013, IEEE Transactions on Multimedia.

[13]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[14]  Dae-Won Kim,et al.  Memetic feature selection algorithm for multi-label classification , 2015, Inf. Sci..

[15]  Jianzhong Li,et al.  A stable gene selection in microarray data analysis , 2006, BMC Bioinformatics.

[16]  Feiping Nie,et al.  Feature Selection at the Discrete Limit , 2014, AAAI.

[17]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[18]  M. Ringnér,et al.  Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks , 2001, Nature Medicine.

[19]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[20]  Víctor Robles,et al.  Feature selection for multi-label naive Bayes classification , 2009, Inf. Sci..

[21]  P. Langley.  Selection of Relevant Features in Machine Learning , 1994 .

[22]  Nicu Sebe,et al.  Web Image Annotation Via Subspace-Sparsity Collaborated Feature Selection , 2012, IEEE Transactions on Multimedia.

[23]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[24]  Y. Zhou,et al.  A cDNA microarray gene expression database for cancer drug discovery , 1999, Nature Genetics.

[25]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[26]  Francisco Herrera,et al.  A multi-objective evolutionary approach to training set selection for support vector machine , 2018, Knowl. Based Syst..

[27]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[28]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[29]  Chris H. Q. Ding,et al.  Minimum redundancy feature selection from microarray gene expression data , 2003, Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003.

[30]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[31]  Yuwen Li,et al.  Attribute reduction for multi-label learning with fuzzy rough set , 2018, Knowl. Based Syst..

[32]  Song Xiang.  A Multi-Label Learning Algorithm Based on Sparse Representation , 2012 .

[33]  Gil Alterovitz,et al.  Subtype dependent biomarker identification and tumor classification from gene expression profiles , 2018, Knowl. Based Syst..

[34]  Kemal Kilic,et al.  A Novel Hybrid Genetic Local Search Algorithm for Feature Selection and Weighting with an Application in Strategic Decision Making in Innovation Management , 2017, Inf. Sci..

[35]  Kilian Stoffel,et al.  Theoretical Comparison between the Gini Index and Information Gain Criteria , 2004, Annals of Mathematics and Artificial Intelligence.

[36]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[37]  Cun-Hui Zhang,et al.  Adaptive Lasso for sparse high-dimensional regression models , 2008 .

[38]  Ghulam Muhammad,et al.  Feature Selection Based Verification/Identification System Using Fingerprints and Palm Print , 2013 .

[39]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[40]  Ben Niu,et al.  A discrete bacterial algorithm for feature selection in classification of microarray gene expression cancer data , 2017, Knowl. Based Syst..