A novel dynamic rough subspace based selective ensemble

Ensemble learning has been a hot topic in machine learning due to its successful utilization in many applications. Rough set theory has been proved to be an excellent mathematical tool for dimension reduction. In this paper, based on rough set, a novel framework for ensemble is proposed. In our proposed framework, the relationship among attributes in rough subspace is first considered, and the maximum dependency degree of attribute is first employed to effectively reduce the searching space of reducts and augment the diversity of selected reducts. In addition, in order to choose an appropriate reduct from the dynamic reduct searching space, an assessment function which can balance the accuracy and diversity is utilized. At last, a new method, i.e., Dynamic Rough Subspace based Selective Ensemble (DRSSE), which is derived from our framework is given. By repeatedly changing the searching space of reducts and selecting the next reduct from the changed searching space, DRSSE finally trains an ensemble system with these selected reducts. Compared with several available ensemble methods, experimental results with several datasets demonstrate that DRSSE can lead to a comparative or even better performance. A new framework for rough set ensemble and algorithm DRSSE is proposed.Dynamic searching space is used to increase the diversity of rough subspaces.The relationship among attributes is considered to reduce the searching space.Consider the accuracy and diversity of base classifiers in an ensemble system.

[1]  Rajendra Akerkar,et al.  Knowledge Based Systems , 2009, Encyclopedia of GIS.

[2]  K. Thangavel,et al.  Dimensionality reduction based on rough set theory: A review , 2009, Appl. Soft Comput..

[3]  Robert P. W. Duin,et al.  An experimental study on diversity for bagging and boosting with linear classifiers , 2002, Inf. Fusion.

[4]  Jan G. Bazan,et al.  Rough set algorithms in classification problem , 2000 .

[5]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[6]  Yiyu Yao,et al.  Rough Sets: Selected Methods and Applications in Management and Engineering , 2012, Advanced Information and Knowledge Processing.

[7]  Xin Yao,et al.  Diversity creation methods: a survey and categorisation , 2004, Inf. Fusion.

[8]  Lei Xi,et al.  Rough set and ensemble learning based semi-supervised algorithm for text classification , 2011, Expert Syst. Appl..

[9]  Qiang Shen,et al.  Selecting informative features with fuzzy-rough sets and its application for complex systems monitoring , 2004, Pattern Recognit..

[10]  Qinghua Hu,et al.  EROS: Ensemble rough subspaces , 2007, Pattern Recognit..

[11]  Naohiro Ishii,et al.  Control of Variables in Reducts - kNN Classification with Confidence , 2011, KES.

[12]  Jie Gui,et al.  Tumor classification by combining PNN classifier ensemble with neighborhood rough set based gene reduction , 2010, Comput. Biol. Medicine.

[13]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[14]  Zdzisław Pawlak,et al.  Rough set theory and its applications , 2002, Journal of Telecommunications and Information Technology.

[15]  Aleksander Øhrn,et al.  Discernibility and Rough Sets in Medicine: Tools and Applications , 2000 .

[16]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Decui Liang,et al.  Incorporating logistic regression to decision-theoretic rough sets for classifications , 2014, Int. J. Approx. Reason..

[18]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[19]  Wei Tang,et al.  Ensembling neural networks: Many could be better than all , 2002, Artif. Intell..

[20]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[22]  Xinyu Shao,et al.  Integration of rough set and neural network ensemble to predict the configuration performance of a modular product family , 2010 .

[23]  S. K. Michael Wong,et al.  Rough Sets: Probabilistic versus Deterministic Approach , 1988, Int. J. Man Mach. Stud..

[24]  Pawan Lingras,et al.  Unsupervised Rough Set Classification Using GAs , 2001, Journal of Intelligent Information Systems.

[25]  Qingxiang Wu,et al.  Multiknowledge for decision making , 2005, Knowledge and Information Systems.

[26]  Staal A. Vinterbo,et al.  Minimal approximate hitting sets and rule templates , 2000, Int. J. Approx. Reason..

[27]  Jaya Sil,et al.  An efficient classifier design integrating rough set and set oriented database operations , 2011, Appl. Soft Comput..

[28]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[29]  Jemal H. Abawajy,et al.  A rough set approach for selecting clustering attribute , 2010, Knowl. Based Syst..

[30]  C. A. Murthy,et al.  Classification of Web Services Using Tensor Space Model and Rough Ensemble Classifier , 2008, ISMIS.

[31]  Yasuo Kudo,et al.  A sequential pattern mining algorithm using rough set theory , 2011, Int. J. Approx. Reason..

[32]  Lei Xi,et al.  A novel ensemble algorithm for biomedical classification based on Ant Colony Optimization , 2011, Appl. Soft Comput..

[33]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[34]  Sushmita Mitra,et al.  Feature Selection, Classification and Rule Generation Using Rough Sets , 2012 .

[35]  Marek Kurzynski,et al.  Optimal selection of ensemble classifiers using measures of competence and diversity of base classifiers , 2014, Neurocomputing.

[36]  Thomas G. Dietterich.  Machine-Learning Research , 1997, AI Mag..

[37]  Qinghua Hu,et al.  Feature Selection for Monotonic Classification , 2012, IEEE Transactions on Fuzzy Systems.

[38]  Hui Zhao,et al.  Intrusion Detection Ensemble Algorithm based on Bagging and Neighborhood Rough Set , 2013 .

[39]  Zhi-Hua Zhou,et al.  Exploiting unlabeled data to enhance ensemble diversity , 2009, 2010 IEEE International Conference on Data Mining.

[40]  Janusz Zalewski,et al.  Rough sets: Theoretical aspects of reasoning about data , 1996 .

[41]  XiongTao,et al.  A novel dynamic rough subspace based selective ensemble , 2015 .

[42]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[43]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[45]  Jiawei Han,et al.  Semi-supervised Discriminant Analysis , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[46]  Andrzej Skowron,et al.  The Discernibility Matrices and Functions in Information Systems , 1992, Intelligent Decision Support.

[47]  David W. Opitz,et al.  Feature Selection for Ensembles , 1999, AAAI/IAAI.

[48]  Tan Yee Fan,et al.  A Tutorial on Support Vector Machine , 2009 .

[49]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[50]  Jianhua Dai,et al.  Attribute selection based on information gain ratio in fuzzy rough set theory with application to tumor classification , 2013, Appl. Soft Comput..

[51]  Jaime G. Carbonell,et al.  Machine learning research , 1981, SGAR.