Ensembles of extremely randomized trees and some generic applications

In this paper we present a new tree-based en- semble method called "Extra-Trees". This algorithm aver- ages predictions of trees obtained by partitioning the input- space with randomly generated splits, leading to significant improvements of precision, and various algorithmic advan- tages, in particular reduced computational complexity and scalability. We also discuss two generic applications of this algorithm, namely for time-series classification and for the automatic inference of near-optimal sequential decision poli- cies from experimental data.

[1]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[2]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[5]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[6]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[7]  Mathukumalli Vidyasagar,et al.  A Theory of Learning and Generalization: With Applications to Neural Networks and Control Systems , 1997 .

[8]  Pierre Geurts,et al.  Segment and Combine Approach for Non-parametric Time-Series Classification , 2005, PKDD.

[9]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[10]  Raphaël Marée,et al.  Random subwindows for robust image classification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Pierre Geurts,et al.  Tree-Based Batch Mode Reinforcement Learning , 2005, J. Mach. Learn. Res..

[12]  T. Poggio,et al.  General conditions for predictivity in learning theory , 2004, Nature.

[13]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[14]  D. Ernst,et al.  Automatic learning of sequential decision strategies for dynamic security assessment and control , 2006, 2006 IEEE Power Engineering Society General Meeting.

[15]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[17]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[18]  Leo Breiman,et al.  Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) , 2001, Statistical Science.

[19]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[20]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[21]  Pierre Geurts,et al.  Kernelizing the output of tree-based methods , 2006, ICML '06.

[22]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[23]  Raphaël Marée,et al.  Segment and combine: a generic approach for supervised learning of invariant classifiers from topologically structured data , 2006 .

[24]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[25]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[26]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[27]  Mathukumalli Vidyasagar,et al.  A Theory of Learning and Generalization , 1997 .