Weighted Order Statistic Classifiers with Large Rank-Order Margin

We investigate how stack filter function classes like weighted order statistics can be applied to classification problems. This leads to a new design criteria for linear classifiers when inputs are binary-valued and weights are positive. We present a rank-based measure of margin that is directly optimized as a standard linear program and investigate its relationship to regularization. Our approach can robustly combine large numbers of base hypothesis and has similar performance to other types of regularization.

[1]  Adam Cannon,et al.  Simple classifiers from data dependent hypothesis classes , 2003 .

[2]  Gerald W. Kimble,et al.  Information and Computer Science , 1975 .

[3]  Edward J. Coyle,et al.  Minimum mean absolute error estimation over the class of generalized stack filters , 1990, IEEE Trans. Acoust. Speech Signal Process..

[4]  A. Sharkey.  Linear and Order Statistics Combiners for Pattern Classification , 1999 .

[5]  P.-F. YANG,et al.  Min-max classifiers: Learnability, design and application , 1995, Pattern Recognit..

[6]  J. Astola,et al.  Fundamentals of Nonlinear Digital Filtering , 1997 .

[7]  Yrjö Neuvo,et al.  Fast adaptation and performance characteristics of FIR-WOS hybrid filters , 1994, IEEE Trans. Signal Process..

[8]  Dale Schuurmans,et al.  Boosting in the Limit: Maximizing the Margin of Learned Ensembles , 1998, AAAI/IAAI.

[9]  Jr. Kendall Preston,et al.  Detection of weak, subpixel targets using mesh-connected cellular automata , 1990 .

[10]  J. Fitch,et al.  Median filtering by threshold decomposition , 1984 .

[11]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[12]  Gonzalo R. Arce,et al.  Optimization of stack filters based on mirrored threshold decomposition , 2001, IEEE Trans. Signal Process..

[13]  K. Chen.  Bit-serial realizations of a class of nonlinear filters based on positive Boolean functions , 1989 .

[14]  Peter Sussner,et al.  An introduction to morphological neural networks , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[15]  Edward J. Coyle,et al.  Stack filters , 1986, IEEE Trans. Acoust. Speech Signal Process..

[16]  Peter L. Bartlett,et al.  Improved Generalization Through Explicit Optimization of Margins , 2000, Machine Learning.

[17]  James Theiler,et al.  Grafting: Fast, Incremental Feature Selection by Gradient Descent in Function Space , 2003, J. Mach. Learn. Res..

[18]  Edward J. Coyle,et al.  Stack filter lattices , 1994, Signal Process..

[19]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[20]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[21]  Jaakko Astola,et al.  Analysis of the properties of median and weighted median filters using threshold logic and stack filter representation , 1991, IEEE Trans. Signal Process..

[22]  Gonzalo R. Arce,et al.  A general weighted median filter structure admitting negative weights , 1998, IEEE Trans. Signal Process..