Local discriminative learning for pattern recognition

Abstract Local discriminative learning methods approximate a target function (a posteriori class probability function) directly by partitioning the feature space into a set of local regions, and appropriately modeling a simple input–output relationship (function) in each one. This paper presents a new method for judiciously partitioning the input feature space in order to accurately represent the target function. The method accomplishes this by approximating not only the target function itself but also its derivatives. As such, the method partitions the input feature space along those dimensions for which the class probability function changes most rapidly, thus minimizing bias. The efficacy of the method is validated using a variety of simulated and real-world data.

[1]  Tin Kam Ho,et al.  Nearest Neighbors in Random Subspaces , 1998, SSPR/SPR.

[2]  David G. Lowe,et al.  Similarity Metric Learning for a Variable-Kernel Classifier , 1995, Neural Computation.

[3]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[4]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[5]  Trevor J. Hastie,et al.  Discriminative vs Informative Learning , 1997, KDD.

[6]  Shree K. Nayar,et al.  Pattern rejection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Léon Bottou,et al.  Local Learning Algorithms , 1992, Neural Computation.

[8]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[9]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[10]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[11]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[12]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[13]  Donald Ylvisaker,et al.  Prediction and Design , 1987 .

[14]  Simon Kasif,et al.  A System for Induction of Oblique Decision Trees , 1994, J. Artif. Intell. Res..

[15]  Toshio Odanaka,et al.  ADAPTIVE CONTROL PROCESSES , 1990 .

[16]  R.P. Lippmann,et al.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[17]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[18]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[19]  J. Weng.  Cresceptron and Shoslif: toward Comprehensive Visual Learning 1 , 1996 .

[20]  Jing Peng,et al.  Efficient Memory-Based Dynamic Programming , 1995, ICML.

[21]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[22]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.