Open Problems in Geometric Methods for Instance-Based Learning

In the typical approach to instance-based learning, random data (the training set of patterns) are collected and used to design a decision rule (classifier). One of the most well known such rules is the k-nearest-neighbor decision rule in which an unknown pattern is classified into the majority class among its k nearest neighbors in the training set. In the past fifty years many approaches have been proposed to improve the performance of this rule. More recently geometric methods have been found to be the best. Here we mention a variety of open problems of a computational geometric nature that arize in these methods. To provide some context and motivation for these open problems we briefly describe the methods and list some key references.

[1]  Gordon T. Wilfong Nearest neighbor problems , 1991, SCG '91.

[2]  Godfried T. Toussaint,et al.  PATTERN RECOGNITION AND GEOMETRICAL COMPLEXITY. , 1980 .

[3]  Luc Devroye,et al.  On the Inequality of Cover and Hart in Nearest Neighbor Discrimination , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  C. J. Stone,et al.  Optimal Rates of Convergence for Nonparametric Estimators , 1980 .

[5]  Godfried T. Toussaint,et al.  The relative neighbourhood graph of a finite planar set , 1980, Pattern Recognit..

[6]  Chris Mellish,et al.  On the Consistency of Information Filters for Lazy Learning Algorithms , 1999, PKDD.

[7]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[8]  G. McLachlan Discriminant Analysis and Statistical Pattern Recognition , 1992 .

[9]  中澤 真,et al.  Devroye, L., Gyorfi, L. and Lugosi, G. : A Probabilistic Theory of Pattern Recognition, Springer (1996). , 1997 .

[10]  Leo A. Goodman,et al.  Corrigenda: Measures of Association for Cross Classifications , 1957 .

[11]  Manabu Ichino,et al.  The relative neighborhood graph for mixed feature variables , 1985, Pattern Recognit..

[12]  Giuseppe Liotta,et al.  The rectangle of influence drawability problem , 1996, Comput. Geom..

[13]  Richard Nock,et al.  Boosting Neighborhood-Based Classifiers , 2001, ICML.

[14]  Tony R. Martinez,et al.  Instance Pruning Techniques , 1997, ICML.

[15]  Godfried T. Toussaint,et al.  Some new algorithms and software implementation methods for pattern recognition research , 1979, COMPSAC.

[16]  Filiberto Pla,et al.  On the use of neighbourhood-based non-parametric classifiers , 1997, Pattern Recognit. Lett..

[17]  David L. Waltz,et al.  Toward memory-based reasoning , 1986, CACM.

[18]  Godfried T. Toussaint Comment: Algorithms for computing relative neighbourhood graph , 1980 .

[19]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[20]  L. A. Goodman,et al.  Measures of association for cross classifications , 1979 .

[21]  Charles T. Zahn,et al.  Graph-Theoretical Methods for Detecting and Describing Gestalt Clusters , 1971, IEEE Transactions on Computers.

[22]  Sanjeev R. Kulkarni,et al.  Learning Pattern Classification - A Survey , 1998, IEEE Trans. Inf. Theory.

[23]  Tony R. Martinez,et al.  Reduction Techniques for Instance-Based Learning Algorithms , 2000, Machine Learning.

[24]  F. Frances Yao,et al.  On Nearest-Neighbor Graphs , 1992, ICALP.

[25]  G. Gates The Reduced Nearest Neighbor Rule , 1998 .

[26]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[27]  Peter L. Hammer,et al.  Distance-Based Classification Methods , 1999 .

[28]  Larry D. Hostetler,et al.  k-nearest-neighbor Bayes-risk estimation , 1975, IEEE Trans. Inf. Theory.

[29]  G. Gates,et al.  The reduced nearest neighbor rule (Corresp.) , 1972, IEEE Trans. Inf. Theory.

[30]  F. Pla,et al.  Improving the k-NCN classification rule through heuristic modifications , 1998, Pattern Recognit. Lett..

[31]  Chris Mellish,et al.  Advances in Instance Selection for Instance-Based Learning Algorithms , 2002, Data Mining and Knowledge Discovery.

[32]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[33]  Godfried T. Toussaint,et al.  Bibliography on estimation of misclassification , 1974, IEEE Trans. Inf. Theory.

[34]  Peter E. Hart,et al.  The condensed nearest neighbor rule (Corresp.) , 1968, IEEE Trans. Inf. Theory.

[35]  Demetri Psaltis,et al.  On the finite sample performance of the nearest neighbor classifier , 1993, IEEE Trans. Inf. Theory.

[36]  Godfried T. Toussaint,et al.  Relative neighborhood graphs and their relatives , 1992, Proc. IEEE.

[37]  David G. Stork,et al.  Pattern Classification , 1973 .

[38]  Godfried T. Toussaint ALGORITHMS FOR COMPUTING RELATIVE NEIGHBOURHOOD GRAPH. , 1980 .

[39]  Hugh B. Woodruff,et al.  An algorithm for a selective nearest neighbor decision rule (Corresp.) , 1975, IEEE Trans. Inf. Theory.