Interval Estimation Naïve Bayes

Recent work in supervised learning has shown that a surprisingly simple Bayesian classifier called naive Bayes is competitive with state of the art classifiers. This simple approach stands from assumptions of conditional independence among features given the class. In this paper a new naive Bayes classifier called Interval Estimation naive Bayes is proposed. Interval Estimation naive Bayes is performed in two phases. First, an interval estimation of each probability necessary to specify the naive Bayes is calculated. On the second phase the best combination of values inside these intervals is calculated using a heuristic search that is guided by the accuracy of the classifiers. The founded values in the search are the new parameters for the naive Bayes classifier. Our new approach has shown to be quite competitive related to simple naive Bayes. Experimental tests have been done with 21 data sets from the UCI repository.

[1]  Paola Sebastiani,et al.  Robust Learning with Missing Data , 2001, Machine Learning.

[2]  Ron Kohavi,et al.  Improving simple Bayes , 1997 .

[3]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[4]  Irving John Good,et al.  The Estimation of Probabilities: An Essay on Modern Bayesian Methods , 1965 .

[5]  Ron Kohavi,et al.  MLC++: a machine learning library in C++ , 1994, Proceedings Sixth International Conference on Tools with Artificial Intelligence. TAI 94.

[6]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[7]  D. Hand,et al.  Idiot's Bayes—Not So Stupid After All? , 2001 .

[8]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[9]  Geoffrey I. Webb,et al.  Adjusted Probability Naive Bayesian Induction , 1998, Australian Joint Conference on Artificial Intelligence.

[10]  Pat Langley,et al.  Induction of Selective Bayesian Classifiers , 1994, UAI.

[11]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[12]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[13]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[14]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[15]  Pat Langley,et al.  Induction of Recursive Bayesian Classifiers , 1993, ECML.

[16]  Michael J. Pazzani,et al.  Searching for Dependencies in Bayesian Classifiers , 1995, AISTATS.

[17]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[18]  João Gama,et al.  Iterative Bayes , 2000, Intell. Data Anal..

[19]  Igor Kononenko,et al.  Semi-Naive Bayesian Classifier , 1991, EWSL.

[20]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[21]  Pedro M. Domingos,et al.  Beyond Independence: Conditions for the Optimality of the Simple Bayesian Classifier , 1996, ICML.

[22]  Marco Zaffalon,et al.  Credal Classification for Dementia Screening , 2001, AIME.