Directional naive Bayes classifiers

Directional data are ubiquitous in science. These data have some special properties that rule out the use of classical statistics. Therefore, different distributions and statistics, such as the univariate von Mises and the multivariate von Mises–Fisher distributions, should be used to deal with this kind of information. We extend the naive Bayes classifier to the case where the conditional probability distributions of the predictive variables follow either of these distributions. We consider the simple scenario, where only directional predictive variables are used, and the hybrid case, where discrete, Gaussian and directional distributions are mixed. The classifier decision functions and their decision surfaces are studied at length. Artificial examples are used to illustrate the behavior of the classifiers. The proposed classifiers are then evaluated over eight datasets, showing competitive performances against other naive Bayes classifiers that use Gaussian distributions or discretization to manage directional data.

[1]  Kurt Hornik,et al.  On conjugate families and Jeffreys priors for von Mises–Fisher distributions , 2013, Journal of statistical planning and inference.

[2]  Richard S. Zemel,et al.  Lending direction to neural networks , 1995, Neural Networks.

[3]  Ronald E. Goldstein,et al.  Principles and techniques , 2009 .

[4]  J. Jossinet,et al.  Classification of breast tissue by electrical impedance spectroscopy , 2006, Medical and Biological Engineering and Computing.

[5]  A. Wood Simulation of the von mises fisher distribution , 1994 .

[6]  Prakash P. Shenoy,et al.  Inference in hybrid Bayesian networks using mixtures of polynomials , 2011, Int. J. Approx. Reason..

[7]  Adelaide Figueiredo,et al.  Discriminant Analysis for the von Mises-Fisher Distribution , 2009, Commun. Stat. Simul. Comput..

[8]  Denis J. Dean,et al.  Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables , 1999 .

[9]  Suvrit Sra,et al.  A short note on parameter approximation for von Mises-Fisher distributions: and a fast implementation of Is(x) , 2012, Comput. Stat..

[10]  Concha Bielza,et al.  The von Mises Naive Bayes Classifier for Angular Data , 2011, CAEPIA.

[11]  Michael I. Jordan,et al.  Variational methods for inference and estimation in graphical models , 1997 .

[12]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[13]  Doug Fisher,et al.  Learning from Data: Artificial Intelligence and Statistics V , 1996 .

[14]  S. R. Jammalamadaka,et al.  Topics in Circular Statistics , 2001 .

[15]  R. Fisher Dispersion on a sphere , 1953, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[16]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[17]  Pat Langley,et al.  Induction of Selective Bayesian Classifiers , 1994, UAI.

[18]  H. A. Guvenir,et al.  A supervised machine learning algorithm for arrhythmia analysis , 1997, Computers in Cardiology 1997.

[19]  Louis-Paul Rivest,et al.  Regression and correlation for 3 × 3 rotation matrices , 2006 .

[20]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[21]  Ian T. Jolliffe,et al.  Fitting mixtures of von Mises distributions: a case study involving sudden infant death syndrome , 2003, Comput. Stat. Data Anal..

[22]  Kanti V. Mardia,et al.  Statistics of Directional Data , 1972 .

[23]  Philipp Berens,et al.  CircStat: AMATLABToolbox for Circular Statistics , 2009, Journal of Statistical Software.

[24]  Remco R. Bouckaert,et al.  Estimating replicability of classifier learning experiments , 2004, ICML.

[25]  Michael J. Pazzani,et al.  Searching for Dependencies in Bayesian Classifiers , 1995, AISTATS.

[26]  Petr Savický,et al.  Methods for multidimensional event classification: A case study using images from a Cherenkov gamma-ray telescope , 2004 .

[27]  Mark A. Peot,et al.  Geometric Implications of the Naive Bayes Assumption , 1996, UAI.

[28]  G. L. D. Haas-Lorentz Die Brownsche Bewegung und einige verwandte Erscheinungen , 2022 .

[29]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[30]  Rafael Rumí,et al.  Learning hybrid Bayesian networks using mixtures of truncated exponentials , 2006, Int. J. Approx. Reason..

[31]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[32]  Bruno Bauwens,et al.  From circular ordinal regression to multilabel classification , 2010 .

[33]  Richard A. Johnson,et al.  Some Angular-Linear Distributions and Related Regression Models , 1978 .

[34]  Pedro Larrañaga,et al.  Supervised classification with conditional Gaussian networks: Increasing the structure complexity from naive Bayes , 2006, Int. J. Approx. Reason..

[35]  A. Agresti An introduction to categorical data analysis , 1990 .

[36]  S. Walker,et al.  A full Bayesian analysis of circular data using the von Mises distribution , 1999 .

[37]  Mohammed Waleed Kadous,et al.  Temporal classification: extending the classification paradigm to multivariate time series , 2002 .

[38]  W. L. Kovach,et al.  Quantitative methods for the study of lycopod megaspore ultrastructure , 1989 .

[39]  Nicholas I. Fisher,et al.  Statistical Analysis of Spherical Data. , 1987 .

[40]  P. J. Laycock,et al.  Discriminant analysis of directional data , 1974 .

[41]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[42]  J. Jossinet Variability of impedivity in normal and pathological breast tissue , 1996, Medical and Biological Engineering and Computing.

[43]  Francisco Herrera Triguero,et al.  An extension on "statistical comparisons of classifiers over multiple data sets" for all pairwise comparisons , 2008 .

[44]  P. Guttorp,et al.  Finding the Location of a Signal: A Bayesian Analysis , 1988 .

[45]  Grace S. Shieh,et al.  A CIRCULAR-CIRCULAR REGRESSION MODEL , 2008 .

[46]  Susanne Bottcher,et al.  Learning Bayesian networks with mixed variables , 2001, AISTATS.

[47]  A. SenGupta,et al.  A Classification Method for Directional Data with Application to the Human Skull , 2011 .

[48]  Nizar Bouguila,et al.  Beyond hybrid generative discriminative learning: spherical data classification , 2013, Pattern Analysis and Applications.

[49]  Mehran Sahami,et al.  Learning Limited Dependence Bayesian Classifiers , 1996, KDD.

[50]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[51]  Paul Levy,et al.  L'addition des variables aléatoires définies sur une circonférence , 1939 .

[52]  Kanti V. Mardia,et al.  Bayesian analysis for bivariate von Mises distributions , 2010 .

[53]  Carmelo Rodríguez,et al.  Selective Naive Bayes for Regression Based on Mixtures of Truncated Exponentials , 2007, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[54]  M. Begg An introduction to categorical data analysis (2nd edn). Alan Agresti, John Wiley & Sons, Inc., Hoboken, New Jersey, 2007. No. of Pages: 400. Price: $100.95. ISBN: 978‐0‐471‐22618‐5 , 2009 .

[55]  Franz Streit,et al.  Identification analysis in directional statistics , 1996 .

[56]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[57]  Inderjit S. Dhillon,et al.  Clustering on the Unit Hypersphere using von Mises-Fisher Distributions , 2005, J. Mach. Learn. Res..

[58]  A. SenGupta,et al.  A Simple Classification Rule for Directional Data , 2005 .

[59]  Marvin Minsky,et al.  Steps toward Artificial Intelligence , 1995, Proceedings of the IRE.

[60]  K. Mardia On some recent advancements in applied shape analysis and directional statistics , 2007 .

[61]  P. Sprent,et al.  Statistical Analysis of Circular Data. , 1994 .

[62]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[63]  Leonard E. Trigg,et al.  Technical Note: Naive Bayes for Regression , 2000, Machine Learning.

[64]  Nicholas I. Fisher,et al.  Statistical Analysis of Circular Data , 1993 .

[65]  Peter B. Krenesky,et al.  Protein Geometry Database: a flexible engine to explore backbone conformations and their relationships to covalent geometry , 2009, Nucleic Acids Res..

[66]  Alan J. Lee,et al.  Regression Models for an Angular Response , 1992 .

[67]  F. Perrin,et al.  Étude mathématique du mouvement brownien de rotation , 1928 .

[68]  Nicholas I. Fisher,et al.  Statistical Analysis of Spherical Data. , 1987 .

[69]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[70]  Adelaide Figueiredo,et al.  Discriminant analysis based on the Watson distribution defined on the hypersphere , 2006 .