Bayesian Generalized Kernel Models

We propose a fully Bayesian approach for generalized kernel models (GKMs), which are extensions of generalized linear models in the feature space induced by a reproducing kernel. We place a mixture of a point-mass distribution and Silverman’s g-prior on the regression vector of GKMs. This mixture prior allows a fraction of the regression vector to be zero. Thus, it serves for sparse modeling and Bayesian computation. For inference, we exploit data augmentation methodology to develop a Markov chain Monte Carlo (MCMC) algorithm in which the reversible jump method is used for model selection and a Bayesian model averaging method is used for posterior prediction.

[1]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[2]  R. Parker,et al.  Discussion of Dr Silverman''s paper , 1985 .

[3]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[4]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[5]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[6]  R. Kohn,et al.  Nonparametric regression using Bayesian variable selection , 1996 .

[7]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[8]  D. Madigan,et al.  Bayesian Model Averaging for Linear Regression Models , 1997 .

[9]  E. George,et al.  APPROACHES FOR BAYESIAN VARIABLE SELECTION , 1997 .

[10]  Ji Zhu,et al.  Kernel Logistic Regression and the Import Vector Machine , 2001, NIPS.

[11]  Robert Kohn,et al.  Nonparametric regression using linear combinations of basis functions , 2001, Stat. Comput..

[12]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[13]  Matthew West,et al.  Bayesian factor regression models in the''large p , 2003 .

[14]  Mário A. T. Figueiredo.  Adaptive Sparseness for Supervised Learning , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[16]  Marina Vannucci,et al.  Bayesian Variable Selection in Multinomial Probit Models to Identify Molecular Signatures of Disease Stage , 2004, Biometrics.

[17]  Peter Sollich,et al.  Bayesian Methods for Support Vector Machines: Evidence and Predictive Class Probabilities , 2002, Machine Learning.

[18]  P. Green,et al.  Bayesian Variable Selection and the Swendsen-Wang Algorithm , 2004 .

[19]  B. Mallick,et al.  Bayesian classification of tumours by using gene expression data , 2005 .

[20]  Zhihua Zhang,et al.  Bayesian Multicategory Support Vector Machines , 2006, UAI.

[21]  C. Holmes,et al.  Bayesian auxiliary variable models for binary and multinomial regression , 2006 .

[22]  Zhihua Zhang,et al.  Posterior Consistency of the Silverman g-prior in Bayesian Model Choice , 2008, NIPS.