Bayesian Backfitting

We propose general procedures for posterior sampling from additive and generalized additive models. The procedure is a stochastic generalization of the well-known backfitting algorithm for fitting additive models. One chooses a linear operator (“smoother”) for each predictor, and the algorithm requires only the application of the operator and its square root. The procedure is general and modular, and we describe its application to nonparametric, semiparametric and mixed models.

[1]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[2]  C. R. Deboor,et al.  A practical guide to splines , 1978 .

[3]  W. W. Muir,et al.  Regression Diagnostics: Identifying Influential Data and Sources of Collinearity , 1980 .

[4]  J. Ware,et al.  Random-effects models for longitudinal data. , 1982, Biometrics.

[5]  G. Wahba.  Bayesian "Confidence Intervals" for the Cross-validated Smoothing Spline , 1983 .

[6]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  T Poggio,et al.  Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks , 1990, Science.

[8]  G. Wahba.  Spline models for observational data , 1990 .

[9]  P. Green.  Bayesian reconstructions from emission tomography data using a modified EM algorithm. , 1990, IEEE transactions on medical imaging.

[10]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[11]  Scott L. Zeger,et al.  Generalized linear models with random e ects: a Gibbs sampling approach , 1991 .

[12]  G. Robinson.  That BLUP is a Good Thing: The Estimation of Random Effects , 1991 .

[13]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[14]  Trevor Hastie,et al.  Statistical Models in S , 1991 .

[15]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[16]  STATE SPACE MODELS WITH DIFFUSE INITIAL CONDITIONS , 1992 .

[17]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[18]  R. Dennis Cook,et al.  Exploring Partial Residual Plots , 1993 .

[19]  J. Friedman,et al.  [A Statistical View of Some Chemometrics Regression Tools]: Response , 1993 .

[20]  B. Silverman,et al.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[21]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[22]  Jun S. Liu,et al.  Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes , 1994 .

[23]  R. Kohn,et al.  On Gibbs sampling for state space models , 1994 .

[24]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[25]  Geoffrey E. Hinton,et al.  An Alternative Model for Mixtures of Experts , 1994, NIPS.

[26]  Carl E. Rasmussen,et al.  In Advances in Neural Information Processing Systems , 2011 .

[27]  A. Gelfand,et al.  Efficient parametrisations for normal linear mixed models , 1995 .

[28]  R. D. Cook.  Graphics for studying the net effects of regression predictors , 1995 .

[29]  R. Tweedie,et al.  Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms , 1996 .

[30]  P. Müller,et al.  Bayesian curve fitting using multivariate normal mixtures , 1996 .

[31]  Robert Kohn,et al.  A BAYESIAN APPROACH TO ESTIMATING AND FORECASTING ADDITIVE NONPARAMETRIC AUTOREGRESSIVE MODELS , 1996 .

[32]  David J. Spiegelhalter,et al.  Hepatitis B: a case study in MCMC methods , 1996 .

[33]  G. Casella,et al.  The Effect of Improper Priors on Gibbs Sampling in Hierarchical Linear Mixed Models , 1996 .

[34]  G. Roberts,et al.  Updating Schemes, Correlation Structure, Blocking and Parameterization for the Gibbs Sampler , 1997 .

[35]  R. Dennis Cook,et al.  Graphics for Assessing the Adequacy of Regression Models , 1997 .

[36]  Stefan Schaal,et al.  Local Dimensionality Reduction , 1997, NIPS.

[37]  J. Heikkinen,et al.  Non‐parametric Bayesian Estimation of a Spatial Poisson Intensity , 1998 .

[38]  Robert Kohn,et al.  Additive nonparametric regression with autocorrelated errors , 1998 .

[39]  Adrian F. M. Smith,et al.  Automatic Bayesian curve fitting , 1998 .

[40]  R. H. Moore,et al.  Regression Graphics: Ideas for Studying Regressions Through Graphics , 1998, Technometrics.

[41]  R. Cook,et al.  Dimension Reduction in Binary Response Regression , 1999 .

[42]  A. Gelfand,et al.  Identifiability, Improper Priors, and Gibbs Sampling for Generalized Linear Models , 1999 .

[43]  UsingSmoothing SplinesbyXihong Liny,et al.  Inference in Generalized Additive Mixed Models , 1999 .

[44]  Zoubin Ghahramani,et al.  Variational Inference for Bayesian Mixtures of Factor Analysers , 1999, NIPS.

[45]  Bani K. Mallick,et al.  Bayesian wavelet networks for nonparametric regression , 2000, IEEE Trans. Neural Networks Learn. Syst..

[46]  J. Brian Gray,et al.  Applied Regression Including Computing and Graphics , 1999, Technometrics.