Semiparametric latent factor models

We propose a semiparametric model for regression and classification problems involving multiple response variables. The model makes use of a set of Gaussian processes to model the relationship to the inputs in a nonparametric fashion. Conditional dependencies between the responses can be captured through a linear mixture of the driving processes. This feature becomes important if some of the responses of predictive interest are less densely supplied by observed data than related auxiliary ones. We propose an efficient approximate inference scheme for this semiparametric model whose complexity is linear in the number of training data points.

[1]  Vladimir Vapnik,et al.  Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics) , 1982 .

[2]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[3]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[4]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[5]  Radford M. Neal.  Priors for Infinite Networks , 1996 .

[6]  J. Friedman,et al.  Predicting Multivariate Responses in Multiple Linear Regression , 1997 .

[7]  U. Menzefricke.  Hierarchical modeling with gaussian processes , 2000 .

[8]  Ole Winther,et al.  Gaussian Processes for Classification: Mean-Field Algorithms , 2000, Neural Computation.

[9]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[10]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[11]  Neil D. Lawrence,et al.  Fast Sparse Gaussian Process Methods: The Informative Vector Machine , 2002, NIPS.

[12]  Ben Taskar,et al.  Max-Margin Markov Networks , 2003, NIPS.

[13]  Matthias W. Seeger,et al.  Bayesian Gaussian process models : PAC-Bayesian generalisation error bounds and sparse approximations , 2003 .

[14]  Martial Hebert,et al.  Discriminative random fields: a discriminative framework for contextual interaction in classification , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[15]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[16]  Ankur Agarwal,et al.  3D human pose from silhouettes by relevance vector regression , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[17]  Neil D. Lawrence,et al.  Learning to learn with the informative vector machine , 2004, ICML.

[18]  M. Seeger.  Low Rank Updates for the Cholesky Decomposition , 2004 .

[19]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[20]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[21]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .