Latent Autoregressive Gaussian Processes Models for Robust System Identification

Abstract We introduce GP-RLARX, a novel Gaussian Process (GP) model for robust system identification. Our approach draws inspiration from nonlinear autoregressive modeling with exogenous inputs (NARX) and it encapsulates a novel and powerful structure referred to as latent autoregression. This structure accounts for the feedback of uncertain values during training and provides a natural framework for free simulation prediction. By using a Student-t likelihood, GP-RLARX can be used in scenarios where the estimation data contain non-Gaussian noise in the form of outliers. Further, a variational approximation scheme is developed to jointly optimize all the hyperparameters of the model from available estimation data. We perform experiments with five widely used artificial benchmarking datasets with different levels of outlier contamination and compare GP-RLARX with the standard GP-NARX model and its robust variant, GP-tVB. GP-RLARX is found to outperform the competing models by a relatively wide margin, indicating that our latent autoregressive structure is more suitable for robust system identification.

[1]  Andreas C. Damianou,et al.  Deep Gaussian processes and variational propagation of uncertainty , 2015 .

[2]  Carl E. Rasmussen,et al.  Integrated pre-processing for Bayesian nonlinear system identification with Gaussian processes , 2013, 52nd IEEE Conference on Decision and Control.

[3]  Agathe Girard,et al.  Dynamic systems identification with Gaussian processes , 2005 .

[4]  Carl E. Rasmussen,et al.  Gaussian Process Training with Input Noise , 2011, NIPS.

[5]  Neil D. Lawrence,et al.  Variational inference for Student-t models: Robust Bayesian interpolation and generalised component analysis , 2005, Neurocomputing.

[6]  K. Obermayer,et al.  Multiple-step ahead prediction for non linear dynamic systems: A Gaussian Process treatment with propagation of the uncertainty , 2003, NIPS 2003.

[7]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[8]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[9]  S. Billings.  Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains , 2013 .

[10]  Neil D. Lawrence,et al.  Bayesian Gaussian Process Latent Variable Model , 2010, AISTATS.

[11]  Malte Kuß,et al.  Gaussian process models for robust regression, classification, and reinforcement learning , 2006 .

[12]  Jus Kocijan,et al.  Evolving Gaussian process models for prediction of ozone concentration in the air , 2013, Simul. Model. Pract. Theory.

[13]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[14]  Biao Huang,et al.  System Identification , 2000, Control Theory for Physicists.

[15]  Guilherme De A. Barreto,et al.  An Empirical Evaluation of Robust Gaussian Process Models for System Identification , 2015, IDEAL.

[16]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[17]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[18]  Carl E. Rasmussen,et al.  Derivative Observations in Gaussian Process Models of Dynamic Systems , 2002, NIPS.

[19]  Jus Kocijan,et al.  Dynamical systems identification using Gaussian process models with incorporated local models , 2011, Eng. Appl. Artif. Intell..

[20]  T. Johansen,et al.  On transient dynamics, off-equilibrium behaviour and identification in blended multiple model structures , 1999, 1999 European Control Conference (ECC).

[21]  V. Peterka.  BAYESIAN APPROACH TO SYSTEM IDENTIFICATION , 1981 .

[22]  Wolfram Burgard,et al.  Learning Non-stationary System Dynamics Online Using Gaussian Processes , 2010, DAGM-Symposium.

[23]  Carl E. Rasmussen,et al.  Variational Gaussian Process State-Space Models , 2014, NIPS.

[24]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.