Stationary and Integrated Autoregressive Neural Network Processes

We consider autoregressive neural network (AR-NN) processes driven by additive noise and demonstrate that the characteristic roots of the shortcutsthe standard conditions from linear time-series analysisdetermine the stochastic behavior of the overall AR-NN process. If all the characteristic roots are outside the unit circle, then the process is ergodic and stationary. If at least one characteristic root lies inside the unit circle, then the process is transient. AR-NN processes with characteristic roots lying on the unit circle exhibit either ergodic, random walk, or transient behavior. We also analyze the class of integrated AR-NN (ARI-NN) processes and show that a standardized ARI-NN process converges to a Wiener process. Finally, least-squares estimation (training) of the stationary models and testing for nonstationarity is discussed. The estimators are shown to be consistent, and expressions on the limiting distributions are given.

[1]  H. White,et al.  Nonlinear Regression with Dependent Observations , 1984 .

[2]  Richard Bellman,et al.  Introduction to matrix analysis (2nd ed.) , 1997 .

[3]  P. Phillips.  Time series regression with a unit root , 1987 .

[4]  Clive W. J. Granger,et al.  Modelling nonlinear relationships between extended-memory variables , 1995 .

[5]  Clive W. J. Granger,et al.  Long memory series with attractors , 2001 .

[6]  R. Tweedie.  Criteria for classifying general Markov chains , 1976, Advances in Applied Probability.

[7]  Norman R. Swanson,et al.  Further developments in the study of cointegrated variables , 2010 .

[8]  P. Perron,et al.  Trends and random walks in macroeconomic time series : Further evidence from a new approach , 1988 .

[9]  Serena Ng,et al.  Useful Modifications to some Unit Root Tests with Dependent Errors and their Local Asymptotic Properties , 1996 .

[10]  Paul D. Feigin,et al.  RANDOM COEFFICIENT AUTOREGRESSIVE PROCESSES:A MARKOV CHAIN ANALYSIS OF STATIONARITY AND FINITENESS OF MOMENTS , 1985 .

[11]  Héctor J. Sussmann,et al.  Uniqueness of the weights for minimal feedforward nets with a given input-output map , 1992, Neural Networks.

[12]  H. White.  Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models , 1989 .

[13]  Benedikt M. Pötscher,et al.  A class of partially adaptive one-step m-estimators for the non-linear regression model with dependent observations , 1986 .

[14]  Paul C. Kainen,et al.  Functionally Equivalent Feedforward Neural Networks , 1994, Neural Computation.

[15]  Kurt Hornik,et al.  Stationarity and Stability of Autoregressive Neural Network Processes , 1998, NIPS.

[16]  Adrian Pagan,et al.  Estimation, Inference and Specification Analysis. , 1996 .

[17]  Richard A. Davis,et al.  Time Series: Theory and Methods (2nd ed.). , 1992 .

[18]  J. T. Hwang,et al.  Prediction Intervals for Artificial Neural Networks , 1997 .

[19]  H. Tong.  Non-linear time series. A dynamical system approach , 1990 .

[20]  Marcelo C. Medeiros,et al.  Modeling exchange rates: smooth transitions, neural networks, and linear models , 2001, IEEE Trans. Neural Networks.

[21]  Richard A. Davis,et al.  Time Series: Theory and Methods (2Nd Edn) , 1993 .

[22]  Tao Wang,et al.  Asymptotic Stationarity of Discrete-time Stochastic Neural Networks , 1996, Neural Networks.

[23]  Chris Chatfield,et al.  Introduction to Statistical Time Series. , 1976 .

[24]  D. Tjøstheim.  Non-linear time series and Markov chains , 1990, Advances in Applied Probability.

[25]  K. Athreya,et al.  Mixing properties of harris chains and autoregressive processes , 1986, Journal of Applied Probability.

[26]  Sastry G. Pantula,et al.  MIXING PROPERTIES OF HARRIS CHAINS , 1986 .

[27]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[28]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[29]  Richard D. De Veaux,et al.  Estimating Prediction Intervals for Arti cial Neural Networks , 2022 .

[30]  Halbert White,et al.  Parametric Statistical Estimation with Artificial Neural Networks: A Condensed Discussion , 1994 .

[31]  Peter C. B. Phillips,et al.  Time Series Regression With a Unit Root and Infinite-Variance Errors , 1990, Econometric Theory.

[32]  P. Phillips.  Testing for a Unit Root in Time Series Regression , 1988 .

[33]  Norman R. Swanson,et al.  Testing for stationarity-ergodicity and for comovements between nonlinear discrete time Markov processes , 2000 .

[34]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[35]  H. Tong,et al.  On the use of the deterministic Lyapunov function for the ergodicity of stochastic difference equations , 1985, Advances in Applied Probability.