Probabilistic Bounds in Tracking a Discrete-Time Varying Process

This paper investigates the minimization problem, under a practical setting where the scalar-valued loss function $L_{k}$ is intrinsically time-varying, and therefore the corresponding minimizer sequence is also time-varying. We focus on the tracking capability of the recursive estimates generated by the constant-gain stochastic gradient algorithm. It is shown that the trajectory of a limiting nonautonomous ordinary differential equation can be associated with the iterates generated by the stochastic gradient descent algorithm with constant gain. The main tool in establishing the connection is the formula for variation of parameters. The established probabilistic bound is applicable in tracking a discrete-time varying process, e.g., a jump process. A synthetic example illustrates the accountability of the trajectory characterization depends on the magnitude of the noise, the drift magnitude, and the constant gain.

[1]  Fred Brauer,et al.  Perturbations of nonlinear systems of differential equations, IV , 1966 .

[2]  R. Bellman Stability theory of differential equations , 1953 .

[3]  G. Teschl Ordinary Differential Equations and Dynamical Systems , 2012 .

[4]  Omar Besbes,et al.  Non-Stationary Stochastic Optimization , 2013, Oper. Res..

[5]  Mohammad A. S. Masoum,et al.  Real-Time Coordination of Plug-In Electric Vehicle Charging in Smart Grids to Minimize Power Losses and Improve Voltage Profile , 2011, IEEE Transactions on Smart Grid.

[6]  J. Lambert Numerical Methods for Ordinary Differential Equations , 1991 .

[7]  Lei Guo,et al.  Stability of Recursive Stochastic Tracking Algorithms , 1994 .

[8]  T. Sideris Ordinary Differential Equations and Dynamical Systems , 2013 .

[9]  H. Kushner,et al.  Asymptotic Properties of Stochastic Approximations with Constant Coefficients. , 1981 .

[10]  James C. Spall,et al.  Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control (Spall, J.C. , 2007 .

[11]  N. Levinson,et al.  Perturbations of Nonlinear Systems of Differential Equations , 2003 .

[12]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[13]  J. Sacks Asymptotic Distribution of Stochastic Approximation Procedures , 1958 .

[14]  A. Dvoretzky On Stochastic Approximation , 1956 .

[15]  Lennart Ljung,et al.  Performance analysis of general tracking algorithms , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[16]  Ming Ye,et al.  Rate of convergence analysis of simultaneous perturbation stochastic approximation algorithm for time-varying loss function , 2014, 2014 American Control Conference.

[17]  Natalia O. Amelina,et al.  Simultaneous Perturbation Stochastic Approximation for Tracking Under Unknown but Bounded Disturbances , 2015, IEEE Transactions on Automatic Control.

[18]  Joel A. Tropp,et al.  An Introduction to Matrix Concentration Inequalities , 2015, Found. Trends Mach. Learn..

[19]  Jingyi Zhu,et al.  Tracking capability of stochastic gradient algorithm with constant gain , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[20]  H. Robbins,et al.  A Convergence Theorem for Non Negative Almost Supermartingales and Some Applications , 1985 .

[21]  George J. Pappas,et al.  Stochastic Source Seeking by Mobile Robots , 2012, IEEE Transactions on Automatic Control.

[22]  Yisheng Zhong,et al.  Time-Varying Formation Control for Unmanned Aerial Vehicles: Theories and Applications , 2015, IEEE Transactions on Control Systems Technology.

[23]  I-Jeng Wang,et al.  Stochastic optimisation with inequality constraints using simultaneous perturbations and penalty functions , 2008, Int. J. Control.

[24]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[25]  J. Blum Approximation Methods which Converge with Probability one , 1954 .

[26]  J. Spall,et al.  Model-free control of nonlinear stochastic systems with discrete-time measurements , 1998, IEEE Trans. Autom. Control..