Cyclic seesaw optimization with applications to state-space model identification

In cyclic (or alternating) method, the full parameter vector is divided into two or more subvectors and the process proceeds by sequentially optimizing each of the subvectors while holding the remaining parameters at their most recent values. One example of the advantage of the scheme is the preservation of large investments in software while allowing for an extension of capability to include new parameters for estimation. A specific case involves cross-sectional data represented in state-space form, where there is interest in estimating the mean vector and covariance matrix of the initial state vector as well as parameters associated with the dynamics of the underlying differential equations (e.g., power spectral density parameters). This paper shows that under reasonable conditions the cyclic scheme will converge to the joint estimate for the full vector of unknown parameters. Convergence conditions here differ from others in the literature

[1]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[2]  T. Moon,et al.  Mathematical Methods and Algorithms for Signal Processing , 1999 .

[3]  James C. Spall,et al.  Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control (Spall, J.C. , 2007 .

[4]  James C. Spall,et al.  Efficient Monte Carlo computation of Fisher information matrix using prior information , 2007, Comput. Stat. Data Anal..

[5]  James C. Bezdek,et al.  Convergence of Alternating Optimization , 2003, Neural Parallel Sci. Comput..

[6]  Wanli Min,et al.  A Statistical Approach to Thermal Management of Data Centers Under Steady State and System Perturbations , 2010, Journal of the American Statistical Association.

[7]  Tim Hesterberg,et al.  Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control , 2004, Technometrics.

[8]  N. L. Johnson,et al.  Linear Statistical Inference and Its Applications , 1966 .

[9]  L. J. Levy Generic Maximum Likelihood Identification Algorithms for Linear State Space Models , 1995 .

[10]  W. Achtziger On simultaneous optimization of truss geometry and topology , 2007 .

[11]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[12]  J. Spall Monte Carlo Computation of the Fisher Information Matrix in Nonstandard Settings , 2005 .

[13]  Katta G. Murty,et al.  Nonlinear Programming Theory and Algorithms , 2007, Technometrics.

[14]  H. Anton,et al.  FUNCTIONS OF SEVERAL VARIABLES , 1982 .

[15]  E. Weinstein,et al.  A new method for evaluating the log-likelihood gradient, the Hessian, and the Fisher information matrix for linear dynamic systems , 1989, IEEE Trans. Inf. Theory.

[16]  D.G. Dudley,et al.  Dynamic system identification experiment design and data analysis , 1979, Proceedings of the IEEE.

[17]  J.C. Spall th International Conference on Information Fusion ( FUSION ) Seesaw Method for Combining Parameter Estimates , 2006 .

[18]  James C. Spall,et al.  Parameter identification for state-space models with nuisance parameters , 1990 .

[19]  R. Shumway,et al.  Estimation and tests of hypotheses for the initial mean and covariance in the kalman filter model , 1981 .

[20]  P. Caines,et al.  Linear system identification from nonstationary cross-sectional data , 1979 .

[21]  P. Caines,et al.  Linear system identification from non-stationary cross-sectional data , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[22]  James C. Spall,et al.  Feedback and Weighting Mechanisms for Improving Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm , 2007, IEEE Transactions on Automatic Control.

[23]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[24]  Brett Ninness,et al.  Robust maximum-likelihood estimation of multivariable dynamic systems , 2005, Autom..

[25]  H. Piaggio Mathematical Analysis , 1955, Nature.

[26]  P. Tseng Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[27]  Fang-Kuo Sun,et al.  A maximum likelihood algorithm for the mean and covariance of nonidentically distributed observations , 1982 .