On Monte Carlo methods for estimating the fisher information matrix in difficult problems

The Fisher information matrix summarizes the amount of information in a set of data relative to the quantities of interest and forms the basis for the Cramér-Rao (lower) bound on the uncertainty in an estimate. There are many applications of the information matrix in modeling, systems analysis, and estimation. This paper presents a resampling-based method for computing the information matrix together with some new theory related to efficient implementation. We show how certain properties associated with the likelihood function and the error in the estimates of the Hessian matrix can be exploited to improve the accuracy of the Monte Carlo-based estimate of the information matrix.

[1]  L. J. Levy.  Generic Maximum Likelihood Identification Algorithms for Linear State Space Models , 1995 .

[2]  E. Weinstein,et al.  A new method for evaluating the log-likelihood gradient, the Hessian, and the Fisher information matrix for linear dynamic systems , 1989, IEEE Trans. Inf. Theory.

[3]  B. Efron,et al.  Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information , 1978 .

[4]  H. Piaggio.  Mathematical Analysis , 1955, Nature.

[5]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[6]  J. Spall.  Monte Carlo Computation of the Fisher Information Matrix in Nonstandard Settings , 2005 .

[7]  J.C. Spall,et al.  Convergence Analysis for Feedback-and Weighting-Based Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[8]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[9]  Tim Hesterberg,et al.  Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control , 2004, Technometrics.

[10]  Manchester Grand Hyatt Hotel.  Convergence Analysis for Feedback- and Weighting-Based Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm , 2006 .

[11]  James C. Spall,et al.  Adaptive stochastic approximation by the simultaneous perturbation method , 2000, IEEE Trans. Autom. Control..

[12]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[13]  P. Bickel,et al.  Mathematical Statistics: Basic Ideas and Selected Topics , 1977 .

[14]  R. F.,et al.  Mathematical Statistics , 1944, Nature.

[15]  Roger G. Ghanem,et al.  An efficient calculation of Fisher information matrix: Monte Carlo approach using prior information , 2007, 2007 46th IEEE Conference on Decision and Control.