Least-informative Bayesian prior distributions for finite samples based on information theory

A procedure is presented, based on Shannon information theory, for producing least-informative prior distributions for Bayesian estimation and identification. This approach relies on constructing an optimal mixture distribution and applies in small sample sizes (unlike certain approaches based on asymptotic theory). The procedure is illustrated in a small-scale numerical study and is contrasted with an approach based on maximum entropy. >

[1]  H. Sorenson,et al.  Nonlinear Bayesian estimation using Gaussian sum approximations , 1972 .

[2]  T. Ferguson.  BAYESIAN DENSITY ESTIMATION BY MIXTURES OF NORMAL DISTRIBUTIONS , 1983 .

[3]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[4]  A. Zellner.  An Introduction to Bayesian Inference in Econometrics , 1971 .

[5]  An extension of bayesian measure of information to regression , 1986 .

[6]  Carlos M. Urzúa.  A Class of Maximum-Entropy Multivariate Distributions , 1988 .

[7]  J. Spall.  Bayesian error isolation for models of large-scale systems , 1988 .

[8]  E. Stein,et al.  Introduction to Fourier Analysis on Euclidean Spaces. , 1971 .

[9]  H. Sorenson,et al.  Recursive bayesian estimation using gaussian sums , 1971 .

[10]  H. Akaike.  On minimum information prior distributions , 1983 .

[11]  Rangasami L. Kashyap,et al.  Prior probability and uncertainty , 1971, IEEE Trans. Inf. Theory.

[12]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[13]  Václav Peterka,et al.  Bayesian system identification , 1979, Autom..

[14]  J. Bernardo.  Reference Posterior Distributions for Bayesian Inference , 1979 .

[15]  W. Fleming.  Functions of Several Variables , 1965 .

[16]  Donald Poskitt,et al.  On the posterior odds of time series models , 1983 .

[17]  James Ting-Ho Lo,et al.  Finite-dimensional sensor orbits and optimal nonlinear filtering , 1972, IEEE Trans. Inf. Theory.

[18]  W. Zangwill.  Non-Linear Programming Via Penalty Functions , 1967 .

[19]  James Spall,et al.  Noninformative Bayesian priors for large samples based on Shannon information theory , 1987, 26th IEEE Conference on Decision and Control.