On Choosing and Bounding Probability Metrics

When studying convergence of measures, an important issue is the choice of probability metric. We provide a summary and some new results concerning bounds among some important probability metrics/distances that are used by statisticians and probabilists. Knowledge of other metrics can provide a means of deriving bounds for another one in an applied problem. Considering other metrics can also provide alternate insights. We also give examples that show that rates of convergence can strongly depend on the metric chosen. Careful consideration is necessary when choosing a metric.

[1]  Ernst Hellinger,et al.  Die Orthogonalinvarianten quadratischer Formen von unendlichvielen Variabelen , 2022 .

[2]  Istituto italiano degli attuari Giornale dell'Istituto italiano degli attuari , 1930 .

[3]  AN Kolmogorov-Smirnov,et al.  Sulla determinazione empírica di uma legge di distribuzione , 1933 .

[4]  S. Kakutani On Equivalence of Infinite Product Measures , 1948 .

[5]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[6]  Yu. V. Prokhorov Convergence of Random Processes and Limit Theorems in Probability Theory , 1956 .

[7]  J. Linnik An Information-Theoretic Proof of the Central Limit Theorem with Lindeberg Conditions , 1959 .

[8]  S. Kullback,et al.  A lower bound for discrimination information in terms of variation (Corresp.) , 1967, IEEE Trans. Inf. Theory.

[9]  R. Dudley Distances of Probability Measures and Random Variables , 1968 .

[10]  L. Cam,et al.  Théorie asymptotique de la décision statistique , 1969 .

[11]  S. Orey Lecture Notes on Limit Theorems for Markov Chain Transition Probabilities , 1971 .

[12]  John Wiley,et al.  A wiley-interscience publication , 1972 .

[13]  Lauwerens Kuipers,et al.  Uniform distribution of sequences , 1974 .

[14]  D. A. Edwards On the existence of probability measures with given marginals , 1978 .

[15]  R. Z. Khasʹminskiĭ,et al.  Statistical estimation : asymptotic theory , 1981 .

[16]  P. Diaconis,et al.  Updating Subjective Probability , 1982 .

[17]  A. Szulga On Minimal Metrics in the Space of Random Variables , 1983 .

[18]  E. Nummelin General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[19]  V. M. Zolotarev,et al.  Addendum: Probability Metrics , 1984 .

[20]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[21]  A. Barron ENTROPY AND THE CENTRAL LIMIT THEOREM , 1986 .

[22]  A. Shiryaev,et al.  Limit Theorems for Stochastic Processes , 1987 .

[23]  P. Diaconis,et al.  Strong uniform times and finite random walks , 1987 .

[24]  R. Graham,et al.  Random Walks Arising in Random Number Generation , 1987 .

[25]  P. Diaconis Group representations in probability and statistics , 1988 .

[26]  R. Reiss Approximate Distributions of Order Statistics , 1989 .

[27]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[28]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[29]  Jim Freeman Probability Metrics and the Stability of Stochastic Models , 1991 .

[30]  T. Lindvall Lectures on the Coupling Method , 1992 .

[31]  P. Diaconis,et al.  Comparison Techniques for Random Walk on Finite Groups , 1993 .

[32]  B. Lindsay Efficiency versus robustness : the case for minimum Hellinger distance and related methods , 1994 .

[33]  L. Tierney Markov Chains for Exploring Posterior Distributions , 1994 .

[34]  Jeffrey S. Rosenthal,et al.  Convergence Rates for Markov Chains , 1995, SIAM Rev..

[35]  Walter R. Gilks,et al.  Introduction to general state-space Markov chain theory , 1995 .

[36]  V. V. Petrov Limit Theorems of Probability Theory: Sequences of Independent Random Variables , 1995 .

[37]  P. Diaconis,et al.  LOGARITHMIC SOBOLEV INEQUALITIES FOR FINITE MARKOV CHAINS , 1996 .

[38]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[39]  Huaiyu Zhu On Information and Sufficiency , 1997 .

[40]  J. Hartigan The maximum likelihood prior , 1998 .

[41]  SU Francisedward CONVERGENCE OF RANDOM WALKS ON THE CIRCLE GENERATED BY AN IRRATIONAL ROTATION , 1998 .

[42]  Ludger Rüschendorf,et al.  Distributions with fixed marginals and related topics , 1999 .

[43]  V. Statulevičius,et al.  Limit Theorems of Probability Theory , 2000 .

[44]  Alison L. Gibbs,et al.  Convergence of Markov chain Monte Carlo algorithms with applications to image restoration , 2000 .

[45]  F. Su Discrepancy Convergence for the Drunkard's Walk on the Sphere , 2001, math/0102205.

[46]  Dudley,et al.  Real Analysis and Probability: Measurability: Borel Isomorphism and Analytic Sets , 2002 .

[47]  T. N. Sriram Asymptotics in Statistics–Some Basic Concepts , 2002 .

[48]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.