Information geometry

Information geometry has emerged from the study of the invariant structure in families of probability distributions. This invariance uniquely determines a second-order symmetric tensor g and third-order symmetric tensor T in a manifold of probability distributions. A pair of these tensors ( g, T ) defines a Riemannian metric and a pair of affine connections which together preserve the metric. Information geometry involves studying a Riemannian manifold having a pair of dual affine connections. Such a structure also arises from an asymmetric divergence function and affine differential geometry. A dually flat Riemannian manifold is particularly useful for various applications, because a generalized Pythagorean theorem and projection theorem hold. The Wasserstein distance gives another important geometry on probability distributions, which is non-invariant but responsible for the metric properties of a sample space. I attempt to construct information geometry of the entropy-regularized Wasserstein distance.

[1]  T. Morimoto.  Markov Processes and the H -Theorem , 1963 .

[2]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[3]  S. Eguchi.  Second Order Efficiency of Minimum Contrast Estimators in a Curved Exponential Family , 1983 .

[4]  T. Matumoto.  Any statistical manifold has a contrast function---on the $C\sp 3$-functions taking the minimum at the diagonal of the product manifold , 1993 .

[5]  Giovanni Pistone,et al.  An Infinite-Dimensional Geometric Structure on the Space of all the Probability Measures Equivalent to a Given One , 1995 .

[6]  Shun-ichi Amari,et al.  Blind source separation-semiparametric statistical approach , 1997, IEEE Trans. Signal Process..

[7]  S. Amari,et al.  Information geometry of estimating functions in semi-parametric statistical models , 1997 .

[8]  H. Matsuzoe.  On realization of conformally-projectively flat statistical manifolds and the divergences , 1998 .

[9]  H. Matsuzoe.  Geometry of contrast functions and conformal geometry , 1999 .

[10]  Shun-ichi Amari,et al.  Estimating Functions of Independent Component Analysis for Temporally Correlated Signals , 2000, Neural Computation.

[11]  Masato Okada,et al.  Estimating Spiking Irregularities Under Changing Environments , 2006, Neural Computation.

[12]  Hông Vân Lê.  Statistical manifolds are statistical models , 2006 .

[13]  Giovanni Pistone,et al.  Exponential statistical manifold , 2007 .

[14]  Hiroshi Matsuzoe,et al.  Statistical manifolds and affine differential geometry , 2010 .

[15]  Shun-ichi Amari,et al.  Geometry of deformed exponential families: Invariant, dually-flat and conformal geometries , 2012 .

[16]  Martin Bauer,et al.  Uniqueness of the Fisher–Rao metric on the space of smooth densities , 2014, 1411.5577.

[17]  Gabriel Peyré,et al.  A Smoothed Dual Approach for Variational Wasserstein Problems , 2015, SIAM J. Imaging Sci..

[18]  Shun-ichi Amari,et al.  Information geometry connecting Wasserstein distance and Kullback–Leibler divergence via the entropy-relaxed transportation problem , 2017, Information Geometry.

[19]  James G. Dowty,et al.  Chentsov’s theorem for exponential families , 2017, Information Geometry.

[20]  Ting-Kam Leonard Wong.  Logarithmic divergences from optimal transport and Rényi geometry , 2017, ArXiv.

[21]  Shun-ichi Amari,et al.  Information Geometry for Regularized Optimal Transport and Barycenters of Patterns , 2019, Neural Computation.