Hierarchies of Generalized Kolmogorov Complexities and Nonenumerable Universal Measures Computable in the Limit

The traditional theory of Kolmogorov complexity and algorithmic probability focuses on monotone Turing machines with one-way write-only output tape. This naturally leads to the universal enumerable Solomonoff-Levin measure. Here we introduce more general, nonenumerable but cumulatively enumerable measures (CEMs) derived from Turing machines with lexicographically nondecreasing output and random input, and even more general approximable measures and distributions computable in the limit. We obtain a natural hierarchy of generalizations of algorithmic probability and Kolmogorov complexity, suggesting that the "true" information content of some (possibly infinite) bitstring x is the size of the shortest nonhalting program that converges to x and nothing but x on a Turing machine that can edit its previous outputs. Among other things we show that there are objects computable in the limit yet more random than Chaitin's "number of wisdom" Omega, that any approximable measure of x is small for any x lacking a sh...

[1]  K. Gödel Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I , 1931 .

[2]  A. Turing On computable numbers, with an application to the Entscheidungsproblem , 1937, Proc. London Math. Soc..

[3]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[4]  David A. Huffman,et al.  A method for the construction of minimum-redundancy codes , 1952, Proceedings of the IRE.

[5]  D. Huffman A Method for the Construction of Minimum-Redundancy Codes , 1952 .

[6]  A. Mostowski On computable sequences , 1957 .

[7]  A. Grzegorczyk On the definitions of computable real continuous functions , 1957 .

[8]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[9]  E. Mark Gold,et al.  Limiting recursion , 1965, Journal of Symbolic Logic.

[10]  Hilary Putnam,et al.  Trial and error predicates and the solution to a problem of Mostowski , 1965, Journal of Symbolic Logic.

[11]  Per Martin-Löf,et al.  The Definition of Random Sequences , 1966, Inf. Control..

[12]  A. Kolmogorov Three approaches to the quantitative definition of information , 1968 .

[13]  Gregory J. Chaitin,et al.  On the Length of Programs for Computing Finite Binary Sequences: statistical considerations , 1969, JACM.

[14]  L. Levin,et al.  THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS , 1970 .

[15]  Claus-Peter Schnorr,et al.  Process complexity and effective random tests , 1973 .

[16]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[17]  Péter Gács On the relation between descriptional complexity and algorithmic probability , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[18]  Stuart A. Kurtz On the Random Oracle Hypothesis , 1983, Inf. Control..

[19]  H. Cantor Ueber eine Eigenschaft des Inbegriffs aller reellen algebraischen Zahlen. , 1984 .

[20]  M. Beeson Foundations of Constructive Mathematics , 1985 .

[21]  Gregory. J. Chaitin,et al.  Algorithmic information theory , 1987, Cambridge tracts in theoretical computer science.

[22]  David Haussler,et al.  Occam's Razor , 1987, Inf. Process. Lett..

[23]  Shirley Dex,et al.  JR 旅客販売総合システム(マルス)における運用及び管理について , 1991 .

[24]  Antonín Kučera,et al.  On Relative Randomness , 1993, Ann. Pure Appl. Log..

[25]  Günter Hotz,et al.  Analytic Machines , 1999, Theor. Comput. Sci..

[26]  C. H. Bennett,et al.  Quantum Information and Computation , 1995 .

[27]  J. Urgen Schmidhuber A Computer Scientist's View of Life, the Universe, and Everything , 1997 .

[28]  Vladimir V. V'yugin,et al.  Non-Stochastic Infinite and Finite Sequences , 1998, Theor. Comput. Sci..

[29]  Robert M. Solovay A Version of Ω for which ZFC Cannot Predict a Single Bit , 2000, Finite Versus Infinite.

[30]  Jürgen Schmidhuber,et al.  Algorithmic Theories of Everything , 2000, ArXiv.

[31]  Marcus Hutter New Error Bounds for Solomonoff Prediction , 2001, J. Comput. Syst. Sci..

[32]  Marcus Hutter General Loss Bounds for Universal Sequence Prediction , 2001, ICML.

[33]  Antonín Kucera,et al.  Randomness and Recursive Enumerability , 2001, SIAM J. Comput..

[34]  Cristian S. Calude,et al.  Recursively enumerable reals and Chaitin Ω numbers , 2001, Theoretical Computer Science.

[35]  Cristian S. Calude Chaitin Omega numbers, Solovay machines, and Gödel incompleteness , 2002, Theor. Comput. Sci..