Universal Algorithmic Intelligence: A Mathematical Top→Down Approach

Sequential decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental prior probability distribution is known. Solomonoff’s theory of universal induction formally solves the problem of sequence prediction for unknown prior distribution. We combine both ideas and get a parameter-free theory of universal Artificial Intelligence. We give strong arguments that the resulting AIXI model is the most intelligent unbiased agent possible. We outline how the AIXI model can formally solve a number of problem classes, including sequence prediction, strategic games, function minimization, reinforcement and supervised learning. The major drawback of the AIXI model is that it is un-computable. To overcome this problem, we construct a modified algorithm AIXItl that is still effectively more intelligent than any other time t and length l bounded agent. The computation time of AIXItl is of the order t·2l. The discussion includes formal definitions of intelligence order relations, the horizon problem and relations of the AIXI theory to other AI approaches.

[1]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[2]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[3]  J. Lucas.  Minds, Machines and Gödel , 1961, Philosophy.

[4]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[5]  D. Michie.  GAME-PLAYING AND GAME-LEARNING AUTOMATA , 1966 .

[6]  Gregory J. Chaitin,et al.  On the Length of Programs for Computing Finite Binary Sequences , 1966, JACM.

[7]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[8]  Robert P. Daley.  Minimal-Program Complexity of Sequences with Restricted Resources , 1973, Inf. Control..

[9]  G. Chaitin.  A Theory of Program Size Formally Identical to Information Theory , 1975, JACM.

[10]  Robert P. Daley.  On the Inference of Optimal Descriptions , 1977, Theor. Comput. Sci..

[11]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[12]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[13]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[14]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[15]  A. P. Dawid,et al.  Present position and potential developments: some personal views , 1984 .

[16]  Peter C. Cheeseman,et al.  In Defense of Probability , 1985, IJCAI.

[17]  Ray J. Solomonoff,et al.  The Application of Algorithmic Probability to Problems in Artificial Intelligence , 1985, UAI.

[18]  Ker-I Ko,et al.  On the Notion of Infinite Pseudorandom Sequences , 1986, Theor. Comput. Sci..

[19]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[20]  Peter C. Cheeseman,et al.  An inquiry into computer understanding , 1988, Comput. Intell..

[21]  H. Stowell.  The emperor's new mind R. Penrose, Oxford University Press, New York (1989) 466 pp. $24.95 , 1990, Neuroscience.

[22]  R. T. Cox.  Probability, frequency and reasonable expectation , 1990 .

[23]  Ming Li,et al.  Learning Simple Concept Under Simple Distributions , 1991, SIAM J. Comput..

[24]  Ming Li,et al.  Philosophical Issues in Kolmogorov Complexity , 1992, ICALP.

[25]  Ming Li,et al.  Inductive Reasoning and Kolmogorov Complexity , 1992, J. Comput. Syst. Sci..

[26]  Neri Merhav,et al.  Universal prediction of individual sequences , 1992, IEEE Trans. Inf. Theory.

[27]  Vladimir Vovk,et al.  Universal Forecasting Algorithms , 1992, Inf. Comput..

[28]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[29]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[30]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[31]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[32]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[33]  Melvin Fitting,et al.  First-Order Logic and Automated Theorem Proving , 1990, Graduate Texts in Computer Science.

[34]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[35]  Ivanoe De Falco,et al.  Genetic Programming Estimates of Kolmogorov Complexity , 1997, ICGA.

[36]  Jürgen Schmidhuber,et al.  Discovering Neural Nets with Low Kolmogorov Complexity and High Generalization Capability , 1997, Neural Networks.

[37]  Ray J. Solomonoff,et al.  The Discovery of Algorithmic Probability , 1997, J. Comput. Syst. Sci..

[38]  William G. Faris.  Shadows of the Mind: A Search for the Missing Science of Consciousness , 1997 .

[39]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[40]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[41]  Vladimir Vovk,et al.  Universal portfolio selection , 1998, COLT' 98.

[42]  Ray J. Solomonoff,et al.  Two Kinds of Probabilistic Induction , 1999, Comput. J..

[43]  Martin Schmidt.  Time-Bounded Kolmogorov Complexity May Help in Search for Extra Terrestrial Intelligence (SETI) , 1999, Bull. EATCS.

[44]  Marcus Hutter,et al.  A Theory of Universal Artificial Intelligence based on Algorithmic Complexity , 2000, ArXiv.

[45]  Jürgen Schmidhuber,et al.  Gradient-based Reinforcement Planning in Policy-Search Methods , 2001, ArXiv.

[46]  Jeffrey D. Ullman,et al.  Introduction to automata theory, languages, and computation, 2nd edition , 2001, SIGA.

[47]  Marcus Hutter.  New Error Bounds for Solomonoff Prediction , 2001, J. Comput. Syst. Sci..

[48]  Jürgen Schmidhuber,et al.  Market-Based Reinforcement Learning in Partially Observable Worlds , 2001, ICANN.

[49]  Marcus Hutter.  General Loss Bounds for Universal Sequence Prediction , 2001, ICML.

[50]  Marcus Hutter.  Convergence and Error Bounds for Universal Prediction of Nonbinary Sequences , 2001, ECML.

[51]  Marcus Hutter,et al.  Towards a Universal Theory of Artificial Intelligence Based on Algorithmic Probability and Sequential Decisions , 2000, ECML.

[52]  Marcus Hutter.  Universal sequential decisions in unknown environments , 2001 .

[53]  Ofi rNw8x'pyzm,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002 .

[54]  Jürgen Schmidhuber,et al.  Bias-Optimal Incremental Problem Solving , 2002, NIPS.

[55]  Marcus Hutter.  The Fastest and Shortest Algorithm for all Well-Defined Problems , 2002, Int. J. Found. Comput. Sci..

[56]  Marcus Hutter,et al.  Self-Optimizing and Pareto-Optimal Policies in General Environments based on Bayes-Mixtures , 2002, COLT.

[57]  Marcus Hutter.  Optimality of universal Bayesian prediction for general loss and alphabet , 2003 .

[58]  Marcus Hutter,et al.  On the Existence and Convergence of Computable Universal Priors , 2003, ALT.

[59]  Marcus Hutter.  Convergence and Loss Bounds for Bayesian Sequence Prediction , 2003, IEEE Trans. Inf. Theory.

[60]  Jürgen Schmidhuber,et al.  Optimal Ordered Problem Solver , 2002, Machine Learning.

[61]  Jürgen Schmidhuber,et al.  Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement , 1997, Machine Learning.

[62]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[63]  Marcus Hutter.  Simulation Algorithms for Computational Systems Biology , 2017, Texts in Theoretical Computer Science. An EATCS Series.

[64]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[65]  S. Legg.  Machine super intelligence , 2008 .