The new AI is general and mathematically rigorous

Most traditional artificial intelligence (AI) systems of the past decades are either very limited, or based on heuristics, or both. The new millennium, however, has brought substantial progress in the field of theoretically optimal and practically feasible algorithms for prediction, search, inductive inference based on Occam’s razor, problem solving, decision making, and reinforcement learning in environments of a very general type. Since inductive inference is at the heart of all inductive sciences, some of the results are relevant not only for AI and computer science but also for physics, provoking nontraditional predictions based on Zuse’s thesis of the computer-generated universe. We first briefly review the history of AI since Gödel’s 1931 paper, then discuss recent post-2000 approaches that are currently transforming general AI research into a formal science.

[1]  Jürgen Schmidhuber,et al.  New Millennium AI and the Convergence of History: Update of 2012 , 2012 .

[2]  M. Beeson.  Foundations of Constructive Mathematics , 1985 .

[3]  Martin V. Butz,et al.  Anticipatory Behavior in Adaptive Learning Systems , 2003, Lecture Notes in Computer Science.

[4]  Jürgen Schmidhuber,et al.  Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts , 2006, Connect. Sci..

[5]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[6]  J. Bell.  On the Problem of Hidden Variables in Quantum Mechanics , 1966 .

[7]  Jürgen Schmidhuber,et al.  Optimal Ordered Problem Solver , 2002, Machine Learning.

[8]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[9]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[10]  Jürgen Schmidhuber,et al.  Bias-Optimal Incremental Problem Solving , 2002, NIPS.

[11]  F. Cajori.  A history of mathematics , 1989 .

[12]  Nils J. Nilsson,et al.  Principles of Artificial Intelligence , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  E. Feigenbaum,et al.  Computers and Thought , 1963 .

[15]  C. Monroe,et al.  Experimental entanglement of four particles , 2000, Nature.

[16]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[17]  SolomonoffR..  Complexity-based induction systems , 2006 .

[18]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[19]  H. Cantor.  Ueber eine Eigenschaft des Inbegriffs aller reellen algebraischen Zahlen. , 1984 .

[20]  Jürgen Schmidhuber,et al.  A possibility for implementing curiosity and boredom in model-building neural controllers , 1991 .

[21]  C. Schmidhuber.  Strings from Logic , 2000, hep-th/0011065.

[22]  Jürgen Schmidhuber,et al.  Artificial Scientists & Artists Based on the Formal Theory of Creativity , 2010, AGI 2010.

[23]  Shigeyoshi Tsutsui,et al.  Advances in Evolutionary Computing , 2003 .

[24]  Jürgen Schmidhuber,et al.  Exploring the predictable , 2003 .

[25]  Jürgen Schmidhuber,et al.  Low-Complexity Art , 2017 .

[26]  Jürgen Schmidhuber,et al.  Algorithmic Theories of Everything , 2000, ArXiv.

[27]  Jürgen Schmidhuber,et al.  Gödel Machines: Fully Self-referential Optimal Universal Self-improvers , 2007, Artificial General Intelligence.

[28]  Jürgen Schmidhuber,et al.  Completely Self-referential Optimal Reinforcement Learners , 2005, ICANN.

[29]  Jürgen Schmidhuber.  2006: Celebrating 75 Years of AI - History and Outlook: The Next 25 Years , 2006, 50 Years of Artificial Intelligence.

[30]  Gregory J. Chaitin,et al.  Algorithmic Information Theory , 1987, IBM J. Res. Dev..

[31]  Luca Maria Gambardella,et al.  Ant Algorithms for Discrete Optimization , 1999, Artificial Life.

[32]  Marcus Hutter.  The Fastest and Shortest Algorithm for all Well-Defined Problems , 2002, Int. J. Found. Comput. Sci..

[33]  Jürgen Schmidhuber,et al.  The New AI: General & Sound & Relevant for Physics , 2003, Artificial General Intelligence.

[34]  L. Levin,et al.  THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS , 1970 .

[35]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[36]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[37]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[38]  Jürgen Schmidhuber,et al.  A Computer Scientist's View of Life, the Universe, and Everything , 1999, Foundations of Computer Science: Potential - Theory - Cognition.

[39]  Andrew P. Sage,et al.  Uncertainty in Artificial Intelligence , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[40]  Wilfried Brauer,et al.  Foundations of computer science : potential--theory--cognition , 1997 .

[41]  Reinhold Behringer,et al.  The seeing passenger car 'VaMoRs-P' , 1994, Proceedings of the Intelligent Vehicles '94 Symposium.

[42]  Sepp Hochreiter,et al.  Learning to Learn Using Gradient Descent , 2001, ICANN.

[43]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[44]  Ray J. Solomonoff,et al.  The Application of Algorithmic Probability to Problems in Artificial Intelligence , 1985, UAI.

[45]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[46]  K. Gödel.  Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I , 1931 .

[47]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[48]  John von Neumann,et al.  Theory Of Self Reproducing Automata , 1967 .

[49]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[50]  Michael Barr,et al.  The Emperor's New Mind , 1989 .

[51]  Jürgen Schmidhuber,et al.  Solving POMDPs with Levin Search and EIRA , 1996, ICML.

[52]  C. S. Wallace,et al.  An Information Measure for Classification , 1968, Comput. J..

[53]  L. E. J. Brouwer,et al.  Over de Grondslagen der Wiskunde , 2009 .

[54]  Jürgen Schmidhuber,et al.  Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement , 1997, Machine Learning.

[55]  J. Schmidhuber.  What''s interesting? , 1997 .

[56]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[57]  H. Everett.  "Relative State" Formulation of Quantum Mechanics , 1957 .

[58]  Jürgen Schmidhuber,et al.  An on-line algorithm for dynamic reinforcement learning and planning in reactive environments , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[59]  David P. DiVincenzo,et al.  Quantum information and computation , 2000, Nature.

[60]  Gregory J. Chaitin,et al.  A recent technical report , 1974, SIGA.

[61]  Jürgen Schmidhuber,et al.  Simple Algorithmic Principles of Discovery, Subjective Beauty, Selective Attention, Curiosity & Creativity , 2007, Discovery Science.

[62]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[63]  Leopold Löwenheim.  Über Möglichkeiten im Relativkalkül , 1915 .

[64]  A. Turing.  On computable numbers, with an application to the Entscheidungsproblem , 1937, Proc. London Math. Soc..

[65]  Jürgen Schmidhuber,et al.  Hierarchies of Generalized Kolmogorov Complexities and Nonenumerable Universal Measures Computable in the Limit , 2002, Int. J. Found. Comput. Sci..

[66]  Konrad Zuse,et al.  Rechnender Raum , 1991, Physik und Informatik.

[67]  K. Popper,et al.  The Logic of Scientific Discovery , 1960 .

[68]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[69]  J. Schmidhuber.  Don't forget randomness is still just a hypothesis , 2006, Nature.

[70]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[71]  Rodney A. Brooks,et al.  Intelligence Without Reason , 1991, IJCAI.

[72]  T. Erber,et al.  Randomness in quantum mechanics—nature's ultimate cryptogram? , 1985, Nature.

[73]  Jürgen Schmidhuber,et al.  Driven by Compression Progress , 2008, KES.

[74]  Fumiya Iida,et al.  50 Years of Artificial Intelligence, Essays Dedicated to the 50th Anniversary of Artificial Intelligence , 2007, 50 Years of Artificial Intelligence.

[75]  John E. Laird,et al.  The soar papers : research on integrated intelligence , 1993 .

[76]  R. Reiter,et al.  IJCAI-91: proceedings of the twelfth International Joint Conference on Artificial Intelligence : Darling Harbour, Sydney, Australia, 24-30 august 1991 , 1991 .

[77]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[78]  G. Hooft.  Quantum gravity as a dissipative deterministic system , 1999, gr-qc/9903084.

[79]  Jürgen Schmidhuber,et al.  Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes , 2008, ABiALS.

[80]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[81]  Rolf Pfeifer,et al.  Understanding intelligence , 2020, Inequality by Design.

[82]  T. Toffoli,et al.  Conservative logic , 2002, Collision-Based Computing.

[83]  Jürgen Schmidhuber,et al.  Discovering Neural Nets with Low Kolmogorov Complexity and High Generalization Capability , 1997, Neural Networks.

[84]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[85]  Allen Newell,et al.  GPS, a program that simulates human thought , 1995 .

[86]  Ray J. Solomonofi,et al.  A SYSTEM FOR INCREMENTAL LEARNING BASED ON ALGORITHMIC PROBABILITY , 1989 .

[87]  Jürgen Schmidhuber.  Discovering Solutions with Low Kolmogorov Complexity and High Generalization Capability , 1995, ICML.

[88]  Jürgen Schmidhuber,et al.  Simple algorithmic theory of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes (特集 高次機能の学習と創発--脳・ロボット・人間研究における新たな展開) , 2009 .

[89]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[90]  Hilary Putnam,et al.  Trial and error predicates and the solution to a problem of Mostowski , 1965, Journal of Symbolic Logic.

[91]  Gregory. J. Chaitin,et al.  Algorithmic information theory , 1987, Cambridge tracts in theoretical computer science.

[92]  A. Kolmogoroff.  Grundbegriffe der Wahrscheinlichkeitsrechnung , 1933 .

[93]  Paul E. Utgoff,et al.  Shift of bias for inductive concept learning , 1984 .

[94]  Neri Merhav,et al.  Universal Prediction , 1998, IEEE Trans. Inf. Theory.

[95]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[96]  Jürgen Schmidhuber,et al.  Ultimate Cognition à la Gödel , 2009, Cognitive Computation.

[97]  Dr. Marcus Hutter,et al.  Universal artificial intelligence , 2004 .

[98]  Jürgen Schmidhuber,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002, COLT.

[99]  Péter Gács,et al.  On the relation between descriptional complexity and algorithmic probability , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).