Five Answers on Randomness

Five brief and highly biased answers to five questions on rand omness posed by Hector Zenil: Why were you initially drawn to the study of c omputation and randomness? What have we learned? What don’t we know (yet)? W hat are the most important open problems? What are the prospects for pro g ess? 1 Why were you initially drawn to the study of computation and randomness? The topic is so all-encompassing and sexy. It helps to formal ize the notions of Occam’s razor and inductive inference [9, 36, 10, 37, 12, 19], which a re at the heart of all inductive sciences. It is relevant not only for Artificial Inte lligence [7, 30, 28, 33] and computer science but also for physics and philosophy [14, 18 , 20]. Every scientist and philosopher should know about it. Even artists should, as th ere are complexity-based explanations of essential aspects of aesthetics and art [16 , 15, 32]. 2 What have we learned? In the new millennium the study of computation and randomnes s, pioneered in the 1930s [5, 39, 8, 36, 9, 12], has brought substantial progress in the field of theoretically optimal algorithms for prediction, search, inductiv e nference based on Occam’s razor, problem solving, decision making, and reinforcemen t l arning in environments of a very general type [7, 23, 25, 26, 21, 30, 28, 33]. It led to a symptotically optimal universal program search techniques [6, 22, 33] for extreme ly broad classes of problems. Some of the results even provoke nontraditional predi ctions regarding the future of the universe [14, 18, 20, 29] based on Zuse’s thesis [40, 41 ] of computable physics [14, 18, 19, 27]. The field also is relevant for art, and for cla rifying what science and art have in common [16, 15, 17, 24, 31, 32].

[1]  Marcus Hutter Simulation Algorithms for Computational Systems Biology , 2017, Texts in Theoretical Computer Science. An EATCS Series.

[2]  P. Vitányi,et al.  An Introduction to Kolmogorov Complexity and Its Applications, Third Edition , 1997, Texts in Computer Science.

[3]  Jürgen Schmidhuber,et al.  Simple Algorithmic Principles of Discovery, Subjective Beauty, Selective Attention, Curiosity & Creativity , 2007, Discovery Science.

[4]  Jürgen Schmidhuber,et al.  Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts , 2006, Connect. Sci..

[5]  J. Schmidhuber Facial beauty and fractal geometry , 1998 .

[6]  Jan Poland A coding theorem for Enumerable Output Machines , 2004, Inf. Process. Lett..

[7]  Jürgen Schmidhuber,et al.  Gödel Machines: Fully Self-referential Optimal Universal Self-improvers , 2007, Artificial General Intelligence.

[8]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[9]  Konrad Zuse,et al.  Rechnender Raum , 1991, Physik und Informatik.

[10]  Robert M. Solovay A Version of Ω for which ZFC Cannot Predict a Single Bit , 2000, Finite Versus Infinite.

[11]  Jürgen Schmidhuber,et al.  Optimal Ordered Problem Solver , 2002, Machine Learning.

[12]  Jürgen Schmidhuber,et al.  Simple algorithmic theory of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes (特集 高次機能の学習と創発--脳・ロボット・人間研究における新たな展開) , 2009 .

[13]  Péter Gács,et al.  On the relation between descriptional complexity and algorithmic probability , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[14]  Jürgen Schmidhuber,et al.  Low-Complexity Art , 2017 .

[15]  Jürgen Schmidhuber,et al.  New Millennium AI and the Convergence of History: Update of 2012 , 2012 .

[16]  Jürgen Schmidhuber,et al.  Ultimate Cognition à la Gödel , 2009, Cognitive Computation.

[17]  A. Kolmogorov Three approaches to the quantitative definition of information , 1968 .

[18]  Jürgen Schmidhuber,et al.  A Computer Scientist's View of Life, the Universe, and Everything , 1999, Foundations of Computer Science: Potential - Theory - Cognition.

[19]  Jürgen Schmidhuber,et al.  The New AI: General & Sound & Relevant for Physics , 2003, Artificial General Intelligence.

[20]  Ofi rNw8x'pyzm,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002 .

[21]  Jürgen Schmidhuber,et al.  Hierarchies of Generalized Kolmogorov Complexities and Nonenumerable Universal Measures Computable in the Limit , 2002, Int. J. Found. Comput. Sci..

[22]  Antonín Kucera,et al.  Randomness and Recursive Enumerability , 2001, SIAM J. Comput..

[23]  Jürgen Schmidhuber 2006: Celebrating 75 Years of AI - History and Outlook: The Next 25 Years , 2006, 50 Years of Artificial Intelligence.

[24]  Jürgen Schmidhuber,et al.  Completely Self-referential Optimal Reinforcement Learners , 2005, ICANN.

[25]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[26]  Marcus Hutter,et al.  Algorithmic Information Theory , 1977, IBM J. Res. Dev..

[27]  A. Turing On computable numbers, with an application to the Entscheidungsproblem , 1937, Proc. London Math. Soc..

[28]  Marcus Hutter The Fastest and Shortest Algorithm for all Well-Defined Problems , 2002, Int. J. Found. Comput. Sci..

[29]  Helly Grundbegriffe der Wahrscheinlichkeitsrechnung , 1936 .

[30]  Péter Gács On the relation between descriptional complexity and algorithmic probability , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[31]  Jürgen Schmidhuber,et al.  Algorithmic Theories of Everything , 2000, ArXiv.

[32]  K. Gödel Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I , 1931 .

[33]  K. Gödel Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I , 1931 .