Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes

I argue that data becomes temporarily interesting by itself to some self-improving, but computationally limited, subjective observer once he learns to predict or compress the data in a better way, thus making it subjectively simpler and more beautiful. Curiosity is the desire to create or discover more non-random, non-arbitrary, regular data that is novel and surprising not in the traditional sense of Boltzmann and Shannon but in the sense that it allows for compression progress because its regularity was not yet known. This drive maximizes interestingness, the first derivative of subjective beauty or compressibility, that is, the steepness of the learning curve. It motivates exploring infants, pure mathematicians, composers, artists, dancers, comedians, yourself, and (since 1990) artificial systems.

[1]  J. Galton Recent Observations upon the Placentation of the Sloths , 1878, Nature.

[2]  F. Galton Composite Portraits, Made by Combining Those of Many Different Persons Into a Single Resultant Figure. , 1879 .

[3]  K. Gödel Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I , 1931 .

[4]  A. Turing On computable numbers, with an application to the Entscheidungsproblem , 1937, Proc. London Math. Soc..

[5]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[6]  David A. Huffman,et al.  A method for the construction of minimum-redundancy codes , 1952, Proceedings of the IRE.

[7]  D. Huffman A Method for the Construction of Minimum-Redundancy Codes , 1952 .

[8]  J. Piaget The child's construction of reality , 1954 .

[9]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[10]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[11]  W. Stolz Information Theory and Esthetic Perception. , 1967 .

[12]  R. Arnheim,et al.  Information Theory and Esthetic Perception , 1968 .

[13]  A. Kolmogorov Three approaches to the quantitative definition of information , 1968 .

[14]  C. S. Wallace,et al.  An Information Measure for Classification , 1968, Comput. J..

[15]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[16]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[17]  H. Franke,et al.  Ästhetik als Informationsverarbeitung , 1974 .

[18]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[19]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[20]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[21]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[22]  Sandy Lovie How the mind works , 1980, Nature.

[23]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[24]  C. S. Wallace,et al.  Estimation and Inference by Compact Coding , 1987 .

[25]  PAUL J. WERBOS,et al.  Generalization of backpropagation with application to a recurrent gas market model , 1988, Neural Networks.

[26]  H. B. Barlow,et al.  Finding Minimum Entropy Codes , 1989, Neural Computation.

[27]  C. Watkins Learning from delayed rewards , 1989 .

[28]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[29]  Jürgen Schmidhuber,et al.  Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem , 1990 .

[30]  Jürgen Schmidhuber,et al.  An on-line algorithm for dynamic reinforcement learning and planning in reactive environments , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[31]  Jürgen Schmidhuber,et al.  A possibility for implementing curiosity and boredom in model-building neural controllers , 1991 .

[32]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[33]  Jürgen Schmidhuber,et al.  Learning to Generate Artificial Fovea Trajectories for Target Detection , 1991, Int. J. Neural Syst..

[34]  Jenq-Neng Hwang,et al.  Query-based learning applied to partially trained multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[35]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[36]  Jürgen Schmidhuber,et al.  Learning Complex, Extended Sequences Using the Principle of History Compression , 1992, Neural Computation.

[37]  J. Urgen Schmidhuber,et al.  Learning Factorial Codes by Predictability Minimization , 1992 .

[38]  Jürgen Schmidhuber,et al.  A Fixed Size Storage O(n3) Time Complexity Learning Algorithm for Fully Recurrent Continually Running Networks , 1992, Neural Computation.

[39]  Steven Douglas Whitehead,et al.  Reinforcement learning for the adaptive control of perception and action , 1992 .

[40]  David A. Cohn,et al.  Neural Network Exploration Using Optimal Experiment Design , 1993, NIPS.

[41]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[42]  Xin Yao,et al.  A review of evolutionary artificial neural networks , 1993, Int. J. Intell. Syst..

[43]  Garrison W. Cottrell,et al.  Learning Mackey-Glass from 25 Examples, Plus or Minus 2 , 1993, NIPS.

[44]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[45]  Karl Sims,et al.  Evolving virtual creatures , 1994, SIGGRAPH.

[46]  Randall D. Beer,et al.  Sequential Behavior and Learning in Evolved Dynamical Neural Networks , 1994, Adapt. Behav..

[47]  Stefano Nolfi,et al.  How to Evolve Autonomous Robots: Different Approaches in Evolutionary Robotics , 1994 .

[48]  D. Perrett,et al.  Facial shape and judgements of female attractiveness , 1994, Nature.

[49]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[50]  S. Hochreiter,et al.  REINFORCEMENT DRIVEN INFORMATION ACQUISITION IN NONDETERMINISTIC ENVIRONMENTS , 1995 .

[51]  Barak A. Pearlmutter Gradient calculations for dynamic recurrent neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[52]  Roland Olsson,et al.  Inductive Functional Programming Using Incremental Program Transformation , 1995, Artif. Intell..

[53]  Stefano Nolfi,et al.  Evolving Mobile Robots in Simulated and Real Environments , 1995, Artificial Life.

[54]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[55]  Jürgen Schmidhuber,et al.  Sequential neural text compression , 1996, IEEE Trans. Neural Networks.

[56]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[57]  Jürgen Schmidhuber,et al.  Low-Complexity Art , 2017 .

[58]  J. Schmidhuber What''s interesting? , 1997 .

[59]  William I. Gasarch,et al.  Book Review: An introduction to Kolmogorov Complexity and its Applications Second Edition, 1997 by Ming Li and Paul Vitanyi (Springer (Graduate Text Series)) , 1997, SIGACT News.

[60]  Jürgen Schmidhuber,et al.  A Computer Scientist's View of Life, the Universe, and Everything , 1999, Foundations of Computer Science: Potential - Theory - Cognition.

[61]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[62]  Pat Langley,et al.  Learning Cooperative Lane Selection Strategies for Highways , 1998, AAAI/IAAI.

[63]  J. Schmidhuber Facial beauty and fractal geometry , 1998 .

[64]  Jürgen Schmidhuber,et al.  Reinforcement Learning with Self-Modifying Policies , 1998, Learning to Learn.

[65]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[66]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[67]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[68]  Jürgen Schmidhuber,et al.  Algorithmic Theories of Everything , 2000, ArXiv.

[69]  Jürgen Schmidhuber,et al.  Sequential Decision Making Based on Direct Search , 2001, Sequence Learning.

[70]  Jürgen Schmidhuber,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002, COLT.

[71]  Ofi rNw8x'pyzm,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002 .

[72]  Jürgen Schmidhuber,et al.  Hierarchies of Generalized Kolmogorov Complexities and Nonenumerable Universal Measures Computable in the Limit , 2002, Int. J. Found. Comput. Sci..

[73]  Risto Miikkulainen,et al.  Active Guidance for a Finless Rocket Using Neuroevolution , 2003, GECCO.

[74]  Jürgen Schmidhuber,et al.  Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements , 2003, ArXiv.

[75]  Aaron Sloman,et al.  Virtual Machines and Consciousness , 2003 .

[76]  Dolores Canamero,et al.  Designing emotions for activity selection in autonomous agents , 2003 .

[77]  Risto Miikkulainen,et al.  Robust non-linear control through neuroevolution , 2003 .

[78]  P. Haikonen The Cognitive Approach to Conscious Machines , 2003 .

[79]  Jürgen Schmidhuber,et al.  Exploring the predictable , 2003 .

[80]  Lakhmi C. Jain,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2004, Lecture Notes in Computer Science.

[81]  Nuttapong Chentanez,et al.  Intrinsically Motivated Reinforcement Learning , 2004, NIPS.

[82]  M. Balter Seeking the Key to Music , 2004, Science.

[83]  Jürgen Schmidhuber,et al.  Optimal Ordered Problem Solver , 2002, Machine Learning.

[84]  Jürgen Schmidhuber,et al.  Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement , 1997, Machine Learning.

[85]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[86]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning through Symbiotic Evolution , 2004 .

[87]  Marcus Hutter Simulation Algorithms for Computational Systems Biology , 2017, Texts in Theoretical Computer Science. An EATCS Series.

[88]  Jürgen Schmidhuber,et al.  Gödel Machines: Towards a Technical Justification of Consciousness , 2005, Adaptive Agents and Multi-Agent Systems.

[89]  Hod Lipson,et al.  Nonlinear system identification using coevolution of models and tests , 2005, IEEE Transactions on Evolutionary Computation.

[90]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[91]  Jürgen Schmidhuber,et al.  Completely Self-referential Optimal Reinforcement Learners , 2005, ICANN.

[92]  I. Aleksander The World in My Mind, My Mind in the World: Key Mechanisms of Consciousness in People, Animals and Machines , 2005 .

[93]  Marcus Hutter,et al.  Strong Asymptotic Assertions for Discrete MDL in Regression and Classification , 2005, ArXiv.

[94]  Jürgen Schmidhuber,et al.  Co-evolving recurrent neurons learn deep memory POMDPs , 2005, GECCO '05.

[95]  J. Schmidhuber Don't forget randomness is still just a hypothesis , 2006, Nature.

[96]  Jürgen Schmidhuber,et al.  Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts , 2006, Connect. Sci..

[97]  Anil K Seth,et al.  Theories and measures of consciousness: an extended framework. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[98]  Risto Miikkulainen,et al.  Efficient Non-linear Control Through Neuroevolution , 2006, ECML.

[99]  Douglas S. Blank,et al.  Introduction to developmental robotics , 2006, Connect. Sci..

[100]  Jürgen Schmidhuber 2006: Celebrating 75 Years of AI - History and Outlook: The Next 25 Years , 2006, 50 Years of Artificial Intelligence.

[101]  Jürgen Schmidhuber,et al.  Simple Algorithmic Principles of Discovery, Subjective Beauty, Selective Attention, Curiosity & Creativity , 2007, Discovery Science.

[102]  Jürgen Schmidhuber,et al.  Gödel Machines: Fully Self-referential Optimal Universal Self-improvers , 2007, Artificial General Intelligence.

[103]  Jürgen Schmidhuber,et al.  Simple Algorithmic Principles of Discovery, Subjective Beauty, Selective Attention, Curiosity and Creativity , 2007, ALT.

[104]  Jürgen Schmidhuber,et al.  New Millennium AI and the Convergence of History: Update of 2012 , 2012 .

[105]  Jürgen Schmidhuber,et al.  The New AI: General & Sound & Relevant for Physics , 2003, Artificial General Intelligence.

[106]  B. Baars,et al.  Cognition, Brain, and Consciousness: Introduction to Cognitive Neuroscience , 2007 .

[107]  Marcus Hutter,et al.  On Universal Prediction and Bayesian Confirmation , 2007, Theor. Comput. Sci..

[108]  Jürgen Schmidhuber,et al.  Policy Gradient Critics , 2007, ECML.

[109]  Martin Volker Butz How and Why the Brain Lays the Foundations for a Conscious Self , 2008 .

[110]  Jürgen Schmidhuber,et al.  Driven by Compression Progress , 2008, KES.

[111]  Jürgen Schmidhuber,et al.  State-Dependent Exploration for Policy Gradient Methods , 2008, ECML/PKDD.

[112]  Tom Schaul,et al.  Fitness Expectation Maximization , 2008, PPSN.

[113]  Risto Miikkulainen,et al.  Accelerated Neural Evolution through Cooperatively Coevolved Synapses , 2008, J. Mach. Learn. Res..

[114]  Frank Sehnke,et al.  Policy Gradients with Parameter-Based Exploration for Control , 2008, ICANN.

[115]  Tom Schaul,et al.  Natural Evolution Strategies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[116]  Jürgen Schmidhuber,et al.  Simple algorithmic theory of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes (特集 高次機能の学習と創発--脳・ロボット・人間研究における新たな展開) , 2009 .

[117]  Jürgen Schmidhuber,et al.  Ultimate Cognition à la Gödel , 2009, Cognitive Computation.

[118]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.