Simple algorithmic theory of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes (特集 高次機能の学習と創発--脳・ロボット・人間研究における新たな展開)

In this summary of previous work, I argue that data becomes temporarily interesting by itself to some selfimproving, but computationally limited, subjective observer once he learns to predict or compress the data in a better way, thus making it subjectively more “beautiful.” Curiosity is the desire to create or discover more non-random, non-arbitrary, “truly novel,” regular data that allows for compression progress because its regularity was not yet known. This drive maximizes “interestingness,” the first derivative of subjective beauty or compressibility, that is, the steepness of the learning curve. It motivates exploring infants, pure mathematicians, composers, artists, dancers, comedians, yourself, and recent artificial systems.

[1]  F. Galton.  Composite Portraits, Made by Combining Those of Many Different Persons Into a Single Resultant Figure. , 1879 .

[2]  K. Gödel.  Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I , 1931 .

[3]  A. Church.  Review: A. M. Turing, On Computable Numbers, with an Application to the Entscheidungsproblem , 1937 .

[4]  A. Turing.  On computable numbers, with an application to the Entscheidungsproblem , 1937, Proc. London Math. Soc..

[5]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[6]  J. Piaget.  The child's construction of reality , 1954 .

[7]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[8]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[9]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[10]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[11]  PAUL J. WERBOS,et al.  Generalization of backpropagation with application to a recurrent gas market model , 1988, Neural Networks.

[12]  H. B. Barlow,et al.  Finding Minimum Entropy Codes , 1989, Neural Computation.

[13]  Jürgen Schmidhuber,et al.  Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem , 1990 .

[14]  Stewart W. Wilson,et al.  A Possibility for Implementing Curiosity and Boredom in Model-Building Neural Controllers , 1991 .

[15]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[16]  Jürgen Schmidhuber,et al.  Learning to Generate Artificial Fovea Trajectories for Target Detection , 1991, Int. J. Neural Syst..

[17]  Jenq-Neng Hwang,et al.  Query-based learning applied to partially trained multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[18]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[19]  Jürgen Schmidhuber,et al.  A Fixed Size Storage O(n3) Time Complexity Learning Algorithm for Fully Recurrent Continually Running Networks , 1992, Neural Computation.

[20]  Jürgen Schmidhuber,et al.  Learning Factorial Codes by Predictability Minimization , 1992, Neural Computation.

[21]  David A. Cohn,et al.  Neural Network Exploration Using Optimal Experiment Design , 1993, NIPS.

[22]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[23]  Garrison W. Cottrell,et al.  Learning Mackey-Glass from 25 Examples, Plus or Minus 2 , 1993, NIPS.

[24]  D. Perrett,et al.  Facial shape and judgements of female attractiveness , 1994, Nature.

[25]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[26]  S. Hochreiter,et al.  REINFORCEMENT DRIVEN INFORMATION ACQUISITION IN NONDETERMINISTIC ENVIRONMENTS , 1995 .

[27]  Barak A. Pearlmutter.  Gradient calculations for dynamic recurrent neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[28]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[29]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[30]  Jürgen Schmidhuber,et al.  Sequential neural text compression , 1996, IEEE Trans. Neural Networks.

[31]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning through Symbiotic Evolution , 1996, Machine Learning.

[32]  Jürgen Schmidhuber,et al.  Low-Complexity Art , 2017 .

[33]  J. Schmidhuber.  What''s interesting? , 1997 .

[34]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[35]  J. Urgen Schmidhuber.  A Computer Scientist's View of Life, the Universe, and Everything , 1997 .

[36]  J. Schmidhuber.  Facial beauty and fractal geometry , 1998 .

[37]  Jürgen Schmidhuber,et al.  Reinforcement Learning with Self-Modifying Policies , 1998, Learning to Learn.

[38]  S. Pinker.  How the Mind Works , 1999, Annals of the New York Academy of Sciences.

[39]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[40]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[41]  Jürgen Schmidhuber,et al.  Algorithmic Theories of Everything , 2000, ArXiv.

[42]  Jürgen Schmidhuber,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002, COLT.

[43]  Jürgen Schmidhuber,et al.  Hierarchies of Generalized Kolmogorov Complexities and Nonenumerable Universal Measures Computable in the Limit , 2002, Int. J. Found. Comput. Sci..

[44]  Risto Miikkulainen,et al.  Active Guidance for a Finless Rocket Using Neuroevolution , 2003, GECCO.

[45]  Dolores Canamero,et al.  Designing emotions for activity selection in autonomous agents , 2003 .

[46]  Jürgen Schmidhuber,et al.  Exploring the predictable , 2003 .

[47]  Schmidhuber Juergen,et al.  The New AI: General & Sound & Relevant for Physics , 2003 .

[48]  Nuttapong Chentanez,et al.  Intrinsically Motivated Reinforcement Learning , 2004, NIPS.

[49]  M. Balter.  Seeking the Key to Music , 2004, Science.

[50]  Jürgen Schmidhuber,et al.  Optimal Ordered Problem Solver , 2002, Machine Learning.

[51]  Jürgen Schmidhuber,et al.  Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement , 1997, Machine Learning.

[52]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[53]  Dr. Marcus Hutter,et al.  Universal artificial intelligence , 2004 .

[54]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[55]  Jürgen Schmidhuber,et al.  Completely Self-referential Optimal Reinforcement Learners , 2005, ICANN.

[56]  Jürgen Schmidhuber,et al.  Co-evolving recurrent neurons learn deep memory POMDPs , 2005, GECCO '05.

[57]  Jürgen Schmidhuber,et al.  Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts , 2006, Connect. Sci..

[58]  Risto Miikkulainen,et al.  Efficient Non-linear Control Through Neuroevolution , 2006, ECML.

[59]  Douglas S. Blank,et al.  Introduction to developmental robotics , 2006, Connect. Sci..

[60]  Jürgen Schmidhuber.  2006: Celebrating 75 Years of AI - History and Outlook: The Next 25 Years , 2006, 50 Years of Artificial Intelligence.

[61]  Jürgen Schmidhuber,et al.  Simple Algorithmic Principles of Discovery, Subjective Beauty, Selective Attention, Curiosity & Creativity , 2007, Discovery Science.

[62]  Jürgen Schmidhuber,et al.  Gödel Machines: Fully Self-referential Optimal Universal Self-improvers , 2007, Artificial General Intelligence.

[63]  Jürgen Schmidhuber,et al.  New Millennium AI and the Convergence of History: Update of 2012 , 2012 .

[64]  Jürgen Schmidhuber,et al.  Policy Gradient Critics , 2007, ECML.

[65]  Marcus Hutter,et al.  Sequential Decisions based on Algorithmic Probability , 2008 .

[66]  Tom Schaul,et al.  Fitness Expectation Maximization , 2008, PPSN.

[67]  Jürgen Schmidhuber,et al.  Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes , 2008, ABiALS.

[68]  Tom Schaul,et al.  Natural Evolution Strategies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[69]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.