Optimal Ordered Problem Solver

We introduce a general and in a certain sense time-optimal way of solving one problem after another, efficiently searching the space of programs that compute solution candidates, including those programs that organize and manage and adapt and reuse earlier acquired knowledge. The Optimal Ordered Problem Solver (OOPS) draws inspiration from Levin's Universal Search designed for single problems and universal Turing machines. It spends part of the total search time for a new problem on testing programs that exploit previous solution-computing programs in computable ways. If the new problem can be solved faster by copy-editing/invoking previous code than by solving the new problem from scratch, then OOPS will find this out. If not, then at least the previous solutions will not cause much harm. We introduce an efficient, recursive, backtracking-based way of implementing OOPS on realistic computers with limited storage. Experiments illustrate how OOPS can greatly profit from metalearning or metasearching, that is, searching for faster search procedures.

[1]  K. Gödel.  Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I , 1931 .

[2]  K. Gödel.  Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I , 1931 .

[3]  A. Church.  Review: A. M. Turing, On Computable Numbers, with an Application to the Entscheidungsproblem , 1937 .

[4]  A. Turing.  On computable numbers, with an application to the Entscheidungsproblem , 1937, Proc. London Math. Soc..

[5]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[6]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[7]  John von Neumann,et al.  Theory Of Self Reproducing Automata , 1967 .

[8]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[9]  C. Cordell Green,et al.  Application of Theorem Proving to Problem Solving , 1969, IJCAI.

[10]  Richard C. T. Lee,et al.  PROW: A Step Toward Automatic Program Writing , 1969, IJCAI.

[11]  Charles H. Moore,et al.  Forth - a language for interactive computing , 1970 .

[12]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[13]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[14]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[15]  G. Chaitin.  A Theory of Program Size Formally Identical to Information Theory , 1975, JACM.

[16]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[17]  David Bulman.  Stack Computers [Guest editor's introduction] , 1977, Computer.

[18]  Michiharu Tsukamoto.  Program Stacking Technique , 1977 .

[19]  Hans J. Bremermann,et al.  Minimum energy requirements of information transfer and computing , 1982 .

[20]  Charles H. Bennett,et al.  The thermodynamics of computation—a review , 1982 .

[21]  T. Toffoli,et al.  Conservative logic , 2002, Collision-Based Computing.

[22]  Douglas B. Lenat,et al.  Theory Formation by Heuristic Search , 1983, Artificial Intelligence.

[23]  Stephen Wolfram,et al.  Universality and complexity in cellular automata , 1983 .

[24]  Leonid A. Levin,et al.  Randomness Conservation Inequalities; Information and Independence in Mathematical Theories , 1984, Inf. Control..

[25]  Paul E. Utgoff,et al.  Shift of bias for inductive concept learning , 1984 .

[26]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[27]  Pat Langley,et al.  Learning to Search: From Weak Methods to Domain-Specific Heuristics , 1985, Cogn. Sci..

[28]  Pat Langley,et al.  Learning to search : from weak methods to domain-specific heuristics , 1985 .

[29]  John H. Holland,et al.  Properties of the Bucket Brigade , 1985, ICGA.

[30]  Ray J. Solomonoff,et al.  The Application of Algorithmic Probability to Problems in Artificial Intelligence , 1985, UAI.

[31]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[32]  Charles W. Anderson,et al.  Learning and problem-solving with multilayer connectionist systems (adaptive, strategy learning, neural networks, reinforcement learning) , 1986 .

[33]  Allen Newell,et al.  GPS, a program that simulates human thought , 1995 .

[34]  B. Widrow,et al.  The truck backer-upper: an example of self-learning in neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[35]  Jr. Philip J. Koopman,et al.  Stack computers: the new wave , 1989 .

[36]  R. Solomonoff.  A SYSTEM FOR INCREMENTAL LEARNING BASED ON ALGORITHMIC PROBABILITY , 1989 .

[37]  Ray J. Solomonofi,et al.  A SYSTEM FOR INCREMENTAL LEARNING BASED ON ALGORITHMIC PROBABILITY , 1989 .

[38]  Jürgen Schmidhuber,et al.  Reinforcement Learning in Markovian and Non-Markovian Environments , 1990, NIPS.

[39]  Konrad Zuse,et al.  Rechnender Raum , 1991, Physik und Informatik.

[40]  Vladimir Vapnik,et al.  Principles of Risk Minimization for Learning Theory , 1991, NIPS.

[41]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[42]  Jürgen Schmidhuber,et al.  A ‘Self-Referential’ Weight Matrix , 1993 .

[43]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[44]  J. Schmidhuber.  An 'introspective' network that can learn to run its own weight change algorithm , 1993 .

[45]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[46]  Yves Deville,et al.  Logic Program Synthesis , 1994, J. Log. Program..

[47]  Juergen Schmidhuber,et al.  On learning how to learn learning strategies , 1994 .

[48]  Corso Elvezia.  Discovering Solutions with Low Kolmogorov Complexity and High Generalization Capability , 1995 .

[49]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[50]  Roland Olsson,et al.  Inductive Functional Programming Using Incremental Program Transformation , 1995, Artif. Intell..

[51]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[52]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[53]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[54]  Jürgen Schmidhuber,et al.  Solving POMDPs with Levin Search and EIRA , 1996, ICML.

[55]  Jieyu Zhao,et al.  Simple Principles of Metalearning , 1996 .

[56]  Wolfgang Banzhaf,et al.  Genetic Programming: An Introduction , 1997 .

[57]  Jürgen Schmidhuber,et al.  Discovering Neural Nets with Low Kolmogorov Complexity and High Generalization Capability , 1997, Neural Networks.

[58]  William I. Gasarch,et al.  Book Review: An introduction to Kolmogorov Complexity and its Applications Second Edition, 1997 by Ming Li and Paul Vitanyi (Springer (Graduate Text Series)) , 1997, SIGACT News.

[59]  Rafal Salustowicz,et al.  Probabilistic Incremental Program Evolution , 1997, Evolutionary Computation.

[60]  Bernhard Nebel,et al.  Extending Planning Graphs to an ADL Subset , 1997, ECP.

[61]  Jürgen Schmidhuber,et al.  Reinforcement Learning with Self-Modifying Policies , 1998, Learning to Learn.

[62]  Jürgen Schmidhuber,et al.  Evolving Structured Programs with Hierarchical Instructions and Skip Nodes , 1998, ICML.

[63]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[64]  Luca Maria Gambardella,et al.  Ant Algorithms for Discrete Optimization , 1999, Artificial Life.

[65]  Luca Maria Gambardella,et al.  An Ant Colony System Hybridized with a New Local Search for the Sequential Ordering Problem , 2000, INFORMS J. Comput..

[66]  S. Lloyd.  Ultimate physical limits to computation , 1999, Nature.

[67]  Jürgen Schmidhuber,et al.  Algorithmic Theories of Everything , 2000, ArXiv.

[68]  Sepp Hochreiter,et al.  Learning to Learn Using Gradient Descent , 2001, ICANN.

[69]  Jürgen Schmidhuber,et al.  Market-Based Reinforcement Learning in Partially Observable Worlds , 2001, ICANN.

[70]  Marcus Hutter,et al.  Towards a Universal Theory of Artificial Intelligence Based on Algorithmic Probability and Sequential Decisions , 2000, ECML.

[71]  Jürgen Schmidhuber,et al.  Sequential Decision Making Based on Direct Search , 2001, Sequence Learning.

[72]  Ofi rNw8x'pyzm,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002 .

[73]  Jürgen Schmidhuber,et al.  Hierarchies of Generalized Kolmogorov Complexities and Nonenumerable Universal Measures Computable in the Limit , 2002, Int. J. Found. Comput. Sci..

[74]  Jürgen Schmidhuber,et al.  Bias-Optimal Incremental Problem Solving , 2002, NIPS.

[75]  Marcus Hutter.  The Fastest and Shortest Algorithm for all Well-Defined Problems , 2002, Int. J. Found. Comput. Sci..

[76]  Marcus Hutter,et al.  Self-Optimizing and Pareto-Optimal Policies in General Environments based on Bayes-Mixtures , 2002, COLT.

[77]  Jürgen Schmidhuber,et al.  Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements , 2003, ArXiv.

[78]  Jürgen Schmidhuber,et al.  Exploring the predictable , 2003 .

[79]  Schmidhuber Juergen,et al.  The New AI: General & Sound & Relevant for Physics , 2003 .

[80]  Eric B. Baum,et al.  Toward a Model of Intelligence as an Economy of Agents , 1999, Machine Learning.

[81]  Andrea Bonarini,et al.  Bias-Optimal Incremental Learning of Control Sequences for Virtual Robots , 2004 .

[82]  Jürgen Schmidhuber,et al.  Learning Team Strategies: Soccer Case Studies , 1998, Machine Learning.

[83]  Jürgen Schmidhuber,et al.  Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement , 1997, Machine Learning.