Inductive Inference: Theory and Methods

There has been a great deal of theoretical and experimental work in computer science on inductive inference systems, that is, systems that try to infer general rules from examples. However, a complete and applicable theory of such systems is still a distant goal. This survey highlights and explains the main ideas that have been developed in the study of inductive inference, with special emphasis on the relations between the general theory and the specific algorithms and implementations. 154 references.

[1]  Richard M. Friedberg,et al.  Three theorems on recursive enumeration. I. Decomposition. II. Maximal set. III. Enumeration without duplication , 1958, Journal of Symbolic Logic.

[2]  Michael Satosi Watanabe,et al.  Information-Theoretical Aspects of Inductive and Deductive Inference , 1960, IBM J. Res. Dev..

[3]  H A SIMON,et al.  HUMAN ACQUISITION OF CONCEPTS FOR SEQUENTIAL PATTERNS. , 1963, Psychological review.

[4]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[5]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[6]  Per Martin-Löf,et al.  The Definition of Random Sequences , 1966, Inf. Control..

[7]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[8]  Manuel Blum.  On the Size of Machines , 1967, Inf. Control..

[9]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[10]  Staffan Persson.  Some sequence extrapolating programs : a study of representation and modeling in inquiring systems , 1968 .

[11]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[12]  S. Reder,et al.  Grammatical complexity and inference , 1969 .

[13]  James Jay Horning,et al.  A study of grammatical inference , 1969 .

[14]  R. Harré,et al.  Probability and Confirmation , 1970 .

[15]  Patrick Henry Winston,et al.  Learning structural descriptions from examples , 1970 .

[16]  L. Levin,et al.  THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS , 1970 .

[17]  Stefano Crespi-Reghizzi,et al.  An Effective Model for Grammar Interference , 1971, IFIP Congress.

[18]  P. Martin-Löf.  Complexity oscillations in infinite binary sequences , 1971 .

[19]  J. M. Barzdin,et al.  Prognostication of Automata and Functions , 1971, IFIP Congress.

[20]  Janis Barzdins.  On synthesizing programs given by examples , 1972, International Sympoisum on Theoretical Programming.

[21]  Jerome A. Feldman,et al.  On the Synthesis of Finite-State Machines from Samples of Their Behavior , 1972, IEEE Transactions on Computers.

[22]  E. Mark Gold,et al.  System identification via state characterization , 1972 .

[23]  G. Plotkin.  Automatic Methods of Inductive Inference , 1972 .

[24]  Jerome A. Feldman,et al.  Some Decidability Results on Grammatical Inference and Complexity , 1972, Inf. Control..

[25]  J. Feldman,et al.  A SURVEY OF RESULTS IN GRAMMATICAL INFERENCE , 1972 .

[26]  Alan W. Biermann,et al.  On the Inference of Turing Machines from Sample Computations , 1972, Artif. Intell..

[27]  Gabor T. Herman,et al.  Algorithms for Producing Grammars from Sample Derivations: A Common Problem of Formal Language Theory and Developmental Biology , 1973, J. Comput. Syst. Sci..

[28]  Stefano Crespi-Reghizzi,et al.  The use of grammatical inference for designing programming languages , 1973, Commun. ACM.

[29]  Robert P. Daley.  An Example of Information and Computation Resource Trade-Off , 1973, JACM.

[30]  Gregory J. Chaitin,et al.  Information-Theoretic Limitations of Formal Systems , 1974, JACM.

[31]  P. G. Doucet.  The Syntactic Inference Problem for D0L-Sequences , 1974, L Systems.

[32]  R. M. Wharton.  Approximate Language Identification , 1974, Inf. Control..

[33]  Karlis Podnieks.  Comparing various concepts of function prediction. Part 1. , 1974 .

[34]  R. V. Freivald.  Minimal Gödel Numbers and Their Identification in the Limit , 1975, MFCS.

[35]  G. Chaitin.  A Theory of Program Size Formally Identical to Information Theory , 1975, JACM.

[36]  Alan W. Biermann,et al.  Speeding up the Synthesis of Programs from Traces , 1975, IEEE Transactions on Computers.

[37]  Rolf Wiehagen.  Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..

[38]  Laurent Siklóssy,et al.  Automatic Program Synthesis from Example Problems , 1975, IJCAI.

[39]  R. J. Solomonoff,et al.  Inductive inference theory: a unified approach to problems in pattern recognition and artificial intelligence , 1975, IJCAI 1975.

[40]  J. Brayer,et al.  A note on the k-tail method of tree grammar inference , 1975 .

[41]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[42]  P. D. Summers,et al.  Program construction from examples. , 1975 .

[43]  Eliana Minicozzi,et al.  Some Natural Properties of Strong-Identification in Inductive Inference , 1976, Theor. Comput. Sci..

[44]  Kathleen Knobe,et al.  A Method for Inferring Context-free Grammars , 1976, Inf. Control..

[45]  Azriel Rosenfeld,et al.  Grammatical inference by hill climbing , 1976, Inf. Sci..

[46]  Gregory J. Chaitin.  Information-Theoretic Characterizations of Recursive Infinite Strings , 1976, Theor. Comput. Sci..

[47]  Alan W. Biermann,et al.  Constructing Programs from Example Computations , 1976, IEEE Transactions on Software Engineering.

[48]  Brian R. Gaines,et al.  Behaviour/structure transformations under uncertainty , 1976 .

[49]  Rolf Wiehagen,et al.  Charakteristische Eigenschaften von erkennbaren Klassen rekursiver Funktionen , 1976, J. Inf. Process. Cybern..

[50]  Jean-Pierre Jouannaud,et al.  SISP/1: An Interactive System Able to Synthesize Functions from Examples , 1977, IJCAI.

[51]  R. M. Wharton.  Grammar Enumeration and Inference , 1977, Inf. Control..

[52]  Robert P. Daley.  On the Inference of Optimal Descriptions , 1977, Theor. Comput. Sci..

[53]  L. Laudan.  Progress and Its Problems , 1977 .

[54]  Peter Kugel,et al.  Induction, Pure and Simple , 1977, Inf. Control..

[55]  Taylor L. Booth,et al.  Inference of Finite-State Probabilistic Grammars , 1977, IEEE Transactions on Computers.

[56]  Moshé M. Zloof.  Query-by-Example: A Data Base Language , 1977, IBM Syst. J..

[57]  Phillip D. Summers,et al.  A Methodology for LISP Program Construction from Examples , 1977, J. ACM.

[58]  Rolf Wiehagen,et al.  Identification of Formal Languages , 1977, MFCS.

[59]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[60]  Rolf Wiehagen,et al.  Characterization Problems in the Theory of Inductive Inference , 1978, ICALP.

[61]  John Case,et al.  Anomaly hierarchies of mechanized inductive inference , 1978, STOC.

[62]  Giovanni Guida,et al.  Noncounting Context-Free Languages , 1978, JACM.

[63]  Bruce G. Buchanan,et al.  Dendral and Meta-Dendral: Their Applications Dimension , 1978, Artif. Intell..

[64]  Alan W. Biermann,et al.  The Inference of Regular LISP Programs from Examples , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[65]  L. P. J. Veelenturf,et al.  Inference of Sequential Machines from Sample Computations , 1978, IEEE Transactions on Computers.

[66]  Claude Pair,et al.  Inference for Regular Bilanguages , 1978, J. Comput. Syst. Sci..

[67]  Adrian Walker,et al.  On the Inference of Stochastic Regular Grammars , 1978, Inf. Control..

[68]  Frederick Hayes-Roth,et al.  An interference matching technique for inducing abstractions , 1978, CACM.

[69]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[70]  E. Mark Gold,et al.  Complexity of Automaton Identification from Given Data , 1978, Inf. Control..

[71]  DANA ANGLUIN,et al.  On the Complexity of Minimum Inference of Regular Sets , 1978, Inf. Control..

[72]  John W. Carr,et al.  A Solution of the Syntactical Induction-Inference Problem for Regular Languages , 1978, Comput. Lang..

[73]  Klaus P. Jantke,et al.  Natural Properties of Strategies Identifying Recursive Functions , 1979, J. Inf. Process. Cybern..

[74]  Salvatore J. Stolfo,et al.  Automatic Discovery of Heuristics for Nondeterministic Programs , 1979, IJCAI.

[75]  Daniel Kayser,et al.  Construction of Natural Language Sentence Acceptors by a Supervised-Learning Technique , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  Rolf Wiehagen,et al.  Inductive Inference with Additional Information , 1979, J. Inf. Process. Cybern..

[77]  Thomas G. Dietterich,et al.  Learning and Generalization of Characteristic Descriptions: Evaluation Criteria and Comparative Review of Selected Methods , 1979, IJCAI.

[78]  Brian R. Gaines.  Maryanski's Grammatical Inferencer , 1979, IEEE Transactions on Computers.

[79]  Jean-Pierre Jouannaud,et al.  Characterization of a Class of Functions Synthesized from Examples by a SUMMERS Like Method Using a "B.M.W." Matching Technique , 1979, IJCAI.

[80]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[81]  Philip Klahr,et al.  Design issues for exemplary programming , 1980 .

[82]  Kenneth Wexler,et al.  Formal Principles of Language Acquisition , 1980 .

[83]  Rolf Wiehagen,et al.  Research in the theory of inductive inference by GDR mathematicians - A survey , 1980, Inf. Sci..

[84]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[85]  Laurent Miclet.  Regular Inference with a Tail-Clustering Method , 1980, IEEE Transactions on Systems, Man, and Cybernetics.

[86]  Ryszard S. Michalski,et al.  Pattern Recognition as Rule-Guided Inductive Inference , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[87]  Steven A. Vere,et al.  Multilevel Counterfactuals for Generalizations of Relational Concepts and Productions , 1980, Artif. Intell..

[88]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[89]  Dana Angluin,et al.  A Note on the Number of Queries Needed to Identify Regular Languages , 1981, Inf. Control..

[90]  Michael G. Thomason,et al.  Syntactic Pattern Recognition, An Introduction , 1978, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[91]  Péter Gács.  On the relation between descriptional complexity and algorithmic probability , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[92]  Barry Levine,et al.  Derivatives of Tree Sets with Applications to Grammatical Inference , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[93]  Robert P. Daley.  On the Error Correcting Power of Pluralism in Inductive Inference , 1981, FCT.

[94]  Dana Angluin,et al.  Inference of Reversible Languages , 1982, JACM.

[95]  Tom M. Mitchell,et al.  Generalization as Search , 2002 .

[96]  Takeshi Shinohara,et al.  Polynomial Time Inference of Extended Regular Pattern Languages , 1983, RIMS Symposium on Software Science and Engineering.

[97]  Rusins Freivalds,et al.  Inductive Inference and Computable One-One Numberings , 1982, Math. Log. Q..

[98]  Keh-Jiann Chen.  Tradeoffs in the Inductive Inference of Nearly Minimal Size Programs , 1982, Inf. Control..

[99]  Alan Bundy,et al.  A Critical Survey of Rule Learning Programs , 1982, ECAI.

[100]  Michael G. Thomason,et al.  Syntactic Methods in Pattern Recognition , 1982 .

[101]  Carl H. Smith,et al.  The Power of Pluralism for Automatic Program Synthesis , 1982, JACM.

[102]  Barry Levine.  The Use of Tree Derivatives and a Sample Support Parameter for Inferring Tree Systems , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[103]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[104]  Thomas G. Dietterich,et al.  Learning and Inductive Inference , 1982 .

[105]  Ehud Y. Shapiro,et al.  Algorithmic program diagnosis , 1982, POPL '82.

[106]  Daniel N. Osherson,et al.  Criteria of Language Learning , 1982, Inf. Control..

[107]  Ehud Shapiro,et al.  Algorithmic Program Debugging , 1983 .

[108]  Péter Gács,et al.  On the relation between descriptional complexity and algorithmic probability , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[109]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[110]  O. Firschein,et al.  Syntactic pattern recognition and applications , 1983, Proceedings of the IEEE.

[111]  Robert P. Daley.  On the Error Correcting Power of Pluralism in BC-Type Inductive Inference , 1983, Theoretical Computer Science.

[112]  Robert P. Nix,et al.  Editing by example , 1984 .

[113]  Carl H. Smith,et al.  On the Complexity of Inductive Inference (Preliminary Report) , 1984, MFCS.