Towards Representation Independence in PAC Learning

In the recent development of various models of learning inspired by the PAC learning model (introduced by Valiant) there has been a trend towards models which are as representation independent as possible. We review this development and discuss the advantages of representation independence. Motivated by the research in learning, we propose a framework for studying the combinatorial properties of representations.

[1]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[2]  J. M. Barzdin,et al.  Prognostication of Automata and Functions , 1971, IFIP Congress.

[3]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[4]  Karlis Podnieks.  Comparing various concepts of function prediction. Part 1. , 1974 .

[5]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[6]  John Gill,et al.  Computational Complexity of Probabilistic Turing Machines , 1977, SIAM J. Comput..

[7]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[8]  Andrew Chi-Chih Yao,et al.  Theory and application of trapdoor functions , 1982, 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982).

[9]  Vladimir Vapnik,et al.  Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics) , 1982 .

[10]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[11]  D. Pollard.  Convergence of stochastic processes , 1984 .

[12]  R. Dudley.  A course on empirical processes , 1984 .

[13]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[14]  École d'été de probabilités de Saint-Flour,et al.  École d'Été de Probabilités de Saint-Flour XII - 1982 , 1984 .

[15]  Leonid A. Levin,et al.  One-way functions and pseudorandom generators , 1985, STOC '85.

[16]  Silvio Micali,et al.  How to construct random functions , 1986, JACM.

[17]  David Haussler,et al.  Epsilon-nets and simplex range queries , 1986, SCG '86.

[18]  Ronald L. Rivest,et al.  Learning decision lists , 2004, Machine Learning.

[19]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[20]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[21]  David Haussler,et al.  ɛ-nets and simplex range queries , 1987, Discret. Comput. Geom..

[22]  Ronald L. Rivest,et al.  Training a 3-node neural network is NP-complete , 1988, COLT '88.

[23]  Leonard Pitt,et al.  Reductions among prediction problems: on the difficulty of predicting automata , 1988, [1988] Proceedings. Structure in Complexity Theory Third Annual Conference.

[24]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[25]  David Haussler,et al.  Learning decision trees from random examples , 1988, COLT '88.

[26]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[27]  David Haussler,et al.  Equivalence of models for polynomial learnability , 1988, COLT '88.

[28]  Hugo Krawczyk,et al.  On the Existence of Pseudorandom Generators , 1988, CRYPTO.

[29]  M. Kearns,et al.  Crytographic limitations on learning Boolean formulae and finite automata , 1989, STOC '89.

[30]  B. K. Natarajan,et al.  Some results on learning , 1989 .

[31]  D. Haussler.  Generalizing the PAC model: sample size bounds from metric dimension-based uniform convergence results , 1989, 30th Annual Symposium on Foundations of Computer Science.

[32]  Jeffrey Scott Vitter,et al.  Complexity issues in learning by neural nets , 1989, COLT '89.

[33]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[34]  Manfred K. Warmuth,et al.  Learning nested differences of intersection-closed concept classes , 2004, Machine Learning.

[35]  Leonard Pitt,et al.  A polynomial-time algorithm for learning k-variable pattern languages from examples , 1989, COLT '89.

[36]  Robert E. Schapire,et al.  The strength of weak learnability , 1990, Mach. Learn..