Learning automata from ordered examples

Connectionist learning models have had considerable empirical success, but it is hard to characterize exactly what they learn. The learning of finite-state languages (FSL) from example strings is a domain which has been extensively studied and might provide an opportunity to help understand connectionist learning. A major problem is that traditional FSL learning assumes the storage of all examples and thus violates connectionist principles. This paper presents a provably correct algorithm for inferring any minimum-state deterministic finite-state automata (FSA) from a complete ordered sample using limited total storage and without storing example strings. The algorithm is an iterative strategy that uses at each stage a current encoding of the data considered so far, and one single sample string. One of the crucial advantages of our algorithm is that the total amount of space used in the course of learning for encoding any finite prefix of the sample is polynomial in the size of the inferred minimum state deterministic FSA. The algorithm is also relatively efficient in time and has been implemented. More importantly, there is a connectionist version of the algorithm that preserves these properties. The connectionist version requires much more structure than the usual models and has been implemented using the Rochester Connectionist Simulator. We also show that no machine with finite working storage can iteratively identify the FSL from arbitrary presentations.

[1]  DANA ANGLUIN,et al.  On the Complexity of Minimum Inference of Regular Sets , 1978, Inf. Control..

[2]  Jerome A. Feldman,et al.  On the Synthesis of Finite-State Machines from Samples of Their Behavior , 1972, IEEE Transactions on Computers.

[3]  Ronald L. Rivest,et al.  Diversity-based inference of finite automata , 1994, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[4]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[5]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[6]  Balas K. Natarajan,et al.  On learning Boolean functions , 1987, STOC.

[7]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[8]  Jerome A. Feldman,et al.  Connectionist models and their implications , 1988 .

[9]  E. Mark Gold,et al.  System identification via state characterization , 1972 .

[10]  E. Mark Gold,et al.  Complexity of Automaton Identification from Given Data , 1978, Inf. Control..

[11]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[12]  Tao Jiang,et al.  Learning regular languages from counterexamples , 1988, Annual Conference Computational Learning Theory.

[13]  James Jay Horning,et al.  A study of grammatical inference , 1969 .

[14]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[15]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[16]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[17]  守屋 悦朗,et al.  J.E.Hopcroft, J.D. Ullman 著, "Introduction to Automata Theory, Languages, and Computation", Addison-Wesley, A5変形版, X+418, \6,670, 1979 , 1980 .

[18]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[19]  Dana Angluin,et al.  A Note on the Number of Queries Needed to Identify Regular Languages , 1981, Information and Control.

[20]  Dana Charmian Angluin,et al.  An application of the theory of computational complexity to the study of inductive inference. , 1976 .

[21]  Robert E. Schapire,et al.  A new approach to unsupervised learning in deterministic environments , 1990 .

[22]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[23]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..