Bottom-Up Induction of Oblivious Read-Once Decision Graphs: Strengths and Limitations

We report improvements to HOODG, a supervised learning algorithm that induces concepts from labelled instances using oblivious, read-once decision graphs as the underlying hypothesis representation structure. While it is shown that the greedy approach to variable ordering is locally optimal, we also show an inherent limitation of all bottom-up induction algorithms, including HOODG, that construct such decision graphs bottom-up by minimizing the width of levels in the resulting graph. We report our empirical experiments that demonstrate the algorithm's generalization power.

[1]  Masahiro Fujita,et al.  Variable ordering algorithms for ordered binary decision diagrams and their evaluation , 1993, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[2]  Ravi B. Boppana,et al.  The Complexity of Finite Functions , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[3]  Ron Kohavi,et al.  Irrelevant Features and the Subset Selection Problem , 1994, ICML.

[4]  Kenneth J. Supowit,et al.  Finding the Optimal Variable Ordering for Binary Decision Diagrams , 1987, 24th ACM/IEEE Design Automation Conference.

[5]  Masahiro Fujita,et al.  On variable ordering of binary decision diagrams for the application of multi-level logic synthesis , 1991, Proceedings of the European Conference on Design Automation..

[6]  Randal E. Bryant,et al.  Efficient implementation of a BDD package , 1991, DAC '90.

[7]  Yasuhiko Takenaga,et al.  NP-completeness of Minimum Binary Decision Diagram Identification Problems , 1993 .

[8]  Randal E. Bryant,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 1986, IEEE Transactions on Computers.

[9]  Nagisa Ishiura,et al.  Shared binary decision diagram with attributed edges for efficient Boolean function manipulation , 1990, 27th ACM/IEEE Design Automation Conference.

[10]  Christoph Meinel,et al.  Branching Programs - An Efficient Data Structure for Computer-Aided Circuit Design , 1992, Bull. EATCS.

[11]  Sebastian Thrun,et al.  The MONK''s Problems-A Performance Comparison of Different Learning Algorithms, CMU-CS-91-197, Sch , 1991 .

[12]  Edmund M. Clarke,et al.  Symbolic Model Checking: 10^20 States and Beyond , 1990, Inf. Comput..

[13]  Sheldon B. Akers,et al.  Binary Decision Diagrams , 1978, IEEE Transactions on Computers.

[14]  Hiroshi Sawada,et al.  Minimization of binary decision diagrams based on exchanges of variables , 1991, 1991 IEEE International Conference on Computer-Aided Design Digest of Technical Papers.

[15]  Carsten Lund,et al.  On the hardness of approximating minimization problems , 1994, JACM.

[16]  David A. Mix Barrington,et al.  Bounded-width polynomial-size branching programs recognize exactly those languages in NC1 , 1986, STOC '86.

[17]  Claude E. Shannon,et al.  The synthesis of two-terminal switching circuits , 1949, Bell Syst. Tech. J..

[18]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[19]  Randal E. Bryant,et al.  Symbolic Boolean manipulation with ordered binary-decision diagrams , 1992, CSUR.

[20]  J. Ross Quinlan,et al.  An Empirical Comparison of Genetic and Decision-Tree Classifiers , 1988, ML.

[21]  Thomas G. Dietterich,et al.  Readings in Machine Learning , 1991 .

[22]  George Boole,et al.  An Investigation of the Laws of Thought: Frontmatter , 2009 .

[23]  C. Y. Lee.  Representation of switching circuits by binary-decision programs , 1959 .

[24]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[25]  G. Boole.  An Investigation of the Laws of Thought: On which are founded the mathematical theories of logic and probabilities , 2007 .

[26]  Ingo Wegener,et al.  The Complexity of Symmetric Boolean Functions , 1987, Computation Theory and Logic.

[27]  Bernard M. E. Moret,et al.  Decision Trees and Diagrams , 1982, CSUR.

[28]  V. Dvorak,et al.  An optimization technique for ordered (binary) decision diagrams , 1992, CompEuro 1992 Proceedings Computer Systems and Software Engineering.

[29]  Shin-ichi Minato,et al.  Minimum-Width Method of Variable Ordering for Binary Decision Diagrams , 1992 .

[30]  Jonathan J. Oliver.  Decision Graphs - An Extension of Decision Trees , 1993 .

[31]  Ingo Wegener,et al.  The complexity of Boolean functions , 1987 .

[32]  Edmund M. Clarke,et al.  Representing circuits more efficiently in symbolic model checking , 1991, 28th ACM/IEEE Design Automation Conference.

[33]  William Joseph Masek,et al.  A fast algorithm for the string editing problem and decision graph complexity , 1976 .