Unlabeled sample compression schemes and corner peelings for ample and maximum classes

We examine connections between combinatorial notions that arise in machine learning and topological notions in cubical/simplicial geometry. These connections enable to export results from geometry to machine learning. Our first main result is based on a geometric construction by Tracy Hall (2004) of a partial shelling of the cross-polytope which can not be extended. We use it to derive a maximum class of VC dimension 3 that has no corners. This refutes several previous works in machine learning from the past 11 years. In particular, it implies that all previous constructions of optimal unlabeled sample compression schemes for maximum classes are erroneous. On the positive side we present a new construction of an unlabeled sample compression scheme for maximum classes. We leave as open whether our unlabeled sample compression scheme extends to ample (a.k.a. lopsided or extremal) classes, which represent a natural and far-reaching generalization of maximum classes. Towards resolving this question, we provide a geometric characterization in terms of unique sink orientations of the 1-skeletons of associated cubical complexes.

[1]  G. Ziegler Lectures on Polytopes , 1994 .

[2]  Hunter R. Johnson,et al.  Some new maximum VC classes , 2013, Inf. Process. Lett..

[3]  S. Ben-David,et al.  Combinatorial Variability of Vapnik-chervonenkis Classes with Applications to Sample Compression Schemes , 1998, Discrete Applied Mathematics.

[4]  Ron Aharoni,et al.  Independent systems of representatives in weighted graphs , 2007, Comb..

[5]  Boting Yang,et al.  Generalizing Labeled and Unlabeled Sample Compression to Multi-label Concept Classes , 2014, ALT.

[6]  Paul H. Edelman,et al.  The theory of convex geometries , 1985 .

[7]  Manfred K. Warmuth,et al.  Relating Data Compression and Learnability , 2003 .

[8]  Manfred K. Warmuth,et al.  Unlabeled Compression Schemes for Maximum Classes, , 2007, COLT.

[9]  Jirí Matousek The Number Of Unique-Sink Orientations of the Hypercube* , 2006, Comb..

[10]  David Haussler,et al.  Predicting {0,1}-functions on randomly drawn points , 1988, COLT '88.

[11]  Shay Moran,et al.  Teaching and compressing for low VC-dimension , 2015, Electron. Colloquium Comput. Complex..

[12]  Shay Moran,et al.  Sample compression schemes for VC classes , 2015, 2016 Information Theory and Applications Workshop (ITA).

[13]  Tibor Szabó,et al.  Unique sink orientations of cubes , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[14]  Glynn Winskel,et al.  Events in computation , 1980 .

[15]  James L. Peterson,et al.  Petri Nets , 1977, CSUR.

[16]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[17]  J. Lawrence Lopsided sets and orthant-intersection by convex sets , 1983 .

[18]  Benjamin I. P. Rubinstein,et al.  A Geometric Approach to Sample Compression , 2009, J. Mach. Learn. Res..

[19]  Manfred K. Warmuth,et al.  Sample Compression, Learnability, and the Vapnik-Chervonenkis Dimension , 1995, Machine Learning.

[20]  Bruno Benedetti,et al.  Metric Geometry, Convexity and Collapsibility , 2011 .

[21]  Shay Moran,et al.  Labeled Compression Schemes for Extremal Classes , 2015, ALT.

[22]  Claes Johnson,et al.  Mathematics and Computation , 2023, Springer Proceedings in Mathematics & Statistics.

[23]  H. Bandelt,et al.  Metric graph theory and geometry: a survey , 2006 .

[24]  Manfred K. Warmuth Compressing to VC Dimension Many Points , 2003, COLT.

[25]  Michah Sageev,et al.  CAT(0) cube complexes and groups , 2014 .

[26]  Peter L. Bartlett,et al.  Bounding Embeddings of VC Classes into Maximum Classes , 2014, ArXiv.

[27]  de Ng Dick Bruijn A combinatorial problem , 1946 .

[28]  Emo Welzl,et al.  Vapnik-Chervonenkis dimension and (pseudo-)hyperplane arrangements , 1994, Discret. Comput. Geom..

[29]  Alex M. Andrew,et al.  Boosting: Foundations and Algorithms , 2012 .

[30]  Hans-Jürgen Bandelt,et al.  Combinatorics of lopsided sets , 2006, Eur. J. Comb..

[31]  Béla Bollobás,et al.  Defect Sauer Results , 1995, J. Comb. Theory, Ser. A.

[32]  H. Tracy Hall Counterexamples in Discrete Geometry , 2004 .

[33]  Sally Floyd,et al.  Space-bounded learning and the Vapnik-Chervonenkis dimension , 1989, COLT '89.

[34]  Ingo Andreas Schurr,et al.  Unique sink orientations of cubes , 2004 .

[35]  Bruno Benedetti,et al.  Collapsibility of CAT(0) spaces , 2011, Geometriae Dedicata.

[36]  Yoav Freund,et al.  Boosting: Foundations and Algorithms , 2012 .

[37]  Gábor Tardos,et al.  Unlabeled Compression Schemes Exceeding the VC-dimension , 2018, Discret. Appl. Math..

[38]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[39]  Norbert Sauer,et al.  On the Density of Families of Sets , 1972, J. Comb. Theory, Ser. A.

[40]  Shay Moran,et al.  Shattering, Graph Orientations, and Connectivity , 2012, Electron. J. Comb..

[41]  Glynn Winskel,et al.  Petri Nets, Event Structures and Domains , 1979, Semantics of Concurrent Computation.

[42]  Shay Moran,et al.  Shattering-Extremal Systems , 2012, ArXiv.

[43]  S. Shelah A combinatorial problem; stability and order for models and theories in infinitary languages. , 1972 .

[44]  Martin Tancer,et al.  d-collapsibility is NP-complete for d greater or equal to 4 , 2008, Chic. J. Theor. Comput. Sci..

[45]  Glynn Winskel,et al.  Petri Nets, Event Structures and Domains, Part I , 1981, Theor. Comput. Sci..

[46]  David Newnham Shattering news. , 2016, Nursing standard (Royal College of Nursing (Great Britain) : 1987).

[47]  Lajos Rónyai,et al.  Shattering-Extremal Set Systems of VC Dimension at most 2 , 2014, Electron. J. Comb..

[48]  A. O. Houcine On hyperbolic groups , 2006 .

[49]  R. Forman Morse Theory for Cell Complexes , 1998 .

[50]  Hans Ulrich Simon,et al.  Recursive teaching dimension, VC-dimension and sample compression , 2014, J. Mach. Learn. Res..

[51]  Hans-Jürgen Bandelt,et al.  COMs: Complexes of oriented matroids , 2015, J. Comb. Theory, Ser. A.

[52]  Ćemal B. Dolićanin,et al.  The Geometric Approach , 2014 .

[53]  Andreas W. M. Dress,et al.  Towards a theory of holistic clustering , 1996, Mathematical Hierarchies and Biology.