Hedging Structured Concepts

We develop an online algorithm called Component Hedge for learning structured concept classes when the loss of a structured concept sums over its components. Example classes include paths through a graph (composed of edges) and partial permutations (composed of assignments). The algorithm maintains a parameter vector with one non-negative weight per component, which always lies in the convex hull of the structured concept class. The algorithm predicts by decomposing the current parameter vector into a convex combination of concepts and choosing one of those concepts at random. The parameters are updated by first performing a multiplicative update and then projecting back into the convex hull. We show that Component Hedge has optimal regret bounds for a large variety of structured concept classes.

[1]  Manfred K. Warmuth,et al.  Repeated Games against Budgeted Adversaries , 2010, NIPS.

[2]  S. V. N. Vishwanathan,et al.  Entropy Regularized LPBoost , 2008, ALT.

[3]  Elad Hazan,et al.  Competing in the Dark: An Efficient Algorithm for Bandit Linear Optimization , 2008, COLT.

[4]  Manfred K. Warmuth,et al.  When Random Play is Optimal Against an Adversary , 2008, COLT.

[5]  Manfred K. Warmuth,et al.  Optimal strategies from random walks , 2008, COLT 2008.

[6]  Manfred K. Warmuth,et al.  Randomized Online PCA Algorithms with Regret Bounds that are Logarithmic in the Dimension , 2008 .

[7]  Manfred K. Warmuth,et al.  Learning Permutations with Exponential Weights , 2007, COLT.

[8]  Xavier Carreras,et al.  Structured Prediction Models via the Matrix-Tree Theorem , 2007, EMNLP.

[9]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[10]  Manfred K. Warmuth,et al.  Optimum Follow the Leader Algorithm , 2005, COLT.

[11]  Marcus Hutter,et al.  Adaptive Online Prediction by Following the Perturbed Leader , 2005, J. Mach. Learn. Res..

[12]  Eric Vigoda,et al.  A polynomial-time approximation algorithm for the permanent of a matrix with nonnegative entries , 2004, JACM.

[13]  Mark Herbster,et al.  Tracking the Best Expert , 1995, Machine Learning.

[14]  Santosh S. Vempala,et al.  Efficient algorithms for online decision problems , 2005, J. Comput. Syst. Sci..

[15]  Alexander Schrijver,et al.  Combinatorial optimization. Polyhedra and efficiency. , 2003 .

[16]  Manfred K. Warmuth,et al.  Path Kernels and Multiplicative Updates , 2002, J. Mach. Learn. Res..

[17]  Manfred K. Warmuth,et al.  Tracking a Small Set of Experts by Mixing Past Posteriors , 2003, J. Mach. Learn. Res..

[18]  Mark Herbster,et al.  Tracking the Best Linear Predictor , 2001, J. Mach. Learn. Res..

[19]  David Haussler,et al.  Sequential Prediction of Individual Sequences Under General Loss Functions , 1998, IEEE Trans. Inf. Theory.

[20]  Manfred K. Warmuth,et al.  How to use expert advice , 1997, JACM.

[21]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[22]  Paul Brace Follow the leader , 1992 .

[23]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[24]  Dan Gusfield,et al.  Connectivity and Edge-Disjoint Spanning Trees , 1983, Information Processing Letters.