Barrier Boosting

Boosting algorithms like AdaBoost and Arc-GV are iterative strategies to minimize a constrained objective function, equivalent to Barrier algorithms. Based on this new understanding it is shown that convergence of Boosting-type algorithms becomes simpler to prove and we outline directions to develop further Boosting schemes. In particular a new Boosting technique for regression – "-Boost – is proposed.

[1]  H. Schwenk,et al.  Adaboosting neural networks , 1997 .

[2]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[3]  Paola Campadelli,et al.  A Boosting Algorithm for Regression , 1997, ICANN.

[5]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[6]  Leonid Mosheyev,et al.  Penalty/Barrier multiplier algorthm for semidefinit programming , 2000 .

[7]  David W. Opitz,et al.  An Empirical Evaluation of Bagging and Boosting , 1997, AAAI/IAAI.

[8]  Peter L. Bartlett,et al.  Functional Gradient Techniques for Combining Hypotheses , 2000 .

[9]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[10]  O. Mangasarian,et al.  Robust linear programming discrimination of two linearly inseparable sets , 1992 .

[11]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[12]  Gunnar Rätsch,et al.  An asymptotical Analysis and Improvement of AdaBoost in the binary classification case , 2000 .

[13]  J. Dussault,et al.  Stable exponential-penalty algorithm with superlinear convergence , 1994 .

[14]  Peter L. Bartlett,et al.  Neural Network Learning - Theoretical Foundations , 1999 .

[15]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[16]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[17]  Manfred K. Warmuth,et al.  Additive versus exponentiated gradient updates for linear prediction , 1995, STOC '95.

[18]  Gunnar Rätsch,et al.  Regularizing AdaBoost , 1998, NIPS.

[19]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[20]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[21]  Manfred K. Warmuth,et al.  Boosting as entropy projection , 1999, COLT '99.

[22]  David P. Helmbold,et al.  Potential Boosters? , 1999, NIPS.

[23]  Harris Drucker,et al.  Comparison of learning algorithms for handwritten digit recognition , 1995 .

[24]  O. Mangasarian.  Linear and Nonlinear Separation of Patterns by Linear Programming , 1965 .

[25]  Harris Drucker,et al.  Boosting Performance in Neural Networks , 1993, Int. J. Pattern Recognit. Artif. Intell..

[26]  Robert J. Vanderbei,et al.  An Interior-Point Algorithm for Nonconvex Nonlinear Programming , 1999, Comput. Optim. Appl..

[27]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[28]  J. Lafferty.  Additive models, boosting, and inference for generalized divergences , 1999, COLT '99.

[30]  David P. Helmbold,et al.  Leveraging for Regression , 2000, COLT.

[31]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[32]  Leo Breiman,et al.  Prediction Games and Arcing Algorithms , 1999, Neural Computation.

[33]  B. Schölkopf,et al.  Linear programs for automatic accuracy control in regression. , 1999 .

[34]  Gunnar Rätsch,et al.  Robust Ensemble Learning , 2000 .

[35]  Alexander J. Smola,et al.  Advances in Large Margin Classifiers , 2000 .

[36]  John Shawe-Taylor,et al.  A Column Generation Algorithm For Boosting , 2000, ICML.

[37]  Marc Teboulle,et al.  An Interior Proximal Algorithm and the Exponential Multiplier Method for Semidefinite Programming , 1998, SIAM J. Optim..

[38]  Yoram Singer,et al.  Leveraged Vector Machines , 1999, NIPS.

[39]  Dale Schuurmans,et al.  Boosting in the Limit: Maximizing the Margin of Learned Ensembles , 1998, AAAI/IAAI.

[40]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[41]  D. Bertsekas,et al.  Multiplier methods for convex programming , 1973, CDC 1973.

[42]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.