Worst-case quadratic loss bounds for a generalization of the Widrow-Hoff rule

We prove worst-case bounds on the sum of squared errors incurred by a generalization of the classical Widrow-Hoff algorithm to inner product spaces. We describe applications of this result to obtain worst-case agnostic learning results for classes of smooth functions and of linear functions.

[1]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[2]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[3]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[4]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[5]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[6]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.

[7]  M. Zwaan.  An introduction to hilbert space , 1990 .

[8]  Philip M. Long,et al.  On-line learning of linear functions , 1991, STOC '91.

[9]  W. Härdle.  Smoothing Techniques: With Implementation in S , 1991 .

[10]  Jan Mycielski,et al.  Application of learning theorems , 1991, Fundam. Informaticae.

[11]  Neri Merhav,et al.  Universal sequential learning and decision from individual data sequences , 1992, COLT '92.

[12]  Philip M. Long,et al.  The learning complexity of smooth functions of a single variable , 1992, COLT '92.

[13]  Neri Merhav,et al.  Universal prediction of individual sequences , 1992, IEEE Trans. Inf. Theory.

[14]  David Haussler,et al.  How to use expert advice , 1993, STOC.