Boosting information fusion

Ensemble methods provide a principled framework for building high performance classifiers and representing many types of data. As a result, these methods can be useful for making inferences in many domains such as classification and multi-modal biometrics. We introduce a novel ensemble method for combining multiple representations (or views). The method is a multiple view generalization of AdaBoost. Similar to AdaBoost, base classifiers are independently built from each representation. Unlike AdaBoost, however, all data types share the same sampling distribution as the view whose weighted training error is the smallest among all the views. As a result, the most consistent data type dominates over time, thereby significantly reducing sensitivity to noise. In addition, our proposal is provably better than AdaBoost trained on any single type of data. The proposed method is applied to the problems of facial and gender prediction based on biometric traits as well as of protein classification. Experimental results show that our method outperforms several competing techniques including kernel-based data fusion.

[1]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[2]  Olivier Bousquet,et al.  On the Complexity of Learning the Kernel Matrix , 2002, NIPS.

[3]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[4]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[5]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Robert Bregovic,et al.  Multirate Systems and Filter Banks , 2002 .

[7]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[8]  UedaNaonori.  Optimal Linear Combination of Neural Networks for Improving Classification Performance , 2000 .

[9]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[11]  Josef Kittler,et al.  Combining classifiers: A theoretical framework , 1998, Pattern Analysis and Applications.

[12]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[13]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[15]  Josef Kittler,et al.  A Framework for Classifier Fusion: Is It Still Needed? , 2000, SSPR/SPR.

[16]  Dmitry Panchenko,et al.  Some New Bounds on the Generalization Error of Combined Classifiers , 2000, NIPS.

[17]  Dmitrij Frishman,et al.  MIPS: a database for genomes and protein sequences , 1999, Nucleic Acids Res..

[18]  Paul A. Viola,et al.  Fast and Robust Classification using Asymmetric AdaBoost and a Detector Cascade , 2001, NIPS.

[19]  Ludmila I. Kuncheva,et al.  A Theoretical Study on Six Classifier Fusion Strategies , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .