Generalization performance of Gaussian kernels SVMC based on Markov sampling

In this paper we consider Gaussian RBF kernels support vector machine classification (SVMC) algorithm with uniformly ergodic Markov chain (u.e.M.c.) samples in reproducing kernel Hilbert spaces (RKHS). We analyze the learning rates of Gaussian RBF kernels SVMC based on u.e.M.c. samples and obtain the fast learning rate of Gaussian RBF kernels SVMC based on u.e.M.c. samples by using the strongly mixing property of u.e.M.c. samples. We also present the numerical studies on the learning performance of Gaussian RBF kernels SVMC based on Markov sampling for real-world datasets. These experimental results show that Gaussian RBF kernels SVMC based on Markov sampling has better learning performance compared to randomly independent sampling.

[1]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[2]  S. Smale,et al.  ONLINE LEARNING WITH MARKOV SAMPLING , 2009 .

[3]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[4]  Felipe Cucker,et al.  Best Choices for Regularization Parameters in Learning Theory: On the Bias—Variance Problem , 2002, Found. Comput. Math..

[5]  M. Mohri,et al.  Stability Bounds for Stationary φ-mixing and β-mixing Processes , 2010 .

[6]  Zongben Xu,et al.  The generalization performance of ERM algorithm with strongly mixing observations , 2009, Machine Learning.

[7]  Yiming Ying,et al.  Support Vector Machine Soft Margin Classifiers: Error Analysis , 2004, J. Mach. Learn. Res..

[8]  R N Curnow.  The use of Markov chain models in studying the evolution of the proteins. , 1988, Journal of theoretical biology.

[9]  Mario Martín,et al.  On-Line Support Vector Machine Regression , 2002, ECML.

[10]  Zongben Xu,et al.  The learning performance of support vector machine classification based on Markov sampling , 2013, Science China Information Sciences.

[11]  Murray Rosenblatt,et al.  Uniform ergodicity and strong mixing , 1972 .

[12]  Don R. Hush,et al.  Learning from dependent observations , 2007, J. Multivar. Anal..

[13]  Lizhong Peng,et al.  Analysis of Support Vector Machines Regression , 2009, Found. Comput. Math..

[14]  Lorenzo Rosasco,et al.  Model Selection for Regularized Least-Squares Algorithm in Learning Theory , 2005, Found. Comput. Math..

[15]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[16]  Dharmendra S. Modha,et al.  Minimum complexity regression estimation with weakly dependent observations , 1996, IEEE Trans. Inf. Theory.

[17]  S. Smale,et al.  Shannon sampling II: Connections to learning theory , 2005 .

[18]  Bin Yu.  RATES OF CONVERGENCE FOR EMPIRICAL PROCESSES OF STATIONARY MIXING SEQUENCES , 1994 .

[19]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[20]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[21]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[22]  Yuan Yan Tang,et al.  Generalization Performance of Fisher Linear Discriminant Based on Markov Sampling , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[24]  Chao Zhang,et al.  Generalization Bounds of ERM-Based Learning Processes for Continuous-Time Markov Chains , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Ingo Steinwart,et al.  Fast rates for support vector machines using Gaussian kernels , 2007, 0708.1838.

[26]  Alexander Gammerman,et al.  Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[27]  Tong Zhang.  Statistical behavior and consistency of classification methods based on convex risk minimization , 2003 .

[28]  Yiming Ying,et al.  Learning Rates of Least-Square Regularized Regression , 2006, Found. Comput. Math..

[29]  Y. Davydov.  Mixing Conditions for Markov Chains , 1974 .

[30]  Mathukumalli Vidyasagar,et al.  Learning and Generalization: With Applications to Neural Networks , 2002 .