Radial kernels and their reproducing kernel Hilbert spaces

We describe how to use Schoenberg's theorem for a radial kernel combined with existing bounds on the approximation error functions for Gaussian kernels to obtain a bound on the approximation error function for the radial kernel. The result is applied to the exponential kernel and Student's kernel. To establish these results we develop a general theory regarding mixtures of kernels. We analyze the reproducing kernel Hilbert space (RKHS) of the mixture in terms of the RKHS's of the mixture components and prove a type of Jensen inequality between the approximation error function for the mixture and the approximation error functions of the mixture components.

[1]  Ingo Steinwart,et al.  Fast rates for support vector machines using Gaussian kernels , 2007, 0708.1838.

[2]  B. Schölkopf,et al.  An Oracle Inequality for Clipped Regularized Risk Minimizers , 2007 .

[3]  H. Bauer.  Measure and integration theory , 2001 .

[4]  C. Berg,et al.  Harmonic Analysis on Semigroups: Theory of Positive Definite and Related Functions , 1984 .

[5]  C. Castaing,et al.  Convex analysis and measurable multifunctions , 1977 .

[6]  Yuesheng Xu,et al.  Universal Kernels , 2006, J. Mach. Learn. Res..

[7]  Robert D. Nowak,et al.  TEMPLAR: a wavelet-based framework for pattern learning and analysis , 2004, IEEE Transactions on Signal Processing.

[8]  Hans Ulrich Simon,et al.  General Polynomial Time Decomposition Algorithms , 2005, J. Mach. Learn. Res..

[9]  Kai Wing Yip,et al.  A generalized Jensen’s inequality , 1975 .

[10]  F. Smithies.  Linear Operators , 2019, Nature.

[11]  I. J. Schoenberg.  Metric spaces and completely monotone functions , 1938 .

[12]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[13]  Ingo Steinwart,et al.  A new concentration result for regularized risk minimizers , 2006, math/0612779.

[14]  A. E. Taylor,et al.  Linear Functionals on Certain Spaces of Abstractly-Valued Functions , 1938 .

[15]  Ingo Steinwart,et al.  Fast Rates for Support Vector Machines , 2005, COLT.

[16]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[17]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[18]  Irene A. Stegun,et al.  Handbook of Mathematical Functions. , 1966 .

[19]  Don R. Hush,et al.  QP Algorithms with Guaranteed Accuracy and Run Time for Support Vector Machines , 2006, J. Mach. Learn. Res..

[20]  D. Widder,et al.  The Laplace Transform , 1943, The Mathematical Gazette.

[21]  Don R. Hush,et al.  Polynomial-Time Decomposition Algorithms for Support Vector Machines , 2003, Machine Learning.

[22]  V. Barbu,et al.  Convexity and optimization in banach spaces , 1972 .

[23]  R. Ash,et al.  Real analysis and probability , 1975 .

[24]  S. Bernstein,et al.  Sur les fonctions absolument monotones , 1929 .

[25]  E. Hewitt,et al.  Abstract Harmonic Analysis , 1963 .

[26]  S. Bochner.  Monotone Funktionen, Stieltjessche Integrale und harmonische Analyse , 1933 .

[27]  Clayton Scott,et al.  Performance Measures for Neyman–Pearson Classification , 2007, IEEE Transactions on Information Theory.

[28]  Harold R. Parks,et al.  Geometric Integration Theory , 2008 .

[29]  N. Dinculeanu.  Vector Integration and Stochastic Integration in Banach Spaces , 2000, Oxford Handbooks Online.

[30]  Feng Qi (祁锋).  Bounds for the Ratio of Two Gamma Functions , 2009 .

[31]  Robert D. Nowak,et al.  A Neyman-Pearson approach to statistical learning , 2005, IEEE Transactions on Information Theory.

[32]  S. Saitoh.  Integral Transforms, Reproducing Kernels and Their Applications , 1997 .