Knowledge transfer in SVM and neural networks

The paper considers general machine learning models, where knowledge transfer is positioned as the main method to improve their convergence properties. Previous research was focused on mechanisms of knowledge transfer in the context of SVM framework; the paper shows that this mechanism is applicable to neural network framework as well. The paper describes several general approaches for knowledge transfer in both SVM and ANN frameworks and illustrates algorithmic implementations and performance of one of these approaches for several synthetic examples.

[1]  Rauf Izmailov,et al.  Learning with Privileged Information for Improved Target Classification , 2014, Int. J. Monit. Surveillance Technol. Res..

[2]  Christoph H. Lampert,et al.  Learning to Rank Using Privileged Information , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Martin T. Hagan,et al.  Neural network design , 1995 .

[4]  Rauf Izmailov,et al.  Learning with Intelligent Teacher: Similarity Control and Knowledge Transfer - In memory of Alexey Chervonenkis , 2015, SLDS.

[5]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[6]  Heng Yang,et al.  Privileged information-based conditional regression forest for facial feature detection , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[7]  Jeff Heaton,et al.  Introduction to Neural Networks for C#, 2nd Edition , 2008 .

[8]  Rauf Izmailov,et al.  Learning using privileged information: similarity control and knowledge transfer , 2015, J. Mach. Learn. Res..

[9]  Peter Tiño,et al.  Incorporating Privileged Information Through Metric Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Vladimir Vapnik,et al.  A new learning paradigm: Learning using privileged information , 2009, Neural Networks.

[11]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[12]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[13]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[14]  Howard B. Demuth,et al.  Neutral network toolbox for use with Matlab , 1995 .

[15]  Kevin N. Gurney,et al.  An introduction to neural networks , 2018 .

[16]  Hector J. Levesque,et al.  Knowledge Representation and Reasoning , 2004 .

[17]  Vladimir Naumovich Vapnik,et al.  Estimation of dependences based on empirical data ; : Empirical inference science : afterword of 2006 , 2006 .

[18]  Ning Chen,et al.  Enhanced default risk models with SVM+ , 2012, Expert Syst. Appl..

[19]  Thornton C. Fry.  Mathematics as a Profession Today in Industry , 1956 .

[20]  Rauf Izmailov,et al.  SMO-Style Algorithms for Learning Using Privileged Information , 2010, DMIN.