General twin support vector machine with pinball loss function

Abstract The standard twin support vector machine (TSVM) uses the hinge loss function which leads to noise sensitivity and instability. In this paper, we propose a novel general twin support vector machine with pinball loss (Pin-GTSVM) for solving classification problems. We show that the proposed Pin-GTSVM is noise insensitive and more stable for re-sampling. Further, the computational complexity of the proposed Pin-GTSVM is similar to that of the TSVM. Thus, the pinball loss function does not increase the computation time of the proposed Pin-GTSVM. Numerical experiments with different noise are performed on 17 UCI and KEEL benchmark real-world datasets and the results are compared with other baseline methods. The comparisons clearly show that the proposed Pin-GTSVM has better generalization performance for noise corrupted datasets.

[1]  Danushka Bollegala,et al.  A Web Search Engine-Based Approach to Measure Semantic Similarity between Words , 2011, IEEE Transactions on Knowledge and Data Engineering.

[2]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[3]  Yuqun Zhang,et al.  A maximum margin and minimum volume hyper-spheres machine with pinball loss for imbalanced data classification , 2016, Knowl. Based Syst..

[4]  Yuan-Hai Shao,et al.  Nonparallel hyperplane support vector machine for binary classification problems , 2014, Inf. Sci..

[5]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[7]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[8]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[10]  Muhammad Tanveer,et al.  A robust fuzzy least squares twin support vector machine for class imbalance learning , 2018, Appl. Soft Comput..

[11]  Muhammad Tanveer.  Robust and Sparse Linear Programming Twin Support Vector Machines , 2014, Cognitive Computation.

[12]  Mohammad Tanveer,et al.  Smooth Twin Support Vector Machines via Unconstrained Convex Minimization , 2017 .

[13]  Yong Shi,et al.  Large-scale linear nonparallel SVMs , 2018, Soft Comput..

[14]  Xinjun Peng,et al.  PTSVRs: Regression models via projection twin support vector machine , 2018, Inf. Sci..

[15]  S. Balasundaram,et al.  On proximal bilateral-weighted fuzzy support vector machine classifiers , 2012, Int. J. Adv. Intell. Paradigms.

[16]  Lan Bai,et al.  Stochastic Gradient Twin Support Vector Machine for Large Scale Problems , 2017, Inf. Sci..

[17]  Chi-Jen Lu,et al.  Tree Decomposition for Large-Scale SVM Problems , 2010, 2010 International Conference on Technologies and Applications of Artificial Intelligence.

[18]  Izhar Ahmad,et al.  One norm linear programming support vector regression , 2016, Neurocomputing.

[19]  Muhammad Tanveer,et al.  A regularization on Lagrangian twin support vector regression , 2015, International Journal of Machine Learning and Cybernetics.

[20]  Johan A. K. Suykens,et al.  Support Vector Machine Classifier With Pinball Loss , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Muhammad Tanveer,et al.  Robust energy-based least squares twin support vector machines , 2015, Applied Intelligence.

[22]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[23]  Muhammad Tanveer,et al.  Improved universum twin support vector machine , 2018, 2018 IEEE Symposium Series on Computational Intelligence (SSCI).

[24]  Panos M. Pardalos,et al.  A novel perspective on multiclass classification: Regular simplex support vector machine , 2019, Inf. Sci..

[25]  Andreas Christmann,et al.  How SVMs can estimate quantiles and the median , 2007, NIPS.

[26]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[27]  Reshma Khemchandani,et al.  Angle-based twin support vector machine , 2017, Annals of Operations Research.

[28]  Xianli Pan,et al.  A Novel Twin Support-Vector Machine With Pinball Loss , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Yuanqing Li,et al.  Joint feature re-extraction and classification using an iterative semi-supervised support vector machine algorithm , 2008, Machine Learning.

[30]  José Salvador Sánchez,et al.  On the effectiveness of preprocessing methods when dealing with different levels of class imbalance , 2012, Knowl. Based Syst..

[31]  S. Balasundaram,et al.  On Lagrangian twin support vector regression , 2012, Neural Computing and Applications.

[32]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[33]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[34]  Reshma Khemchandani,et al.  Robust least squares twin support vector machine for human activity recognition , 2016, Appl. Soft Comput..

[35]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[36]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[37]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[38]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[39]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[40]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Chih-Jen Lin,et al.  A study on SMO-type decomposition methods for support vector machines , 2006, IEEE Trans. Neural Networks.

[42]  Muhammad Tanveer,et al.  EEG signal classification using universum support vector machine , 2018, Expert Syst. Appl..

[43]  Pei-Yi Hao,et al.  New support vector algorithms with parametric insensitive/margin model , 2010, Neural Networks.

[44]  Xin Shen,et al.  Support vector machine classifier with truncated pinball loss , 2017, Pattern Recognit..

[45]  Paul M. Mather,et al.  Support vector machines for classification in remote sensing , 2005 .