Kernelization of Tensor-Based Models for Multiway Data Analysis: Processing of Multidimensional Structured Data

Tensors (also called multiway arrays) are a generalization of vectors and matrices to higher dimensions based on multilinear algebra. The development of theory and algorithms for tensor decompositions (factorizations) has been an active area of study within the past decade, e.g., [1] and [2]. These methods have been successfully applied to many problems in unsupervised learning and exploratory data analysis. Multiway analysis enables one to effectively capture the multilinear structure of the data, which is usually available as a priori information about the data. Hence, it might provide advantages over matrix factorizations by enabling one to more effectively use the underlying structure of the data. Besides unsupervised tensor decompositions, supervised tensor subspace regression and classification formulations have been also successfully applied to a variety of fields including chemometrics, signal processing, computer vision, and neuroscience.

[1]  M. Aizerman,et al.  Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning , 1964 .

[2]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[3]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[4]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[5]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[6]  Andrzej Cichocki,et al.  Kernel PCA for Feature Extraction and De-Noising in Nonlinear Regression , 2001, Neural Computing & Applications.

[7]  Roman Rosipal,et al.  Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space , 2002, J. Mach. Learn. Res..

[8]  Demetri Terzopoulos,et al.  Multilinear Analysis of Image Ensembles: TensorFaces , 2002, ECCV.

[9]  Dominik Endres,et al.  A new metric for probability distributions , 2003, IEEE Transactions on Information Theory.

[10]  Nuno Vasconcelos,et al.  A Kullback-Leibler Divergence Based Kernel for SVM Classification in Multimedia Applications , 2003, NIPS.

[11]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[12]  Antoni B. Chan,et al.  A Family of Probabilistic Kernels Based on Information Divergence , 2004 .

[13]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[14]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[15]  Yang Li,et al.  Kernel-based multifactor analysis for image synthesis and recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[16]  Kenji Fukumizu,et al.  Statistical Convergence of Kernel CCA , 2005, NIPS.

[17]  Rajesh P. N. Rao,et al.  Learning Shared Latent Structure for Image Synthesis and Robotic Imitation , 2005, NIPS.

[18]  Michael I. Jordan,et al.  A Probabilistic Interpretation of Canonical Correlation Analysis , 2005 .

[19]  Neil D. Lawrence,et al.  Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models , 2005, J. Mach. Learn. Res..

[20]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words , 2006, BMVC.

[21]  Marios Savvides,et al.  Individual Kernel Tensor-Subspaces for Robust Face Recognition: A Computationally Efficient Tensor Framework Without Requiring Mode Factorization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Tae-Kyun Kim,et al.  Learning Motion Categories using both Semantic and Structural Information , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Colin Fyfe,et al.  Elsevier Editorial System(tm) for Neurocomputing Title: Gaussian Processes for Canonical Correlation Analysis Gaussian Processes for Canonical Correlation Analysis , 2022 .

[24]  John Shawe-Taylor,et al.  Decomposing the tensor kernel support vector machine for neuroscience data with structured labels , 2010, Machine Learning.

[25]  Tae-Kyun Kim,et al.  Canonical Correlation Analysis of Video Volume Tensors for Action Categorization and Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[27]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[28]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[29]  Trevor Darrell,et al.  Factorized Orthogonal Latent Spaces , 2010, AISTATS.

[30]  J. Ross Beveridge,et al.  Action classification on product manifolds , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Mubarak Shah,et al.  Human Action Recognition in Videos Using Kinematic Features and Multiple Instance Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Jieping Ye,et al.  Canonical Correlation Analysis for Multilabel Classification: A Least-Squares Formulation, Extensions, and Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Guodong Guo,et al.  Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression , 2011, CVPR 2011.

[34]  Sheng Tang,et al.  Localized Multiple Kernel Learning for Realistic Human Action Recognition in Videos , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Johan A. K. Suykens,et al.  A kernel-based framework to tensorial data analysis , 2011, Neural Networks.

[36]  Changsheng Xu,et al.  Boosted Exemplar Learning for Action Recognition and Annotation , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Frank P. Ferrie,et al.  A Note on Metric Properties for Some Divergence Measures: The Gaussian Case , 2012, ACML.

[38]  Johan A. K. Suykens,et al.  Classification of Multichannel Signals With Cumulant-Based Kernels , 2012, IEEE Transactions on Signal Processing.

[39]  Tanaya Guha,et al.  Learning Sparse Representations for Human Action Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Thomas B. Moeslund,et al.  A Local 3-D Motion Descriptor for Multi-View Human Action Recognition from 4-D Spatio-Temporal Interest Points , 2012, IEEE Journal of Selected Topics in Signal Processing.

[41]  Zenglin Xu,et al.  Infinite Tucker Decomposition: Nonparametric Bayesian Models for Multiway Data Analysis , 2011, ICML.

[42]  Naotaka Fujii,et al.  Higher Order Partial Least Squares (HOPLS): A Generalized Multilinear Regression Method , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.