Common and Discriminative Subspace Kernel-Based Multiblock Tensor Partial Least Squares Regression

In this work, we introduce a new generalized nonlinear tensor regression framework called kernel-based multiblock tensor partial least squares (KMTPLS) for predicting a set of dependent tensor blocks from a set of independent tensor blocks through the extraction of a small number of common and discriminative latent components. By considering both common and discriminative features, KMTPLS effectively fuses the information from multiple tensorial data sources and unifies the single and multiblock tensor regression scenarios into one general model. Moreover, in contrast to multilinear model, KMTPLS successfully addresses the nonlinear dependencies between multiple response and predictor tensor blocks by combining kernel machines with joint Tucker decomposition, resulting in a significant performance gain in terms of predictability. An efficient learning algorithm for KMTPLS based on sequentially extracting common and discriminative latent vectors is also presented. Finally, to show the effectiveness and advantages of our approach, we test it on the real-life regression task in computer vision, i.e., reconstruction of human pose from multiview video sequences.

[1]  Xiaoshan Li,et al.  Tucker Tensor Regression and Neuroimaging Analysis , 2013, Statistics in biosciences.

[2]  Yali Wang,et al.  Online local Gaussian process for tensor-variate regression: Application to fast reconstruction of limb movements from brain signal , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Liqing Zhang,et al.  Multilinear and nonlinear generalizations of partial least squares: an overview of recent advances , 2014, WIREs Data Mining Knowl. Discov..

[4]  Age K. Smilde,et al.  Multiway multiblock component and covariates regression models , 2000 .

[5]  Ruzena Bajcsy,et al.  Berkeley MHAD: A comprehensive Multimodal Human Action Database , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[6]  Weiwei Guo,et al.  Tensor Learning for Regression , 2012, IEEE Transactions on Image Processing.

[7]  Brahim Chaib-draa,et al.  Hierarchical tucker tensor regression: Application to brain imaging data analysis , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[8]  R. Bro Multiway calibration. Multilinear PLS , 1996 .

[9]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[10]  Hongtu Zhu,et al.  Tensor Regression with Applications in Neuroimaging Data Analysis , 2012, Journal of the American Statistical Association.

[11]  Remco C. Veltkamp,et al.  UMPM benchmark: A multi-person dataset with synchronized video and motion capture data for evaluation of articulated human motion and interaction , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[12]  J. Macgregor,et al.  Analysis of multiblock and hierarchical PCA and PLS models , 1998 .

[13]  L. Luo,et al.  Quality prediction based on HOPLS-CP for batch processes , 2015 .

[14]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[15]  Andrzej Cichocki,et al.  Tensor Decompositions: A New Concept in Brain Data Analysis? , 2013, ArXiv.

[16]  Hyunsoo Kim,et al.  A three-stage framework for gene expression data analysis by L1-norm support vector regression , 2005, Int. J. Bioinform. Res. Appl..

[17]  H. Abdi Partial least squares regression and projection on latent structure regression (PLS Regression) , 2010 .

[18]  Roman Rosipal,et al.  Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space , 2002, J. Mach. Learn. Res..

[19]  Andrey Eliseyev,et al.  Recursive N-Way Partial Least Squares for Brain-Computer Interface , 2013, PloS one.

[20]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[21]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[22]  Liqing Zhang,et al.  Kernel-based tensor partial least squares for reconstruction of limb movements , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[23]  Liqing Zhang,et al.  Tensor-variate Gaussian processes regression and its application to video surveillance , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  Naotaka Fujii,et al.  Higher Order Partial Least Squares (HOPLS): A Generalized Multilinear Regression Method , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[26]  R. C. Veltkamp,et al.  Utrecht Multi-Person Motion ( UMPM ) benchmark , 2011 .