Multifactor sparse feature extraction using Convolutive Nonnegative Tucker Decomposition

Multilinear algebra of the higher-order tensor has been proposed as a potential mathematical framework for machine learning to investigate the relationships among multiple factors underlying the observations. One popular model Nonnegative Tucker Decomposition (NTD) allows us to explore the interactions of different factors with nonnegative constraints. In order to reduce degeneracy problem of tensor decomposition caused by component delays, convolutive tensor decomposition model is an appropriate model for exploring temporal correlations. In this paper, a flexible two stage algorithm for K-mode Convolutive Nonnegative Tucker Decomposition (K-CNTD) model is proposed using an alternating least square procedure. This model can be seen as a convolutive extension of Nonnegative Tucker Decomposition. The patterns across columns in convolutive tensor model are investigated to represent audio and image considering multiple factors. We employ the K-CNTD algorithm to extract the shift-invariant sparse features in different subspaces for robust speaker recognition and Alzheimer's Disease(AD) diagnosis task. The experimental results confirm the validity of our proposed algorithm and indicate that it is able to improve the speaker recognition performance especially in noisy conditions and has potential application on AD diagnosis.

[1]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[2]  J. Leeuw,et al.  Principal component analysis of three-mode data by means of alternating least squares algorithms , 1980 .

[3]  Lieven De Lathauwer,et al.  An enhanced line search scheme for complex-valued tensor decompositions. Application in DS-CDMA , 2008, Signal Process..

[4]  Paris Smaragdis,et al.  Convolutive Speech Bases and Their Application to Supervised Speech Separation , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[5]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[6]  Lieven De Lathauwer,et al.  A Block Component Model-Based Blind DS-CDMA Receiver , 2008, IEEE Transactions on Signal Processing.

[7]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[8]  R. Harshman,et al.  Shifted factor analysis—Part I: Models and properties , 2003 .

[9]  A. Stegeman.  Degeneracy in Candecomp/Parafac and Indscal Explained For Several Three-Sliced Arrays With A Two-Valued Typical Rank , 2007, Psychometrika.

[10]  Andrzej Cichocki,et al.  Sparse Super Symmetric Tensor Factorization , 2007, ICONIP.

[11]  HyvärinenAapo.  Sparse code shrinkage , 1999 .

[12]  Michael S. Lewicki,et al.  Efficient auditory coding , 2006, Nature.

[13]  Andrzej Cichocki,et al.  Extended HALS algorithm for nonnegative Tucker decomposition and its applications for multiway analysis and classification , 2011, Neurocomputing.

[14]  Seungjin Choi,et al.  Nonnegative Tucker Decomposition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  L. Lathauwer,et al.  An enhanced plane search scheme for complex-valued tensor decompositions , 2010 .

[16]  Claus A. Andersson,et al.  PARAFAC2—Part II. Modeling chromatographic data with retention time shifts , 1999 .

[17]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part I: Lemmas for Partitioned Matrices , 2008, SIAM J. Matrix Anal. Appl..

[18]  S MarcusDaniel,et al.  Open Access Series of Imaging Studies (OASIS) , 2007 .

[19]  Xuelong Li,et al.  General Tensor Discriminant Analysis and Gabor Features for Gait Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[21]  Richard A. Harshman,et al.  Shifted factor analysis—Part III: N‐way generalization and application , 2003 .

[22]  Sergio Cruces,et al.  Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix Factorization , 2011, Entropy.

[23]  Saeid Sanei,et al.  A new tensor factorization approach for convolutive blind source separation in time domain , 2010, 2010 18th European Signal Processing Conference.

[24]  J W Belliveau,et al.  Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. , 1995, Science.

[25]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part III: Alternating Least Squares Algorithms , 2008, SIAM J. Matrix Anal. Appl..

[26]  Haiping Lu,et al.  A survey of multilinear subspace learning for tensor data , 2011, Pattern Recognit..

[27]  Constantine Kotropoulos,et al.  Non-Negative Multilinear Principal Component Analysis of Auditory Temporal Modulations for Music Genre Classification , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[28]  S. Boll,et al.  Suppression of acoustic noise in speech using spectral subtraction , 1979 .

[29]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[30]  Liqing Zhang,et al.  Robust Feature Extraction for Speaker Recognition Based on Constrained Nonnegative Tensor Factorization , 2010, Journal of Computer Science and Technology.

[31]  Lars Kai Hansen,et al.  Shift-invariant multilinear decomposition of neuroimaging data , 2008, NeuroImage.

[32]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[33]  Inderjit S. Dhillon,et al.  Generalized Nonnegative Matrix Approximations with Bregman Divergences , 2005, NIPS.

[34]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[35]  Kyuwan Choi,et al.  Detecting the Number of Clusters in n-Way Probabilistic Clustering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  John G. Csernansky,et al.  Open Access Series of Imaging Studies (OASIS): Cross-sectional MRI Data in Young, Middle Aged, Nondemented, and Demented Older Adults , 2007, Journal of Cognitive Neuroscience.

[37]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[38]  Erkki Oja,et al.  Sparse Code Shrinkage: Denoising by Nonlinear Maximum Likelihood Estimation , 1998, NIPS.

[39]  Andrzej Cichocki,et al.  Fast and Efficient Algorithms for Nonnegative Tucker Decomposition , 2008, ISNN.

[40]  R. Buxton,et al.  Dynamics of blood flow and oxygenation changes during brain activation: The balloon model , 1998, Magnetic resonance in medicine.

[41]  Tamara G. Kolda,et al.  Pattern Analysis of Directed Graphs Using DEDICOM: An Application to Enron Email , 2006 .

[42]  R. Harshman,et al.  Shifted factor analysis—Part II: Algorithms , 2003 .

[43]  Mikkel N. Schmidt,et al.  Shift Invariant Sparse Coding of Image and Music Data , 2007 .