Robust Multilinear Tensor Rank Estimation Using Higher Order Singular Value Decomposition and Information Criteria

Model selection in tensor decomposition is important for real applications if the rank of the original data tensor is unknown and the observed tensor is noisy. In the Tucker model, the minimum description length (MDL) or Bayesian information criteria have been applied to tensors via matrix unfolding, but these methods are sensitive to noise when the tensors have a multilinear low rank structure given by the Tucker model. In this study, we propose new methods for improving the MDL so it is more robust to noise. The proposed methods are justified theoretically by analyzing the “multilinear low-rank structure” of tensors. Extensive experiments including numerical simulations and a real application to image denoising are provided to illustrate the advantages of the proposed methods.

[1]  Mariya Ishteva Numerical Methods for the Best Low Multilinear Rank Approximation of Higher-Order Tensors (Numerieke methoden voor de beste lage multilineaire rang benadering van hogere-orde tensoren) , 2009 .

[2]  B. Nadler,et al.  Determining the number of components in a factor model from limited noisy data , 2008 .

[3]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[4]  Jan Niesing Simultaneous componenet and factor analysis methods for two or more groups: a comparative study. , 1997 .

[5]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[6]  H. Akaike A new look at the statistical model identification , 1974 .

[7]  Peter David Hoff,et al.  Equivariant and Scale-Free Tucker Decomposition Models , 2013, 1312.6397.

[8]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[9]  Martin Haardt,et al.  Multidimensional prewhitening for enhanced signal reconstruction and parameter estimation in colored noise with Kronecker correlation structure , 2013, Signal Process..

[10]  James P. Reilly,et al.  Detection of the number of signals: a predicted eigen-threshold approach , 1991, IEEE Trans. Signal Process..

[11]  Hsien-Tsai Wu,et al.  Source number estimators using transformed Gerschgorin radii , 1995, IEEE Trans. Signal Process..

[12]  Tom Minka,et al.  Automatic Choice of Dimensionality for PCA , 2000, NIPS.

[13]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[14]  Fan Yang,et al.  Tensor and its tucker core: The invariance relationships , 2016, Numer. Linear Algebra Appl..

[15]  F. L. Hitchcock The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[16]  G. Kitagawa,et al.  Generalised information criteria in model selection , 1996 .

[17]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[18]  R. Bro,et al.  A new efficient method for determining the number of components in PARAFAC models , 2003 .

[19]  Nadia Magnenat-Thalmann Welcome to the year 2016 , 2015, The Visual Computer.

[20]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[21]  Ilan Ziskind,et al.  Detection of the number of coherent signals by the MDL principle , 1989, IEEE Trans. Acoust. Speech Signal Process..

[22]  Florian Roemer,et al.  SEQUENTIAL GSVD BASED PREWHITENING FOR MULTIDIMENSIONAL HOSVD BASED SUBSPACE ESTIMATION , 2009 .

[23]  Pascal Larzabal,et al.  Some properties of ordered eigenvalues of a Wishart matrix: application in detection test and model order selection , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[24]  Soosan Beheshti,et al.  CANDECOMP/PARAFAC model order selection based on Reconstruction Error in the presence of Kronecker structured colored noise , 2016, Digit. Signal Process..

[25]  Sabine Van Huffel,et al.  Best Low Multilinear Rank Approximation of Higher-Order Tensors, Based on the Riemannian Trust-Region Scheme , 2011, SIAM J. Matrix Anal. Appl..

[26]  Renato Pajarola,et al.  Lossy volume compression using Tucker truncation and thresholding , 2016, The Visual Computer.

[27]  H. Kiers,et al.  Three-mode principal components analysis: choosing the numbers of components and sensitivity to local optima. , 2000, The British journal of mathematical and statistical psychology.

[28]  Thomas Kailath,et al.  Detection of signals by information theoretic criteria , 1985, IEEE Trans. Acoust. Speech Signal Process..

[29]  Andrzej Cichocki,et al.  Multilinear tensor rank estimation via Sparse Tucker Decomposition , 2014, 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems (SCIS) and 15th International Symposium on Advanced Intelligent Systems (ISIS).

[30]  Florian Roemer,et al.  Multi-dimensional model order selection , 2011, EURASIP J. Adv. Signal Process..

[31]  Jieping Ye,et al.  Detection of number of components in CANDECOMP/PARAFAC models via minimum description length , 2016, Digit. Signal Process..

[32]  R Bro,et al.  Cross-validation of component models: A critical look at current methods , 2008, Analytical and bioanalytical chemistry.

[33]  Kyuwan Choi,et al.  Detecting the Number of Clusters in n-Way Probabilistic Clustering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Qiang Wu,et al.  A parametric method for determining the number of signals in narrow-band direction finding , 1991, IEEE Trans. Signal Process..

[35]  Martin Haardt,et al.  Model Order Selection for Short Data: An Exponential Fitting Test (EFT) , 2007, EURASIP J. Adv. Signal Process..

[36]  Ryota Tomioka,et al.  Estimation of low-rank tensors via convex optimization , 2010, 1010.0789.

[37]  Jean Pierre Delmas,et al.  Blind channel approximation: effective channel order determination , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[38]  Anand Rangarajan,et al.  Image Denoising Using the Higher Order Singular Value Decomposition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.