Smooth PARAFAC Decomposition for Tensor Completion

In recent years, low-rank based tensor completion, which is a higher order extension of matrix completion, has received considerable attention. However, the low-rank assumption is not sufficient for the recovery of visual data, such as color and 3D images, when the ratio of missing data is extremely high. In this paper, we consider “smoothness” constraints as well as low-rank approximations and propose an efficient algorithm for performing tensor completion that is particularly powerful regarding visual data. The proposed method admits significant advantages, owing to the integration of smooth PARAFAC decomposition for incomplete tensors and the efficient selection of models in order to minimize the tensor rank. Thus, our proposed method is termed as “smooth PARAFAC tensor completion (SPC).” In order to impose the smoothness constraints, we employ two strategies, total variation (SPC-TV) and quadratic variation (SPC-QV), and invoke the corresponding algorithms for model learning. Extensive experimental evaluations on both synthetic and real-world visual data illustrate the significant improvements of our method, in terms of both prediction performance and efficiency, compared with many state-of-the-art tensor completion methods.

[1]  Norman L. Owsley,et al.  Adaptive data orthogonalization , 1978, ICASSP.

[2]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[3]  Ivan W. Selesnick,et al.  Total variation denoising (an MM algorithm) , 2012 .

[4]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[5]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[6]  Lieven De Lathauwer,et al.  Optimization-Based Algorithms for Tensor Decompositions: Canonical Polyadic Decomposition, Decomposition in Rank-(Lr, Lr, 1) Terms, and a New Generalization , 2013, SIAM J. Optim..

[7]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[8]  Bamdev Mishra,et al.  Low-Rank Optimization with Trace Norm Penalty , 2011, SIAM J. Optim..

[9]  Parikshit Shah,et al.  Optimal Low-Rank Tensor Recovery from Separable Measurements: Four Contractions Suffice , 2015, ArXiv.

[10]  Bart Vandereycken,et al.  Low-rank tensor completion by Riemannian optimization , 2014 .

[11]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[12]  Shmuel Friedland,et al.  Nuclear norm of higher-order tensors , 2014, Math. Comput..

[13]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[14]  Yin Li,et al.  Hole-Filling by Rank Sparsity Tensor Decomposition for Medical Imaging , 2010, 2010 International Conference on E-Product E-Service and E-Entertainment.

[15]  L. Senhadji,et al.  Linear Total Variation Approximate Regularized Nuclear Norm Optimization for Matrix Completion , 2014 .

[16]  Wei Li,et al.  Non‐Local Image Inpainting Using Low‐Rank Matrix Completion , 2014, Comput. Graph. Forum.

[17]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Xiaodong Li,et al.  Compressed Sensing and Matrix Completion with Constant Proportion of Corruptions , 2011, ArXiv.

[19]  Zhi-Hua Zhou,et al.  Face Image Modeling by Multilinear Subspace Analysis With Missing Values , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  R. Courant Methods of mathematical physics, Volume I , 1965 .

[21]  Andrzej Cichocki,et al.  B-Spline Smoothing of Feature Vectors in Nonnegative Matrix Factorization , 2014, ICAISC.

[22]  Bin Ran,et al.  Tensor completion via a multi-linear low-n-rank factorization model , 2014, Neurocomputing.

[23]  R. Tibshirani,et al.  A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. , 2009, Biostatistics.

[24]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[25]  Lars Karlsson,et al.  Parallel algorithms for tensor completion in the CP format , 2016, Parallel Comput..

[26]  Genevera I. Allen Multi-way functional principal components analysis , 2013, 2013 5th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[27]  Tamara G. Kolda,et al.  Scalable Tensor Factorizations for Incomplete Data , 2010, ArXiv.

[28]  David B. Dunson,et al.  Scalable Bayesian Low-Rank Decomposition of Incomplete Multiway Tensors , 2014, ICML.

[29]  Dimitri P. Bertsekas,et al.  Convex Optimization Algorithms , 2015 .

[30]  Slim Essid,et al.  Smooth Nonnegative Matrix Factorization for Unsupervised Audiovisual Document Structuring , 2013, IEEE Transactions on Multimedia.

[31]  Xiaoming Yuan,et al.  Matrix completion via an alternating direction method , 2012 .

[32]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[33]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[34]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[35]  Andrzej Cichocki,et al.  A fast automatic low-rank determination algorithm for noisy matrix completion , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[36]  Russ B. Altman,et al.  Missing value estimation methods for DNA microarrays , 2001, Bioinform..

[37]  Baoxin Li,et al.  Tensor completion for on-board compression of hyperspectral images , 2010, 2010 IEEE International Conference on Image Processing.

[38]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[39]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[40]  Inderjit S. Dhillon,et al.  Parallel matrix factorization for recommender systems , 2014, Knowl. Inf. Syst..

[41]  Hong-Yuan Mark Liao,et al.  Simultaneous Tensor Decomposition and Completion Using Factor Priors , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Xiangjian He,et al.  Tensor error correction for corrupted values in visual data , 2010, 2010 IEEE International Conference on Image Processing.

[43]  Danny C. Sorensen,et al.  Minimization of a Large-Scale Quadratic FunctionSubject to a Spherical Constraint , 1997, SIAM J. Optim..

[44]  H. Kiers Towards a standardized notation and terminology in multiway analysis , 2000 .

[45]  Michael Elad,et al.  Image Processing Using Smooth Ordering of its Patches , 2012, IEEE Transactions on Image Processing.

[46]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[47]  Andrzej Cichocki,et al.  Constrained non-Negative Matrix Factorization Method for EEG Analysis in Early Detection of Alzheimer Disease , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[48]  Jian Liu,et al.  Visual saliency detection via rank-sparsity decomposition , 2010, 2010 IEEE International Conference on Image Processing.

[49]  Rafal Zdunek,et al.  Approximation of Feature Vectors in Nonnegative Matrix Factorization with Gaussian Radial Basis Functions , 2012, ICONIP.

[50]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[51]  Qi Dai,et al.  The Physics of Compressive Sensing and the Gradient-Based Recovery Algorithms , 2009, ArXiv.

[52]  Zuowei Shen,et al.  Robust video denoising using low rank matrix completion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[53]  Ivor W. Tsang,et al.  Riemannian Pursuit for Big Matrix Recovery , 2014, ICML.

[54]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Dr. M. G. Worster Methods of Mathematical Physics , 1947, Nature.

[56]  Sun-Yuan Kung,et al.  On gradient adaptation with unit-norm constraints , 2000, IEEE Trans. Signal Process..

[57]  Shimon Ullman,et al.  Uncovering shared structures in multiclass classification , 2007, ICML '07.

[58]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[59]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[60]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[61]  Ming Yuan,et al.  On Tensor Completion via Nuclear Norm Minimization , 2014, Found. Comput. Math..

[62]  Yukihiko Yamashita,et al.  Smooth nonnegative matrix and tensor factorizations for robust multi-way data analysis , 2015, Signal Process..

[63]  Liqi Wang,et al.  On the Global Convergence of the Alternating Least Squares Method for Rank-One Approximation to Generic Tensors , 2014, SIAM J. Matrix Anal. Appl..

[64]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[65]  S. V. Dolgov,et al.  ALTERNATING MINIMAL ENERGY METHODS FOR LINEAR SYSTEMS IN HIGHER DIMENSIONS∗ , 2014 .

[66]  José M. Bioucas-Dias,et al.  Adaptive total variation image deblurring: A majorization-minimization approach , 2009, Signal Process..

[67]  Nicolas Gillis,et al.  Low-Rank Matrix Approximation with Weights or Missing Data Is NP-Hard , 2010, SIAM J. Matrix Anal. Appl..

[68]  Yin Li,et al.  Optimum Subspace Learning and Error Correction for Tensors , 2010, ECCV.

[69]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[70]  A. Uschmajew,et al.  A new convergence proof for the higher-order power method and generalizations , 2014, 1407.4586.

[71]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[72]  F. L. Hitchcock The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[73]  Márcia M. C. Ferreira,et al.  PARAFAC with splines: a case study , 2002 .

[74]  Truong Q. Nguyen,et al.  An Augmented Lagrangian Method for Total Variation Video Restoration , 2011, IEEE Transactions on Image Processing.

[75]  Lieven De Lathauwer,et al.  Structured Data Fusion , 2015, IEEE Journal of Selected Topics in Signal Processing.

[76]  Wotao Yin,et al.  Second-order Cone Programming Methods for Total Variation-Based Image Restoration , 2005, SIAM J. Sci. Comput..

[77]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[78]  André Uschmajew,et al.  Local Convergence of the Alternating Least Squares Algorithm for Canonical Tensor Approximation , 2012, SIAM J. Matrix Anal. Appl..

[79]  Massimiliano Pontil,et al.  Multi-Task Feature Learning , 2006, NIPS.

[80]  Bart Vandereycken,et al.  Greedy rank updates combined with Riemannian descent methods for low-rank optimization , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[81]  Andrzej Cichocki,et al.  Hierarchical ALS Algorithms for Nonnegative Matrix and 3D Tensor Factorization , 2007, ICA.

[82]  Andrzej Cichocki,et al.  Blind Image Separation Using Nonnegative Matrix Factorization with Gibbs Smoothing , 2007, ICONIP.