Computation of low-rank tensor approximation under existence constraint via a forward-backward algorithm

Abstract The Canonical Polyadic (CP) tensor decomposition has become an attractive mathematical tool in several fields during the last ten years. This decomposition is very powerful for representing and analyzing multidimensional data. The most attractive feature of the CP decomposition is its uniqueness, contrary to rank-revealing matrix decompositions, where the problem of rotational invariance remains. This paper presents the performance analysis of iterative descent algorithms for calculating the CP decomposition of tensors when columns of factor matrices are almost collinear – i.e. swamp problems arise. We propose in this paper a new and efficient proximal algorithm based on the Forward Backward splitting method. More precisely, the existence of the best low-rank tensor approximation is ensured thanks to a coherence constraint implemented via a logarithmic regularized barrier. Computer experiments demonstrate the efficiency and stability of the proposed algorithm in comparison to other iterative algorithms in the literature for the normal case, and also producing significant results even in difficult situations.

[1]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[2]  Pierre Comon,et al.  Coherence Constrained Alternating Least Squares , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[3]  Andrzej Cichocki,et al.  Partitioned Alternating Least Squares Technique for Canonical Polyadic Tensor Decomposition , 2016, IEEE Signal Processing Letters.

[4]  Mohamed-Jalal Fadili,et al.  Sparse Image and Signal Processing: Wavelets, Curvelets, Morphological Diversity, by Jean-Luc Starck, Fionn Murtagh, and Jalal M. Fadili , 2010, J. Electronic Imaging.

[5]  A. Stegeman,et al.  On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition , 2007 .

[6]  Tamara G. Kolda,et al.  Orthogonal Tensor Decompositions , 2000, SIAM J. Matrix Anal. Appl..

[7]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[8]  A. Geramita,et al.  Ranks of tensors, secant varieties of Segre varieties and fat points , 2002 .

[9]  Pierre Comon,et al.  Tensor Decompositions, State of the Art and Applications , 2002 .

[10]  Rasmus Bro,et al.  Improving the speed of multi-way algorithms:: Part I. Tucker3 , 1998 .

[11]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[12]  B. Kowalski,et al.  Tensorial resolution: A direct trilinear decomposition , 1990 .

[13]  Pinar Çivicioglu,et al.  Backtracking Search Optimization Algorithm for numerical optimization problems , 2013, Appl. Math. Comput..

[14]  Souleymen Sahnoun,et al.  Joint Source Estimation and Localization , 2015, IEEE Transactions on Signal Processing.

[15]  Andrzej Cichocki,et al.  Partitioned Hierarchical alternating least squares algorithm for CP tensor decomposition , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Nikos D. Sidiropoulos,et al.  Blind PARAFAC receivers for DS-CDMA systems , 2000, IEEE Trans. Signal Process..

[17]  J. Kruskal,et al.  How 3-MFA data can cause degenerate parafac solutions, among other relationships , 1989 .

[18]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[19]  Huan Li,et al.  Accelerated Proximal Gradient Methods for Nonconvex Programming , 2015, NIPS.

[20]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[21]  Pierre Comon,et al.  Blind Multilinear Identification , 2012, IEEE Transactions on Information Theory.

[22]  C. G. Bollini,et al.  On the Reduction Formula of Feinberg and Pais , 1965 .

[23]  M. Fukushima,et al.  A generalized proximal point algorithm for certain non-convex minimization problems , 1981 .

[24]  Richard A. Harshman,et al.  Determination and Proof of Minimum Uniqueness Conditions for PARAFAC1 , 1972 .

[25]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[26]  Na Li,et al.  Some Convergence Results on the Regularized Alternating Least-Squares Method for Tensor Decomposition , 2011, 1109.3831.

[27]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[28]  E. Lee,et al.  APPLYING GRADIENT PROJECTION AND CONJUGATE GRADIENT TO THE OPTIMUM OPERATION OF RESERVOIRS1 , 1970 .

[29]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[30]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[31]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[32]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[33]  Khalid Minaoui,et al.  A progression strategy of proximal algorithm for the unconstrained optimization , 2018, 2018 4th International Conference on Optimization and Applications (ICOA).

[34]  R. Bro,et al.  Fluorescence spectroscopy and multi-way techniques. PARAFAC , 2013 .

[35]  Lieven De Lathauwer,et al.  On the Uniqueness of the Canonical Polyadic Decomposition of Third-Order Tensors - Part I: Basic Results and Uniqueness of One Factor Matrix , 2013, SIAM J. Matrix Anal. Appl..

[36]  Pierre Comon,et al.  Using the proximal gradient and the accelerated proximal gradient as a canonical polyadic tensor decomposition algorithms in difficult situations , 2020, Signal Process..

[37]  Caroline Fossati,et al.  Denoising of Hyperspectral Images Using the PARAFAC Model and Statistical Performance Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[38]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[39]  P. Paatero Construction and analysis of degenerate PARAFAC models , 2000 .

[40]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[41]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[42]  F. L. Hitchcock The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[43]  Pierre Comon,et al.  ROBUST INDEPENDENT COMPONENT ANALYSIS , 2009 .

[44]  Pierre Comon,et al.  Robust Independent Component Analysis by Iterative Maximization of the Kurtosis Contrast With Algebraic Optimal Step Size , 2010, IEEE Transactions on Neural Networks.

[45]  P. Paatero A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[46]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[47]  Rasmus Bro,et al.  Improving the speed of multiway algorithms: Part II: Compression , 1998 .

[48]  Andrzej Cichocki,et al.  Low Complexity Damped Gauss-Newton Algorithms for CANDECOMP/PARAFAC , 2012, SIAM J. Matrix Anal. Appl..

[49]  H. Kiers Towards a standardized notation and terminology in multiway analysis , 2000 .

[50]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[51]  Jacek Gondzio,et al.  Interior point methods 25 years later , 2012, Eur. J. Oper. Res..

[52]  W. Rayens,et al.  Two-factor degeneracies and a stabilization of PARAFAC , 1997 .

[53]  Bijan Afsari,et al.  Sensitivity Analysis for the Problem of Matrix Joint Diagonalization , 2008, SIAM J. Matrix Anal. Appl..

[54]  Driss Aboutajdine,et al.  CP decomposition approach to blind separation for DS-CDMA system using a new performance index , 2014, EURASIP J. Adv. Signal Process..

[55]  Nico Vervliet,et al.  Nonconvex Optimization Tools for Large-Scale Matrix and Tensor Decomposition with Structured Factors , 2020, ArXiv.

[56]  Ben C. Mitchell,et al.  Slowly converging parafac sequences: Swamps and two‐factor degeneracies , 1994 .

[57]  Pierre Comon,et al.  Tensors : A brief introduction , 2014, IEEE Signal Processing Magazine.

[58]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[59]  S. Leurgans,et al.  A Decomposition for Three-Way Arrays , 1993, SIAM J. Matrix Anal. Appl..

[60]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .