Partitioned Alternating Least Squares Technique for Canonical Polyadic Tensor Decomposition

Canonical polyadic decomposition (CPD), also known as parallel factor analysis, is a representation of a given tensor as a sum of rank-one components. Traditional method for accomplishing CPD is the alternating least squares (ALS) algorithm. Convergence of ALS is known to be slow, especially when some factor matrices of the tensor contain nearly collinear columns. We propose a novel variant of this technique, in which the factor matrices are partitioned into blocks, and each iteration jointly updates blocks of different factor matrices. Each partial optimization is quadratic and can be done in closed form. The algorithm alternates between different random partitionings of the matrices. As a result, a faster convergence is achieved. Another improvement can be obtained when the method is combined with the enhanced line search of Rajih et al. Complexity per iteration is between those of the ALS and the Levenberg-Marquardt (damped Gauss-Newton) method. It is important, however, that the idea of alternating quadratic optimization with partitioned factor matrices is general and can be applied to other variants of the tensor decomposition problems, e.g., when non-Gaussian additive noise is considered.