Nonnegative approximations of nonnegative tensors

We study the decomposition of a nonnegative tensor into a minimal sum of outer product of nonnegative vectors and the associated parsimonious naïve Bayes probabilistic model. We show that the corresponding approximation problem, which is central to nonnegative PARAFAC, will always have optimal solutions. The result holds for any choice of norms and, under a mild assumption, even Brègman divergences. Copyright © 2009 John Wiley & Sons, Ltd.

[1]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[2]  P. Paatero.  Construction and analysis of degenerate PARAFAC models , 2000 .

[3]  J. Chisholm.  Approximation by Sequences of Padé Approximants in Regions of Meromorphy , 1966 .

[4]  P. Paatero.  A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[5]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[6]  R. Bro,et al.  A fast non‐negativity‐constrained least squares algorithm , 1997 .

[7]  R. A. Harshman,et al.  Data preprocessing and the extended PARAFAC model , 1984 .

[8]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[9]  Donald E. Knuth,et al.  The art of computer programming. Vol.2: Seminumerical algorithms , 1981 .

[10]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[11]  Alwin Stegeman,et al.  Low-Rank Approximation of Generic p˟q˟2 Arrays and Diverging Components in the Candecomp/Parafac Model , 2008, SIAM J. Matrix Anal. Appl..

[12]  Éric Gaussier,et al.  Relation between PLSA and NMF and implications , 2005, SIGIR '05.

[13]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[14]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[15]  Donald E. Knuth.  The Art of Computer Programming 2 / Seminumerical Algorithms , 1971 .

[16]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[17]  V. Strassen.  Gaussian elimination is not optimal , 1969 .

[18]  J. Carroll,et al.  Fitting of the Latent Class model via iteratively reweighted least squares CANDECOMP with nonnegativity constraints , 1989 .

[19]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[20]  Kyriakos Kalorkoti.  ALGEBRAIC COMPLEXITY THEORY (Grundlehren der Mathematischen Wissenschaften 315) , 1999 .

[21]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[22]  N. Sidiropoulos,et al.  Least squares algorithms under unimodality and non‐negativity constraints , 1998 .

[23]  Jan Brinkhuis,et al.  Optimization Insights and Applications , 2010 .

[24]  Grazia Lotti,et al.  O(n2.7799) Complexity for n*n Approximate Matrix Multiplication , 1979, Inf. Process. Lett..

[25]  Volker Strassen,et al.  Algebraic Complexity Theory , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[26]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[27]  Grazia Lotti,et al.  Approximate Solutions for the Bilinear Form Computational Problem , 1980, SIAM J. Comput..

[28]  I. M. Gelʹfand,et al.  Discriminants, Resultants, and Multidimensional Determinants , 1994 .

[29]  Tamir Hazan,et al.  Non-negative tensor factorization with applications to statistics and computer vision , 2005, ICML.

[30]  Bernd Sturmfels,et al.  Algebraic geometry of Bayesian networks , 2005, J. Symb. Comput..

[31]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[32]  P. Comon,et al.  Optimal solutions to non-negative PARAFAC/multilinear NMF always exist , 2005 .

[33]  Michael Greenacre,et al.  Multiway data analysis , 1992 .

[34]  Michael Clausen,et al.  Algebraic complexity theory , 1997, Grundlehren der mathematischen Wissenschaften.

[35]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[36]  Inderjit S. Dhillon,et al.  Matrix Nearness Problems with Bregman Divergences , 2007, SIAM J. Matrix Anal. Appl..

[37]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[38]  J. Kruskal,et al.  How 3-MFA data can cause degenerate parafac solutions, among other relationships , 1989 .

[39]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..