Matrix and tensor completion using tensor ring decomposition with sparse representation

Completing a data tensor with structured missing components is a challenging task where the missing components are not distributed randomly but they admit some regular patterns, e.g. missing columns and rows or missing blocks/patches. Many of the existing tensor completion algorithms are not able to handle such scenarios. In this paper, we propose a novel and efficient approach for matrix/tensor completion by applying Hankelization and distributed tensor ring decomposition. Our main idea is first Hankelizing an incomplete data tensor in order to obtain high-order tensors and then completing the data tensor by imposing sparse representation on the core tensors in tensor ring format. We apply an efficient over-complete discrete cosine transform dictionary and sparse representation techniques to learn core tensors. Alternating direction methods of multiplier and accelerated proximal gradient approaches are used to solve the underlying optimization problems. Extensive simulations performed on image, video completions and time series forecasting show the validity and applicability of the method for different kinds of structured and random missing elements.

[1]  Hidekata Hontani,et al.  Manifold Modeling in Embedded Space: An Interpretable Alternative to Deep Image Prior , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Xi-Le Zhao,et al.  Tensor completion via nonconvex tensor ring rank minimization with guaranteed convergence , 2020, Signal Process..

[3]  Naoto Yokoya,et al.  Hyperspectral Super-Resolution via Coupled Tensor Ring Factorization , 2020, Pattern Recognit..

[4]  Anh Huy Phan,et al.  Randomized algorithms for fast computation of low rank tensor ring model , 2020, Mach. Learn. Sci. Technol..

[5]  Juan Carrasquilla,et al.  Quantum process tomography with unsupervised learning and tensor networks , 2020, Nature communications.

[6]  Qiquan Shi,et al.  Matrix and Tensor Completion in Multiway Delay Embedded Space Using Tensor Train, With Application to Signal Reconstruction , 2020, IEEE Signal Processing Letters.

[7]  Ce Zhu,et al.  Robust Low-Rank Tensor Ring Completion , 2020, IEEE Transactions on Computational Imaging.

[8]  Justin Reyes,et al.  A Multi-Scale Tensor Network Architecture for Classification and Regression , 2020, ArXiv.

[9]  J. Biamonte,et al.  Quantum Machine Learning Tensor Network States , 2018, Frontiers in Physics.

[10]  Longhao Yuan,et al.  Rank minimization on tensor ring: an efficient approach for tensor decomposition and completion , 2019, Machine Learning.

[11]  Dmitry Vetrov,et al.  A Prior of a Googol Gaussians: a Tensor Ring Induced Prior for Generative Models , 2019, NeurIPS.

[12]  Xi-Le Zhao,et al.  Low-Rank Tensor Completion via Tensor Nuclear Norm With Hybrid Smooth Regularization , 2019, IEEE Access.

[13]  Jianting Cao,et al.  High-order tensor completion via gradient-based optimization under tensor train format , 2019, Signal Process. Image Commun..

[14]  Shutao Li,et al.  Learning a Low Tensor-Train Rank Representation for Hyperspectral Image Super-Resolution , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Zenglin Xu,et al.  Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition , 2018, AAAI.

[16]  Chao Li,et al.  Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion , 2018, AAAI.

[17]  Ce Zhu,et al.  Low Rank Tensor Completion for Multiway Visual Data , 2018, Signal Process..

[18]  James Caverlee,et al.  Tensor Completion Algorithms in Big Data Analytics , 2017, ACM Trans. Knowl. Discov. Data.

[19]  Masashi Sugiyama,et al.  Learning Efficient Tensor Representations with Ring-structured Networks , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  Kun Li,et al.  Tensor Completion From Structurally-Missing Entries by Low-TT-Rankness and Fiber-Wise Sparsity , 2018, IEEE Journal of Selected Topics in Signal Processing.

[21]  Jianting Cao,et al.  Higher-dimension Tensor Completion via Low-rank Tensor Ring Decomposition , 2018, 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[22]  Ming Yang,et al.  Image Denoising via Improved Dictionary Learning with Global Structure and Local Similarity Preservations , 2018, Symmetry.

[23]  Hidekata Hontani,et al.  Missing Slice Recovery for Tensors Using a Low-Rank Model in Embedded Space , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  J. Biamonte,et al.  Tensor Networks in a Nutshell , 2017, 1708.00006.

[25]  V. Aggarwal,et al.  Efficient Low Rank Tensor Ring Completion , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Volker Tresp,et al.  Tensor-Train Recurrent Neural Networks for Video Classification , 2017, ICML.

[27]  Hidekata Hontani,et al.  Simultaneous Visual Data Completion and Denoising Based on Tensor Rank and Total Variation Minimization and Its Primal-Dual Splitting Algorithm , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Masashi Sugiyama,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives , 2017, Found. Trends Mach. Learn..

[29]  Satoshi Nakamura,et al.  Compressing recurrent neural network with tensor train , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[30]  Minh N. Do,et al.  Efficient Tensor Completion for Color Image and Video Recovery: Low-Rank Tensor Train , 2016, IEEE Transactions on Image Processing.

[31]  Andrzej Cichocki,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions , 2016, Found. Trends Mach. Learn..

[32]  Liqing Zhang,et al.  Tensor Ring Decomposition , 2016, ArXiv.

[33]  Christopher T. Chubb,et al.  Hand-waving and interpretive dance: an introductory course on tensor networks , 2016, 1603.03039.

[34]  Muhammad Tayyab Asif,et al.  Matrix and Tensor Based Methods for Missing Data Estimation in Large Traffic Networks , 2016, IEEE Transactions on Intelligent Transportation Systems.

[35]  Andrzej Cichocki,et al.  Smooth PARAFAC Decomposition for Tensor Completion , 2015, IEEE Transactions on Signal Processing.

[36]  David J. Schwab,et al.  Supervised Learning with Tensor Networks , 2016, NIPS.

[37]  Alexander Novikov,et al.  Tensorizing Neural Networks , 2015, NIPS.

[38]  Wen Gao,et al.  Group-Based Sparse Representation for Image Restoration , 2014, IEEE Transactions on Image Processing.

[39]  Andrzej Cichocki,et al.  Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions , 2014, ArXiv.

[40]  Roman Orus,et al.  A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States , 2013, 1306.2164.

[41]  Yuxin Chen,et al.  Spectral Compressed Sensing via Structured Matrix Completion , 2013, ICML.

[42]  Kishore Kumar Naraparaju,et al.  A note on tensor chain approximation , 2012, Comput. Vis. Sci..

[43]  Reinhold Schneider,et al.  Optimization problems in contracted tensor networks , 2011, Comput. Vis. Sci..

[44]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[45]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[46]  B. Khoromskij O(dlog N)-Quantics Approximation of N-d Tensors in High-Dimensional Numerical Modeling , 2011 .

[47]  U. Schollwoeck The density-matrix renormalization group in the age of matrix product states , 2010, 1008.3477.

[48]  Tamara G. Kolda,et al.  Scalable Tensor Factorizations for Incomplete Data , 2010, ArXiv.

[49]  Tamara G. Kolda,et al.  Poblano v1.0: A Matlab Toolbox for Gradient-Based Optimization , 2010 .

[50]  Hans Gerd Evertz,et al.  Efficient matrix-product state method for periodic boundary conditions , 2008, 0801.1947.

[51]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[53]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[54]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[55]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[56]  White,et al.  Density-matrix algorithms for quantum renormalization groups. , 1993, Physical review. B, Condensed matter.

[57]  White,et al.  Density matrix formulation for quantum renormalization groups. , 1992, Physical review letters.

[58]  Richard Bellman,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.