arXiv research

MARS automatically selects tensor decomposition ranks, improving performance in neural network tasks.

Unified algorithm for tensor decomposition supports multiple loss functions and models.

The paper shows how gradient flow on over-parametrized tensor decomposition behaves like deflation.

The report analyzes Legendre decomposition for tensor data.

Unified model for tensor completion using low-rank and sparse Tucker decomposition.

Tensor rank and low-rank tensor decompositions have many applications in learning and complexity theory. Most known algorithms use unfoldings of tensors and can only handle rank up to $n^{\lfloor p/2 \rfloor}$ for a $p$-th order tensor in $\mathbb{R}^{n^p}$. Previously no efficient algorithm can decompose 3rd order ten…

In tensor completion tasks, the traditional low-rank tensor decomposition models suffer from the laborious model selection problem due to their high model sensitivity. In particular, for tensor ring (TR) decomposition, the number of model possibilities grows exponentially with the tensor order, which makes it rather ch…

In this paper, we provide local and global convergence guarantees for recovering CP (Candecomp/Parafac) tensor decomposition. The main step of the proposed algorithm is a simple alternating rank-$1$ update which is the alternating version of the tensor power iteration adapted for asymmetric tensors. Local convergence g…

Gradient descent can find better tensor decompositions than lazy training in over-parameterized settings.

Often, large, high dimensional datasets collected across multiple modalities can be organized as a higher order tensor. Low-rank tensor decomposition then arises as a powerful and widely used tool to discover simple low dimensional structures underlying such data. However, we currently lack a theoretical understanding …

DeepTensor uses deep networks to efficiently decompose tensors with improved performance and robustness.

Revisits CP tensor decomposition for noisy, non-orthogonal data.

Large CNNs have delivered impressive performance in various computer vision applications. But the storage and computation requirements make it problematic for deploying these models on mobile devices. Recently, tensor decompositions have been used for speeding up CNNs. In this paper, we further develop the tensor decom…

Robust tensor CP decomposition involves decomposing a tensor into low rank and sparse components. We propose a novel non-convex iterative algorithm with guaranteed recovery. It alternates between low-rank CP decomposition through gradient ascent (a variant of the tensor power method), and hard thresholding of the resid…

Tensor decomposition is an important technique for capturing the high-order interactions among multiway data. Multi-linear tensor composition methods, such as the Tucker decomposition and the CANDECOMP/PARAFAC (CP), assume that the complex interactions among objects are multi-linear, and are thus insufficient to repres…

NA$_0$CT$^2$ improves tensor regression predictions with $\ell_0$ regularization.

We discuss structured Schatten norms for tensor decomposition that includes two recently proposed norms ("overlapped" and "latent") for convex-optimization-based tensor decomposition, and connect tensor decomposition with wider literature on structured sparsity. Based on the properties of the structured Schatten norms,…

Efficient tensor decomposition for count data models achieves near-optimal multiway analysis.

A new method for decomposing non-negative tensors using energy-based modeling.

Proposes BHT-ARIMA for forecasting multiple short time series.

Paper improves tensor completion by reducing sample entries needed.

Unified framework for non-Euclidean CPD under scalable stochastic mirror descent.

Tensorized random projections reduce high-dimensional tensor size efficiently.

FRAPPE estimates tensor canonical rank without CPD computation.

TGCCA analyzes higher-order tensors using orthogonal rank-R CP decomposition.

We introduce a new parameterization method for deep learning layers using spectral tensor train decomposition.

We propose a new framework for the analysis of low-rank tensors which lies at the intersection of spectral graph theory and signal processing. As a first step, we present a new graph based low-rank decomposition which approximates the classical low-rank SVD for matrices and multi-linear SVD for tensors. Then, building …

A low-rank tensor model simplifies multi-dimensional Markov chains.

The PARAFAC tensor decomposition has enjoyed an increasing success in exploratory multi-aspect data mining scenarios. A major challenge remains the estimation of the number of latent factors (i.e., the rank) of the decomposition, which yields high-quality, interpretable results. Previously, we have proposed an automate…

The paper improves density estimation in high dimensions using tensor decompositions.

A new framework improves tensor completion accuracy by considering numerical priors.

Low rank tensor decompositions are a powerful tool for learning generative models, and uniqueness results give them a significant advantage over matrix decomposition methods. However, tensors pose significant algorithmic challenges and tensors analogs of much of the matrix algebra toolkit are unlikely to exist because …

A new method for traffic data imputation considering spatiotemporal correlations.

Low-rank tensor decomposition and completion have attracted significant interest from academia given the ubiquity of tensor data. However, the low-rank structure is a global property, which will not be fulfilled when the data presents complex and weak dependencies given specific graph structures. One particular applica…

This paper studies how key tensor properties are inherited in subtensors of tensor train decompositions.

Adaptive algorithm learns tensor network structures from data.

GETF efficiently decomposes large-scale Boolean tensors.

New method models matrix time series using tensor CP-decomposition.

Four algorithms improve sparse tensor BR1Approx with theoretical guarantees.

Tensor completion estimates missing components by exploiting the low-rank structure of multi-way data. The recently proposed methods based on tensor train (TT) and tensor ring (TR) show better performance in image recovery than classical ones. Compared with TT and TR, the projected entangled pair state (PEPS), which is…

In this paper, we study a polynomial decomposition model that arises in problems of system identification, signal processing and machine learning. We show that this decomposition is a special case of the X-rank decomposition --- a powerful novel concept in algebraic geometry that generalizes the tensor CP decomposition…

HOTCAKE compresses CNNs by decomposing kernels into smaller parts.

In this paper we focus on the problem of completion of multidimensional arrays (also referred to as tensors) from limited sampling. Our approach is based on a recently proposed tensor-Singular Value Decomposition (t-SVD) [1]. Using this factorization one can derive notion of tensor rank, referred to as the tensor tubal…

We consider the problem of online subspace tracking of a partially observed high-dimensional data stream corrupted by noise, where we assume that the data lie in a low-dimensional linear subspace. This problem is cast as an online low-rank tensor completion problem. We propose a novel online tensor subspace tracking al…

Optimizes neural network training by dynamically updating Tucker decomposition ranks.

A new probabilistic BTD method for tensor data.

New algorithm recovers tensor factors from incomplete measurements efficiently.

In this paper, we analyze the fundamental conditions for low-rank tensor completion given the separation or tensor-train (TT) rank, i.e., ranks of unfoldings. We exploit the algebraic structure of the TT decomposition to obtain the deterministic necessary and sufficient conditions on the locations of the samples to ens…