arXiv research

Unified algorithm for tensor decomposition supports multiple loss functions and models.

The paper shows how gradient flow on over-parametrized tensor decomposition behaves like deflation.

The report analyzes Legendre decomposition for tensor data.

Unified model for tensor completion using low-rank and sparse Tucker decomposition.

Often, large, high dimensional datasets collected across multiple modalities can be organized as a higher order tensor. Low-rank tensor decomposition then arises as a powerful and widely used tool to discover simple low dimensional structures underlying such data. However, we currently lack a theoretical understanding …

Large CNNs have delivered impressive performance in various computer vision applications. But the storage and computation requirements make it problematic for deploying these models on mobile devices. Recently, tensor decompositions have been used for speeding up CNNs. In this paper, we further develop the tensor decom…

DeepTensor uses deep networks to efficiently decompose tensors with improved performance and robustness.

Robust tensor CP decomposition involves decomposing a tensor into low rank and sparse components. We propose a novel non-convex iterative algorithm with guaranteed recovery. It alternates between low-rank CP decomposition through gradient ascent (a variant of the tensor power method), and hard thresholding of the resid…

A low-rank tensor model simplifies multi-dimensional Markov chains.

Proposes BHT-ARIMA for forecasting multiple short time series.

We propose a new framework for the analysis of low-rank tensors which lies at the intersection of spectral graph theory and signal processing. As a first step, we present a new graph based low-rank decomposition which approximates the classical low-rank SVD for matrices and multi-linear SVD for tensors. Then, building …

In tensor completion tasks, the traditional low-rank tensor decomposition models suffer from the laborious model selection problem due to their high model sensitivity. In particular, for tensor ring (TR) decomposition, the number of model possibilities grows exponentially with the tensor order, which makes it rather ch…

Unified framework for non-Euclidean CPD under scalable stochastic mirror descent.

Tensorized random projections reduce high-dimensional tensor size efficiently.

Low-rank tensor decomposition and completion have attracted significant interest from academia given the ubiquity of tensor data. However, the low-rank structure is a global property, which will not be fulfilled when the data presents complex and weak dependencies given specific graph structures. One particular applica…

A new framework improves tensor completion accuracy by considering numerical priors.

NA$_0$CT$^2$ improves tensor regression predictions with $\ell_0$ regularization.

Tensor completion estimates missing components by exploiting the low-rank structure of multi-way data. The recently proposed methods based on tensor train (TT) and tensor ring (TR) show better performance in image recovery than classical ones. Compared with TT and TR, the projected entangled pair state (PEPS), which is…

Tensor rank and low-rank tensor decompositions have many applications in learning and complexity theory. Most known algorithms use unfoldings of tensors and can only handle rank up to $n^{\lfloor p/2 \rfloor}$ for a $p$-th order tensor in $\mathbb{R}^{n^p}$. Previously no efficient algorithm can decompose 3rd order ten…

New algorithm recovers tensor factors from incomplete measurements efficiently.

We consider the problem of online subspace tracking of a partially observed high-dimensional data stream corrupted by noise, where we assume that the data lie in a low-dimensional linear subspace. This problem is cast as an online low-rank tensor completion problem. We propose a novel online tensor subspace tracking al…

We introduce a new parameterization method for deep learning layers using spectral tensor train decomposition.

We investigate the sample size requirement for exact recovery of a high order tensor of low rank from a subset of its entries. In the Tucker decomposition framework, we show that the Riemannian optimization algorithm with initial value obtained from a spectral method can reconstruct a tensor of size $n\times n \times\c…

A new method for decomposing non-negative tensors using energy-based modeling.

We study the problem of learning a distribution from samples, when the underlying distribution is a mixture of product distributions over discrete domains. This problem is motivated by several practical applications such as crowd-sourcing, recommendation systems, and learning Boolean functions. The existing solutions e…

Tensorized Rademacher projections outperform Gaussian projections in reducing tensor dimensions.

New method uses tensor decompositions to overcome the curse of dimensionality for large-scale learning.

ScaledGD algorithm estimates low-rank tensors efficiently from corrupted data.

Bayesian model improves image completion accuracy by automatically learning low rank structure.

This work improves fair tensor decomposition using a kernel criterion.

Low-rank tensor completion recovers missing entries based on different tensor decompositions. Due to its outstanding performance in exploiting some higher-order data structure, low rank tensor ring has been applied in tensor completion. To further deal with its sensitivity to sparse component as it does in tensor princ…

This paper reviews methods for discovering patient subgroups from EHR data.

New method for tensor completion from specific mode observations.

New method selects features via tensor decomposition and submodular optimization.

We discuss structured Schatten norms for tensor decomposition that includes two recently proposed norms ("overlapped" and "latent") for convex-optimization-based tensor decomposition, and connect tensor decomposition with wider literature on structured sparsity. Based on the properties of the structured Schatten norms,…

Optimizes tensor completion using geodesics on Segre manifolds.

Method estimates joint probability density from samples using low-rank decomposition and random projections.

Estimates joint probability distribution from 1-way marginals using low-rank tensors and random projections.

Paper proposes a novel MTL framework for personalized modeling of diverse individuals.

Tensors play a central role in many modern machine learning and signal processing applications. In such applications, the target tensor is usually of low rank, i.e., can be expressed as a sum of a small number of rank one tensors. This motivates us to consider the problem of low rank tensor recovery from a class of lin…

TensorGuide improves LoRA efficiency and expressivity through joint tensor-train optimization.

Tensor decomposition on big data has attracted significant attention recently. Among the most popular methods is a class of algorithms that leverages compression in order to reduce the size of the tensor and potentially parallelize computations. A fundamental requirement for such methods to work properly is that the lo…

Gradient descent can find better tensor decompositions than lazy training in over-parameterized settings.

New tensor model reduces GLM estimation error and sample complexity.

This paper introduces a new multivariate convolutional sparse coding based on tensor algebra with a general model enforcing both element-wise sparsity and low-rankness of the activations tensors. By using the CP decomposition, this model achieves a significantly more efficient encoding of the multivariate signal-partic…

Develops SymGCP for tensor decompositions with general symmetry.

In this paper, we propose a general framework for sparse and low-rank tensor estimation from cubic sketchings. A two-stage non-convex implementation is developed based on sparse tensor decomposition and thresholded gradient descent, which ensures exact recovery in the noiseless case and stable recovery in the noisy cas…

Robust tensor recovery plays an instrumental role in robustifying tensor decompositions for multilinear data analysis against outliers, gross corruptions and missing values and has a diverse array of applications. In this paper, we study the problem of robust low-rank tensor recovery in a convex optimization framework,…