STAT-SVD method reduces high-dimensional data sparsity, achieving optimal estimation.
problem Sparse tensor singular value decomposition for high-dimensional data.
method STAT-SVD method with double projection & thresholding scheme.
result STAT-SVD provides sharp thresholding criterion and minimax rate-optimal estimation.
Paper tackles tensor completion from sparse corrupted data using convex optimization.
problem Estimating multidimensional arrays from a subset of corrupted entries.
method Solves a convex program that minimizes a weighted combination of tubal nuclear norm and ℓ1-norm. result Exact recovery of incoherent tensors with overwhelming probability.
Paper proposes a new method for exact recovery in robust tensor principal component analysis.
problem Exact recovery of low-rank and sparse components in tensors.
method Proposes a new method based on tensor-tensor product and t-SVD to solve a convex optimization problem.
result Exact recovery achieved in a deterministic fashion without randomness assumptions.
New group-sparse SVD models improve biclustering of gene expression data.
problem Identifying block patterns with similar expressions in high-dimensional gene expression data.
method Proposed GL1-SVD, GL0-SVD, OGL1-SVD, and OGL0-SVD models with group Lasso and L0-norm penalties, using alternating iterative strategies and ADMM.
result Effective in identifying biologically interpretable gene modules with gene prior group knowledge.
Paper develops a method to robustly cluster tensors with outliers.
problem Clustering tensors contaminated by outliers or sample-specific corruptions.
method Transformed Tensor Low-Rank Representation (OR-TLRR) method.
result Provably recovers row space of clean data and detects outliers.
Paper improves tensor completion using unitary transforms.
problem Robust tensor completion for various datasets.
method Transformed tensor SVD with unitary matrices.
result Recovered images have better PSNR than traditional methods.
New method tackles non-smooth tensor data for better recovery.
problem Non-smooth changes in tensor data degrade traditional t-SVD methods.
method Learnable tensor nuclear norm, Alternating Proximal Multiplier Method (APMM), multi-objective tensor recovery framework.
result The proposed method effectively recovers tensor data with non-smooth changes.
Ranky solves SVD for large sparse matrices in distributed systems.
problem Rank problem in large sparse matrices for SVD.
method Distributed approach to solve rank problem.
result Recovers SVD with negligible error for large sparse matrices.
ScaledGD algorithm estimates low-rank tensors efficiently from corrupted data.
problem Estimating meaningful information from corrupted tensor data.
method Scaled gradient descent (ScaledGD) algorithm with tailored spectral initializations.
result ScaledGD achieves linear convergence at a constant rate independent of condition number.
In this paper we focus on the problem of completion of multidimensional arrays (also referred to as tensors) from limited sampling. Our approach is based on a recently proposed tensor-Singular Value Decomposition (t-SVD) [1]. Using this factorization one can derive notion of tensor rank, referred to as the tensor tubal…
This work solves TRPCA under linear transforms, recovering low-rank and sparse components.
problem Exact recovery of tensor low-rank and sparse components from their sum.
method Convex optimization with weighted tensor nuclear norm and ℓ1-norm.
result The convex program exactly recovers the components under certain incoherence conditions.
New K-SVD framework speeds up image denoising with active set algorithm.
problem Efficiently denoise images with high noise levels.
method Proposes K-SVDP using Primal-dual active set (PDAS) algorithm. result Demonstrates comparable performance to state-of-the-art methods.
Dictionary learning and component analysis are part of one of the most well-studied and active research fields, at the intersection of signal and image processing, computer vision, and statistical machine learning. In dictionary learning, the current methods of choice are arguably K-SVD and its variants, which learn a …
New method approximates high-dimensional probability densities efficiently.
problem Approximating high-dimensional probability densities accurately and efficiently.
method Hierarchical tensor-network approach using randomized SVD and linear equations.
result The method effectively approximates high-dimensional densities with linear complexity.
Improved SVD-based NMF initialization reduces initial error and is faster.
problem Improving NMF initialization to reduce convergence error and computational cost.
method Nonnegative SVD with low-rank correction (NNSVD-LRC) that considers discarded SVD factors.
result Significantly reduces initial error with negligible additional computational cost.
In this paper, we propose a general framework for tensor singular value decomposition (tensor SVD), which focuses on the methodology and theory for extracting the hidden low-rank structure from high-dimensional tensor data. Comprehensive results are developed on both the statistical and computational limits for tensor …
Unified framework for structured principal subspace estimation with bounds and rates.
problem Structured principal subspace estimation problems.
method Unified framework, minimax lower and upper bounds, information-geometric complexity.
result Minimax rates of convergence for specific settings, including optimal rates for non-negative PCA/SVD.
New method improves tensor completion and robust PCA using non-convex tensor rank and sparsity measures.
problem Challenging tensor rank minimization in machine learning.
method Proposes a non-convex tensor rank surrogate function and sparsity measure, using concavity for optimization.
result Demonstrates improved accuracy and efficiency in tensor completion and robust PCA.
Tensor networks improve data privacy and robustness in convolutional neural networks.
problem Improving data privacy and robustness in convolutional neural networks.
method Tensor network decomposition for data partitioning and adversarial defense.
result Tensor networks can protect data privacy and resist adversarial attacks.
RKCA combines sparse dictionary learning and robust component analysis for robust low-rank modeling.
problem Learning robust low-rank representations from noisy data.
method Kronecker-decomposable component analysis (RKCA) with efficient learning algorithm.
result RKCA achieves robustness to gross corruption and low-rank modeling.
Formula for complex SVD backpropagation developed.
problem No specific problem stated; focuses on complex SVD.
method Back propagation formula for complex SVD developed.
result Back propagation formula for complex SVD created.
Efficiently recovers low-tubal-rank tensors from few measurements.
problem Recovering tensors with low tubal-rank from limited measurements.
method Factorization and factorized gradient descent.
result Factorized gradient descent reduces computational costs and storage requirements.
Paper develops RGN method for estimating low-rank tensors from noisy measurements.
problem Estimating low-rank tensors from noisy linear measurements.
method Riemannian Gauss-Newton (RGN) method for efficient low-rank tensor estimation.
result First local quadratic convergence guarantee of RGN for low-rank tensor estimation in noisy settings.
DeepTensor uses deep networks to efficiently decompose tensors with improved performance and robustness.
problem Efficiently decomposing tensors with deep learning to capture nonlinear structures.
method Low-rank tensor decomposition using deep generative networks trained to minimize approximation error.
result DeepTensor outperforms classical methods like SVD and PCA in various applications, including image denoising and 3D MRI.
Learning the "blocking" structure is a central challenge for high dimensional data (e.g., gene expression data). Recently, a sparse singular value decomposition (SVD) has been used as a biclustering tool to achieve this goal. However, this model ignores the structural information between variables (e.g., gene interacti…
We consider N-way data arrays and low-rank tensor factorizations where the time mode is coded as a sparse linear combination of temporal elements from an over-complete library. Our method, Shape Constrained Tensor Decomposition (SCTD) is based upon the CANDECOMP/PARAFAC (CP) decomposition which produces r-rank appr…
We introduce a new parameterization method for deep learning layers using spectral tensor train decomposition.
problem Efficiency and stability in deep learning models with weight matrix compression.
method Spectral Tensor Train Parameterization (STTP) of weight matrices.
result Improved compression and training stability in neural networks.
A new tensor completion method handles missing data with missing not at random entries.
problem Handling missing data in tensors where the probability of observation depends on other entries.
method Estimate propensities using convex relaxation, then use higher-order SVD with inverse propensities weights.
result Finite-sample error bounds on the completed tensor are provided.
New algorithm for online tensor factorization with provable guarantees.
problem Factorizing structured tensors with unknown factors and non-convex optimization.
method Online CP/PARAFAC decomposition via dictionary learning with incoherence and sparsity constraints.
result Exact recovery of tensor factors at a linear rate under mild conditions.
We propose a new framework for the analysis of low-rank tensors which lies at the intersection of spectral graph theory and signal processing. As a first step, we present a new graph based low-rank decomposition which approximates the classical low-rank SVD for matrices and multi-linear SVD for tensors. Then, building …
Paper improves tensor approximation for streaming data.
problem Challenges in finding accurate low-tubal-rank tensor approximations in streaming settings.
method Extends Frequent Directions for efficient low-tubal-rank tensor approximation.
result The new algorithm achieves arbitrarily small approximation error with linear sketch size growth.
New estimator reduces bias and variance in tensor and matrix denoising.
problem Optimal bias-variance tradeoff in matrix and tensor estimation.
method One-step variant of higher-order SVD (HOSVD) estimator.
result Achieves optimal bias-variance tradeoff in both matrix and tensor settings.
Study uses random matrix theory to improve tensor approximation accuracy.
problem Improving tensor approximation accuracy in the presence of noise.
method Random matrix theory applied to tensor unfoldings.
result Characterizes spectral behavior of tensor unfoldings and predicts reconstruction performance.
Paper develops a new test for high-dimensional matrix-valued data.
problem Hypothesis testing for mean of matrix-valued data in high-dimensional settings.
method Proposes a new test statistic for high-dimensional matrix rank testing.
result Develops a novel approach for sparse singular value decomposition (SVD) estimation.
A Matlab toolbox for tensor operations based on t-product.
problem Extending matrix operations to tensors.
method Developed a Matlab toolbox implementing tensor operations based on t-product.
result Implemented several tensor operations including SVD, spectral norm, and nuclear norm.
We provide guarantees for learning latent variable models emphasizing on the overcomplete regime, where the dimensionality of the latent space can exceed the observed dimensionality. In particular, we consider multiview mixtures, spherical Gaussian mixtures, ICA, and sparse coding models. We provide tight concentration…
In this paper we consider the dictionary learning problem for sparse representation. We first show that this problem is NP-hard by polynomial time reduction of the densest cut problem. Then, using successive convex approximation strategies, we propose efficient dictionary learning schemes to solve several practical for…
SOFARI improves inference on multi-task learning latent factors.
problem Challenges in precise inference on multi-task learning latent factor matrices.
method High-dimensional manifold-based Neyman near-orthogonality inference on Stiefel manifold structure.
result Easy-to-use bias-corrected estimators for latent factor vectors and singular values with asymptotic normal distributions.
A fast randomized PCA for sparse data, up to 9.1X faster.
problem Processing large sparse data efficiently for dimension reduction.
method Fast randomized PCA algorithm optimized for sparse data.
result Up to 9.1X faster than basic rPCA algorithm without accuracy loss.
Paper proposes a new tensor model for mixed memberships and provides error bounds.
problem Estimating mixed memberships in higher-order multiway data.
method Tensor mixed-membership blockmodel, higher-order orthogonal iteration algorithm (HOOI), simplex corner-finding algorithm.
result Consistency of estimation procedure with error bounds under specific conditions.
Faster matrix completion through randomized SVD algorithms.
problem Efficiently completing large sparse matrices for applications like image inpainting and recommender systems.
method Proposed two fast randomized algorithms (rSVD-PI and rSVD-BKI) and a new subspace recycling technique to accelerate singular value thresholding (SVT) method.
result The proposed algorithms achieve up to 15X faster computation time for image inpainting and movie rating estimation problems.
Study on estimating rank-one tensors in noisy data with heavy tails.
problem Estimating rank-one spiked tensors in the presence of heavy tailed errors.
method Analysis of spectral norm of random tensors with iid entries.
result Signal strength requirements for optimal estimation are similar for heavy tailed and Gaussian noise, but vanish for noise with finite fourth moment.
This paper introduces matrix product state (MPS) decomposition as a new and systematic method to compress multidimensional data represented by higher-order tensors. It solves two major bottlenecks in tensor compression: computation and compression quality. Regardless of tensor order, MPS compresses tensors to matrices …
Tensor decompositions have rich applications in statistics and machine learning, and developing efficient, accurate algorithms for the problem has received much attention recently. Here, we present a new method built on Kruskal's uniqueness theorem to decompose symmetric, nearly orthogonally decomposable tensors. Unlik…
The paper tackles tensor factorization and completion from noisy data.
problem Sparse nonnegative tensor factorization and completion from partial and noisy observations.
method Minimizes the sum of maximum likelihood estimation and tensor ℓ0 norm with nonnegativity constraints. result Error bounds and minimax lower bounds are established for the proposed model.
New algorithms improve RPCA for large matrices with upper rank bounds.
problem Efficiently decompose large matrices into low-rank and sparse parts.
method Combine regularization and matrix multiplication approaches with upper rank bounds.
result Proposed algorithms are faster and more robust than existing methods.
Deterministic bounds for tensor singular values and vectors, differing from matrix cases.
problem Spectral learning of higher-order orthogonally decomposable tensors.
method Deterministic perturbation bounds for singular values and vectors of orthogonally decomposable tensors.
result Perturbation affects each essential singular value/vector in isolation, independent of multiplicity and distance from other singular values.
Method interpolates option prices and volatilities without arbitrage.
problem Interpolating option prices and volatilities without arbitrage.
method Sparse modeling approach based on integral equations and SVD.
result Flexible and efficient framework for arbitrage-free interpolation.