Algorithm recovers sparse and low rank matrix components efficiently.
problem Recovery of sparse and low rank components of matrices.
method Iterative method with adaptive thresholding.
result Algorithm performs well with low run-time and suitable for non-sparse noise.
Robust tensor ring completion improves tensor recovery accuracy and efficiency.
problem Tensor completion sensitivity to sparse components.
method Robust Tensor Ring Completion (RTRC) with weighted nuclear norms and l1 regularization.
result Exact recovery guarantees and superior performance in various tasks.
Develops PRPCA for smooth image recovery combining low-rank and smoothness.
problem Image matrix recovery under low-rank and smoothness assumptions.
method Projected Robust PCA (PRPCA) framework combining low-rank and smoothness.
result Explicit statistical guarantees for PRPCA, reducing matrix dimensionality.
Novel algorithm for separating moving camera video into static and dynamic components.
problem Foreground-background separation in noisy, moving camera video.
method Augmented robust PCA with total variation regularization, OptShrink low-rank matrix estimator.
result Panoramic low-rank component spanning entire field of view, automatically stitching corrupted data.
RKCA combines sparse dictionary learning and robust component analysis for robust low-rank modeling.
problem Learning robust low-rank representations from noisy data.
method Kronecker-decomposable component analysis (RKCA) with efficient learning algorithm.
result RKCA achieves robustness to gross corruption and low-rank modeling.
We simplify SSL by approximating redundant structural components with low-rank factorization.
problem Improving self-supervised learning performance with limited labeled data.
method Low-rank approximation of structural redundancy, introducing ε_s to measure approximation quality.
result The proposed method enhances SSL performance, as shown by theoretical and experimental validations.
IRCUR accelerates RPCA by using CUR decomposition for efficient low rank estimation.
problem Dimension reduction in robust principal component analysis.
method IRCUR employs CUR decomposition to update the low rank component efficiently.
result IRCUR achieves significant computational efficiency compared to existing algorithms.
ReFACTor improves low-rank matrix recovery from noisy data.
problem Recovering low-rank matrices from noisy column-sparse data.
method A simple variation of TSVD, leveraging column-sparsity.
result ReFACTor outperforms TSVD and PCA in various scenarios.
Spectral algorithm reduces samples needed for multitask regression.
problem Jointly recover shared and task-specific components in low-rank multitask regression.
method Common mechanism regression (CMR) model with a non-iterative spectral algorithm.
result Provable non-convex bi-linear structure is overcome with spectral algorithm.
Paper optimizes tensor deflation for non-orthogonal signals.
problem Recovering low-rank signals from noisy tensors with correlated components.
method Developed an asymptotic analysis and optimized deflation procedure using random tensor theory.
result Proposed an efficient tensor deflation algorithm that optimizes a parameter introduced in the deflation mechanism.
Method decomposes streaming data into sparse and low-rank components from compressive measurements.
problem Online decomposing compressive streaming data efficiently.
method Solves n-ℓ1 cluster-weighted minimization to decompose sparse and low-rank components. result Outperforms existing methods for numerical and video data.
New robust method for image decomposition and low-rank modeling.
problem Noise and outliers sensitivity in current methods.
method Combines sparse dictionary learning and PCP, using Kronecker-decomposition.
result Efficient and robust to gross corruption, significantly smaller problem size.
Advances robust principal component analysis with transformed ℓ1 regularization.
problem Recovering low-rank structures from noisy, partially observed data corrupted by sparse outliers.
method Proposes transformed ℓ1 (TL1) regularization to improve approximations of rank and ℓ0 functional.
result Achieves higher accuracy in estimating low-rank and sparse components compared to classical convex models, especially under non-uniform sampling schemes.
New model reduces matrix factorization bias, yielding truly low-rank solutions.
problem Gradient descent's implicit bias in matrix factorization.
method Introducing a new factorization model with constrained factors and diagonal components.
result The new model consistently exhibits a strong implicit bias, yielding truly low-rank solutions.
A new method models user-specific parameters as a low-rank plus sparse component for efficient personalization.
problem Efficient personalization of machine learning models for individual users.
method Meta-learning approach that models network weights as a sum of low-rank and sparse components.
result The proposed method, AMHT-LRS, achieves nearly optimal sample complexity for estimating the low-rank and sparse components.
New algorithms improve RPCA for large matrices with upper rank bounds.
problem Efficiently decompose large matrices into low-rank and sparse parts.
method Combine regularization and matrix multiplication approaches with upper rank bounds.
result Proposed algorithms are faster and more robust than existing methods.
Novel low-rank neural decoder improves μ-ECoG neural decoding.
problem Challenging neural decoding from high-dimensional μ-ECoG data. method Low-rank structure in neural network decoder.
result Low-rank decoder outperforms standard PCA.
New nonconvex regularizers improve low-rank matrix recovery efficiency and accuracy.
problem Efficiently recover low-rank matrices from incomplete data.
method Factor group-sparse regularization, related to Schatten-p norms.
result Improved generalization error bounds for Schatten-p norms as p decreases.
CSD learns a common component for domain generalization, outperforming existing methods.
problem Training models to generalize across unseen domains.
method CSD decomposes the model into a common and specific component, discarding the latter.
result CSD outperforms state-of-the-art domain generalization methods.
Determinantal point processes (DPPs) are an elegant model for encoding probabilities over subsets, such as shopping baskets, of a ground set, such as an item catalog. They are useful for a number of machine learning tasks, including product recommendation. DPPs are parametrized by a positive semi-definite kernel matrix…
The paper reveals low-rank structure in neural network gradients, influenced by data and model parameters.
problem Investigating low-rank structure in gradients of neural networks under relaxed assumptions.
method Spiked data model, relaxation of isotropy assumptions, analysis of mean-field and neural-tangent-kernel scalings.
result Gradient of input weights is approximately low rank, dominated by two rank-one terms.
Given the superposition of a low-rank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish deterministic conditions under which exact recovery of the low-rank and sparse components becomes possible. This fundamental identifiability issue arises with tra…
The paper shows how gradient flow on over-parametrized tensor decomposition behaves like deflation.
problem Understanding the training dynamics of gradient flow on tensor decomposition.
method Empirical observation and mathematical proof of gradient flow dynamics for orthogonally decomposable tensors.
result Gradient flow dynamics for orthogonally decomposable tensors follows a tensor deflation process, recovering all tensor components.
RES-PCA efficiently recovers low-rank matrices without precise rank knowledge.
problem Inefficient and computationally expensive RPCA methods.
method RES-PCA, a scalable and linearly efficient RPCA method.
result RES-PCA is faster and more robust than existing scalable methods.
New algorithms for learning latent variables in graphical models.
problem Estimating the low-rank component in Gaussian graphical models.
method Fast, non-convex learning algorithms for the low-rank component.
result Our algorithms match the best possible sample complexity and achieve computational speed-ups.
ScaledGD improves gradient descent for ill-conditioned low-rank matrix estimation.
problem Efficiently solving ill-conditioned low-rank matrix estimation problems.
method Scaled Gradient Descent (ScaledGD) with adaptive pre-conditioners.
result Linear convergence rate independent of condition number, low per-iteration cost.
Proposes GTTN for discovering all low-rank structures in deep multi-task learning.
problem Discovering all low-rank structures among tasks in deep multi-task models.
method Introduces GTTN, a convex combination of matrix trace norms of all tensor flattenings, to automatically determine the importance of components.
result Demonstrates the effectiveness of GTTN on real-world datasets.
StatLoRA uses statistical inference to allocate ranks in LoRA fine-tuning, improving performance.
problem Balancing efficiency, expressiveness, and generalization in LoRA rank allocation.
method Formulates LoRA rank allocation as a statistical hypothesis testing problem, using estimated p-values to determine component retention or pruning.
result StatLoRA achieves comparable or better performance than existing methods under matched rank budgets.
This work solves TRPCA under linear transforms, recovering low-rank and sparse components.
problem Exact recovery of tensor low-rank and sparse components from their sum.
method Convex optimization with weighted tensor nuclear norm and ℓ1-norm.
result The convex program exactly recovers the components under certain incoherence conditions.
Low-rank MPPCA improves importance sampling in high dimensions.
problem Estimating full-rank GMM covariance matrices in high dimensions is numerically unstable.
method Use MPPCA mixtures as low-rank proposals for importance sampling in high-dimensional spaces.
result Consistent gains in sample efficiency and quality of failure distribution characterization.
Unified framework HASSLE-free decomposes large model weights into sparse and low-rank components.
problem Efficiently compress large foundation models to reduce inference costs.
method Designs a unified framework for sparse plus low-rank matrix decomposition with a local layer-wise reconstruction error objective.
result HASSLE-free framework significantly outperforms state-of-the-art methods in compression and evaluation benchmarks.
We present an algorithm, AROFAC2, which detects the (CP-)rank of a degree 3 tensor and calculates its factorization into rank-one components. We provide generative conditions for the algorithm to work and demonstrate on both synthetic and real world data that AROFAC2 is a potentially outperforming alternative to the go…
New method for robust PCA with exponential family distributions.
problem Recovering low-rank structure from data matrices with outliers.
method Alternating Direction Method of Multipliers for eextRPCA. result Demonstrated effectiveness in steel sheet defect detection and crime activity monitoring.
We consider an online version of the robust Principle Component Analysis (PCA), which arises naturally in time-varying source separations such as video foreground-background separation. This paper proposes a compressive online robust PCA with prior information for recursively separating a sequences of frames into spars…
Study shows generative priors improve rank-one matrix recovery with optimal sample complexity.
problem Recovering a rank-one signal matrix from noisy data with additional prior information.
method Analysis of a nonlinear least squares objective with a favorable global optimization landscape.
result Established optimal sample complexity for generative priors in rank-one matrix recovery.
Riemannian gradient descent escapes some spurious critical points on low-rank matrix manifold.
problem Spurious critical points on the boundary of low-rank matrix manifold.
method Riemannian gradient descent with dynamical low-rank approximation and rescaled gradient flow.
result Riemannian gradient descent escapes some spurious critical points on the boundary of the manifold.
New method for separating foreground from background in noisy, moving camera video.
problem Foreground-background separation in noisy, free-moving camera video.
method Registers frames, encodes perspective as missing data, uses OptShrink for low-rank estimation, and weighted total variation for smooth foreground.
result Panoramic background component that stitches together corrupted data from overlapping frames.
We study the estimation of the latent variable Gaussian graphical model (LVGGM), where the precision matrix is the superposition of a sparse matrix and a low-rank matrix. In order to speed up the estimation of the sparse plus low-rank components, we propose a sparsity constrained maximum likelihood estimator based on m…
Paper proposes a new method to separate low rank and sparse matrices without bias.
problem Recovering low rank and sparse matrices from measurements.
method Uses nonconvex regularizers and alternating proximal gradient descent.
result Error bounds for the algorithm applied to sparse optimization, matrix completion, and robust PCA.
Paper solves TRPCA problem for tensor data with new tensor nuclear norm.
problem Exact recovery of tensor low-rank and sparse components.
method Introduces tensor-tensor product and new tensor nuclear norm to solve TRPCA.
result The new tensor nuclear norm guarantees exact recovery of tensor data.
We accelerate the power method for strong low-rank approximation using fast sketching.
problem Efficiency bottleneck in power method for large target ranks.
method Developed an algorithmic and theoretical framework for accelerating the power method using fast sketching.
result Simple and provably efficient methods for singular value decomposition, low-rank factorization, and Nyström approximation.
New method for factor analysis using nuclear and ℓ0 norms.
problem Finding a low-rank plus sparse decomposition from noisy covariance matrix.
method Formulated an optimization problem with nuclear norm, ℓ0 norm, and KL divergence. Used alternating minimization algorithm. result Algorithm effectively decomposes covariance matrices in synthetic and real datasets.
Sparsity-based approaches have been popular in many applications in image processing and imaging. Compressed sensing exploits the sparsity of images in a transform domain or dictionary to improve image recovery from undersampled measurements. In the context of inverse problems in dynamic imaging, recent research has de…
Framework for robust matrix estimation with side information.
problem High-dimensional matrix estimation with restrictive structure.
method Flexible framework decomposing matrix into four components: interaction, row, column, and residual.
result Improved imputation accuracy and treatment-effect estimation with side information.
Paper develops methods for non-quadratic loss low-rank matrix recovery.
problem Recovery of low-rank matrices with non-quadratic losses.
method Projected gradient method with a regularity projection oracle.
result Projected gradient method converges globally and linearly.
In many applications that require matrix solutions of minimal rank, the underlying cost function is non-convex leading to an intractable, NP-hard optimization problem. Consequently, the convex nuclear norm is frequently used as a surrogate penalty term for matrix rank. The problem is that in many practical scenarios th…
In many applications that require matrix solutions of minimal rank, the underlying cost function is non-convex leading to an intractable, NP-hard optimization problem. Consequently, the convex nuclear norm is frequently used as a surrogate penalty term for matrix rank. The problem is that in many practical scenarios th…
Polynomial sketch approximates functions of low-rank matrices efficiently.
problem Approximating element-wise functions of low-rank matrices without full access.
method Combining polynomial approximation and tensor sketch for monomials.
result Efficient algorithm with lower complexity than full matrix access.