We consider the problem of modeling multivariate time series with parsimonious dynamical models which can be represented as sparse dynamic Bayesian networks with few latent nodes. This structure translates into a sparse plus low rank model. In this paper, we propose a Gaussian regression approach to identify such a mod…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “sparse plus low-rank”
Recovery of Low-Rank Plus Compressed Sparse Matrices with Application to Unveiling Traffic Anomaliescs.IT
Given the superposition of a low-rank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish deterministic conditions under which exact recovery of the low-rank and sparse components becomes possible. This fundamental identifiability issue arises with tra…
We propose a unified framework to solve general low-rank plus sparse matrix recovery problems based on matrix factorization, which covers a broad family of objective functions satisfying the restricted strong convexity and smoothness conditions. Based on projected gradient descent and the double thresholding operator, …
Unified framework HASSLE-free decomposes large model weights into sparse and low-rank components.
problem Efficiently compress large foundation models to reduce inference costs.
method Designs a unified framework for sparse plus low-rank matrix decomposition with a local layer-wise reconstruction error objective.
result HASSLE-free framework significantly outperforms state-of-the-art methods in compression and evaluation benchmarks.
New method decomposes corrupted data matrices into sparse and low-rank components.
problem Decomposing corrupted data matrices into sparse and low-rank components.
method Discrete optimization approach with alternating minimization, semidefinite relaxation, and branch-and-bound algorithm.
result High-quality solutions and meaningful bounds for SLR problems.
3BASiL-TM decomposes LLMs into sparse and low-rank matrices for efficient compression.
problem Efficiently compressing large language models without significant performance loss.
method 3-Block ADMM method and transformer-matching refinement step for sparse plus low-rank decomposition.
result 3BASiL-TM reduces perplexity gap by over 30% and speeds up compression by 2.5x.
New guarantees for recovering matrices as low-rank plus sparse from fewer measurements.
problem Recovering matrices as the sum of a low-rank and sparse matrix from a limited number of measurements.
method Developed guarantees for recovery of low-rank plus sparse matrices from measurements, using semidefinite programming and gradient descent algorithms.
result Guarantees for recovery of low-rank plus sparse matrices from fewer measurements than previously possible.
A new method models user-specific parameters as a low-rank plus sparse component for efficient personalization.
problem Efficient personalization of machine learning models for individual users.
method Meta-learning approach that models network weights as a sum of low-rank and sparse components.
result The proposed method, AMHT-LRS, achieves nearly optimal sample complexity for estimating the low-rank and sparse components.
Paper proposes a new method to separate low rank and sparse matrices without bias.
problem Recovering low rank and sparse matrices from measurements.
method Uses nonconvex regularizers and alternating proximal gradient descent.
result Error bounds for the algorithm applied to sparse optimization, matrix completion, and robust PCA.
Efficient solver for nonconvex tensor regularization reduces computational cost.
problem Computational inefficiency in extending nonconvex regularization to tensor learning.
method Proximal average algorithm with adaptive momentum, maintaining sparse plus low-rank structure.
result Shows good statistical performance and accuracy on tensor completion problems.
This paper is concerned with the problem of low rank plus sparse matrix decomposition for big data. Conventional algorithms for matrix decomposition use the entire data to extract the low-rank and sparse components, and are based on optimization problems with complexity that scales with the dimension of the data, which…
factor analysismath.OC
New method for factor analysis using nuclear and norms.
problem Finding a low-rank plus sparse decomposition from noisy covariance matrix.
method Formulated an optimization problem with nuclear norm, norm, and KL divergence. Used alternating minimization algorithm.
result Algorithm effectively decomposes covariance matrices in synthetic and real datasets.
We study the estimation of the latent variable Gaussian graphical model (LVGGM), where the precision matrix is the superposition of a sparse matrix and a low-rank matrix. In order to speed up the estimation of the sparse plus low-rank components, we propose a sparsity constrained maximum likelihood estimator based on m…
We analyze a class of estimators based on convex relaxation for solving high-dimensional matrix decomposition problems. The observations are noisy realizations of a linear transformation of the sum of an approximately) low rank matrix with a second matrix endowed with a complementary …
cuRegOT accelerates GPU-based entropic OT solving.
problem Slow convergence and high computational cost of optimal transport on GPUs.
method High-performance GPU solver with algorithmic and architectural optimizations.
result Significant speedups over state-of-the-art solvers.
Low-Rank Plus Sparse Matrix Transfer Learning under Growing Representations and Ambient Dimensionscs.LG
The paper tackles transfer learning for growing matrix representations, improving estimation accuracy.
problem Structured matrix estimation under growing ambient dimensions and latent representations.
method Proposes a general transfer framework decomposing target parameters into embedded source components, low-rank innovations, and sparse edits. Develops an anchored alternating projection estimator.
result Establishes deterministic error bounds that separate target noise, representation growth, and source estimation error, yielding improved rates.
Gaussian graphical models (GGM) have been widely used in many high-dimensional applications ranging from biological and financial data to recommender systems. Sparsity in GGM plays a central role both statistically and computationally. Unfortunately, real-world data often does not fit well to sparse graphical models. I…
This paper considers the recovery of a low-rank matrix from an observed version that simultaneously contains both (a) erasures: most entries are not observed, and (b) errors: values at a constant fraction of (unknown) locations are arbitrarily corrupted. We provide a new unified performance guarantee on when the natura…
Paper tackles matrix estimation under arbitrary noise, achieving minimax optimality.
problem Noisy low-rank-plus-sparse matrix recovery under arbitrary dependence.
method Incoherent-constrained least-square estimator, novel energy spreading result.
result Achieves minimax optimality in estimating structured Markov transition kernels.
Efficiently finds sparse solutions to max-plus equations for convex regression.
problem Finding sparse solutions to max-plus equations for convex multivariate regression.
method Polynomial-time algorithm for sparse approximate solutions.
result Optimal piecewise-linear fitting with minimum number of regions.
This paper considers a new framework to detect communities in a graph from the observation of signals at its nodes. We model the observed signals as noisy outputs of an unknown network process, represented as a graph filter that is excited by a set of unknown low-rank inputs/excitations. Application scenarios of this m…
Develops PRPCA for smooth image recovery combining low-rank and smoothness.
problem Image matrix recovery under low-rank and smoothness assumptions.
method Projected Robust PCA (PRPCA) framework combining low-rank and smoothness.
result Explicit statistical guarantees for PRPCA, reducing matrix dimensionality.
Robust principal component analysis (RPCA) can recover low-rank matrices when they are corrupted by sparse noises. In practice, many matrices are, however, of high-rank and hence cannot be recovered by RPCA. We propose a novel method called robust kernel principal component analysis (RKPCA) to decompose a partially cor…
Paper develops an online EM algorithm for graph signal inference from streaming data.
problem Joint inference and clustering of graph signals with non-white excitation.
method Mixture model with low-rank plus sparse prior, online EM algorithm.
result Proposed online EM algorithm converges to MAP solution.
A semi-parametric, non-linear regression model in the presence of latent variables is introduced. These latent variables can correspond to unmodeled phenomena or unmeasured agents in a complex networked system. This new formulation allows joint estimation of certain non-linearities in the system, the direct interaction…
Robust high-dimensional data processing has witnessed an exciting development in recent years, as theoretical results have shown that it is possible using convex programming to optimize data fit to a low-rank component plus a sparse outlier component. This problem is also known as Robust PCA, and it has found applicati…
Signed Network Embedding with Application to Simultaneous Detection of Communities and Anomaliescs.SI
Unified model for signed networks separates balance and anomaly effects.
problem Ignoring sign information in signed networks leads to inaccurate analysis.
method Low rank plus sparse matrix decomposition with regularized formulation.
result The model accurately detects communities and anomalies in signed networks.
New MCMC method learns sparse preconditioner for high-dimensional problems.
problem High-dimensional sampling with complex correlation structures.
method Adaptive MCMC with sparse preconditioner using online PCA.
result Significant reduction in computational complexity and improved performance.
We consider the matrix completion problem of recovering a structured matrix from noisy and partial measurements. Recent works have proposed tractable estimators with strong statistical guarantees for the case where the underlying matrix is low--rank, and the measurements consist of a subset, either of the exact individ…
Given a limited number of entries from the superposition of a low-rank matrix plus the product of a known fat compression matrix times a sparse matrix, recovery of the low-rank and sparse components is a fundamental task subsuming compressed sensing, matrix completion, and principal components pursuit. This paper devel…
This work tackles sparse coding in DLRA for interpretable multiway data.
problem Sparse coding in DLRA for interpretable multiway data.
method Proposes a new sparse-coding subproblem (MSC) and several algorithms to solve it.
result DLRA extends low-rank approximations, reducing variance and enhancing interpretability.
New model captures time series dependence across and within blocks.
problem Complex multivariate time series dependence structures.
method Time series Gaussian chain graph models with directed and undirected edges.
result Consistent recovery of time series chain graph structure.
New Max-Plus neural network exploits subgradient sparsity for efficient training.
problem Training Max-Plus neural networks is challenging due to dense subgradients.
method Proposes a sparse subgradient algorithm tailored to Max-Plus models.
result Achieves more efficient updates while retaining theoretical guarantees.
In the past decade, sparse and low-rank recovery have drawn much attention in many areas such as signal/image processing, statistics, bioinformatics and machine learning. To achieve sparsity and/or low-rankness inducing, the norm and nuclear norm are of the most popular regularization penalties due to their co…
Robust Low-rank Matrix Completion via an Alternating Manifold Proximal Gradient Continuation Methodcs.LG
New method improves robust low-rank matrix completion for computer vision.
problem Robust low-rank matrix completion for partially observed data.
method Formulated as a nonsmooth Riemannian optimization problem over Grassmann manifold, solved with an alternating manifold proximal gradient continuation method.
result Demonstrated advantages over existing approaches in background extraction from surveillance videos.
In this letter, we propose an algorithm for recovery of sparse and low rank components of matrices using an iterative method with adaptive thresholding. In each iteration, the low rank and sparse components are obtained using a thresholding operator. This algorithm is fast and can be implemented easily. We compare it w…
RNNs solve modular addition tasks using low rank and sparse Fourier structures.
problem Solving modular addition tasks with recurrent neural networks.
method Identified low rank structures and sparse Fourier representations in RNN weights.
result RNNs robust to removing individual frequencies but degrade with more ablation.
Unified model for tensor completion using low-rank and sparse Tucker decomposition.
problem Estimating missing data from incomplete tensor measurements.
method Unified low-rank and sparse enhanced Tucker decomposition model with ADMM.
result Our model achieves higher recovery accuracy on various real-world data sets.
Smoothed Low Rank and Sparse Matrix Recovery by Iteratively Reweighted Least Squares Minimizationcs.LG
This work presents a general framework for solving the low rank and/or sparse matrix minimization problems, which may involve multiple non-smooth terms. The Iteratively Reweighted Least Squares (IRLS) method is a fast solver, which smooths the objective function and minimizes it by alternately updating the variables an…
This paper develops a new class of nonconvex regularizers for low-rank matrix recovery. Many regularizers are motivated as convex relaxations of the matrix rank function. Our new factor group-sparse regularizers are motivated as a relaxation of the number of nonzero columns in a factorization of the matrix. These nonco…
Paper proposes CLAIR for efficient LLM fine-tuning across clients.
problem Fine-tuning large language models (LLMs) efficiently and collaboratively.
method Federated LoRA fine-tuning with Collaborative Low-rank Alignment and Identifiable Recovery (CLAIR).
result CLAIR achieves better performance and contamination detection compared to local fine-tuning.
Low Rank for Rank: Uncertainty-Aware Task-Specific LLM Ranking under Sparse Pairwise Comparisonsstat.ME
Low-rank framework for task-specific LLM ranking from sparse comparisons.
problem Challenges in reliable task-specific ranking of LLMs under sparse, imbalanced comparisons.
method Low-rank modeling of task-by-model ability matrix, max-norm accurate estimator, task-wise top-K recovery guarantees, uncertainty quantification framework.
result Improves sample efficiency and produces tighter, better-calibrated ranking certificates.
T-Rex uses EM to fit robust factor models in noisy data.
problem Robustly fitting factor models in high-dimensional data with heavy tails and outliers.
method Expectation-Maximization (EM) algorithm based on Tyler's M-estimator for elliptical distributions.
result Demonstrates robustness in direction-of-arrival estimation and subspace recovery.
In this paper, we propose a low-rank approximation method based on discrete least-squares for the approximation of a multivariate function from random, noisy-free observations. Sparsity inducing regularization techniques are used within classical algorithms for low-rank approximation in order to exploit the possible sp…
SKI speeds up Toeplitz Neural Networks by avoiding explicit decay bias and using frequency response.
problem Efficiently compute and update Toeplitz matrices in neural networks.
method Sparse plus low-rank decomposition, asymmetric SKI, frequency response modeling.
result Achieved significant speedup with minimal performance loss.
New method improves tensor completion by selectively preserving important elements.
problem Recovering corrupted high-dimensional tensor data with missing entries and noise.
method Tensor weighted correlated total variation (TWCTV) regularizer with ADMM algorithm.
result Superior performance in image completion, denoising, and background subtraction tasks.
This paper aims at achieving a simultaneously sparse and low-rank estimator from the semidefinite population covariance matrices. We first benefit from a convex optimization which develops -norm penalty to encourage the sparsity and nuclear norm to favor the low-rank property. For the proposed estimator, we then p…
Low-rank tensor completion recovers missing entries based on different tensor decompositions. Due to its outstanding performance in exploiting some higher-order data structure, low rank tensor ring has been applied in tensor completion. To further deal with its sensitivity to sparse component as it does in tensor princ…