Review of algorithms for linear system approximations.
problem Linear approximation of high-dimensional dynamical systems.
method State-of-the-art algorithms for low-rank DMD.
result Provides additional details for comprehensive understanding.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “low-rank decomposition”
Large CNNs have delivered impressive performance in various computer vision applications. But the storage and computation requirements make it problematic for deploying these models on mobile devices. Recently, tensor decompositions have been used for speeding up CNNs. In this paper, we further develop the tensor decom…
Unified algorithm for tensor decomposition supports multiple loss functions and models.
problem Efficient tensor decomposition for various models and loss functions.
method Hierarchical combination of ADMM and MM for optimization.
result Wide-range applications can be solved by the proposed algorithm.
Efficient algorithm for Hadamard decomposition of matrices.
problem Decomposing matrices into low-rank factors efficiently.
method Alternating optimization with SVD-inspired initialization and momentum.
result Significantly improved performance compared to existing methods.
Proposes a faster Isomap algorithm by reducing eigenvalue decomposition complexity.
problem High computational complexity of Isomap, especially in eigenvalue decomposition stage.
method Introduces a projection operator to reduce the complexity of the eigenvalue decomposition stage to linear order.
result Reduces Isomap's computational complexity to linear order while preserving structural information.
The report analyzes Legendre decomposition for tensor data.
problem Finding effective lower dimensional representations of tensors.
method Theoretical analysis of dual parameters and dually flat manifold properties, followed by experimental verification and clustering.
result Parameters on submanifold cannot be directly used as low-rank representations.
Unified model for tensor completion using low-rank and sparse Tucker decomposition.
problem Estimating missing data from incomplete tensor measurements.
method Unified low-rank and sparse enhanced Tucker decomposition model with ADMM.
result Our model achieves higher recovery accuracy on various real-world data sets.
Often, large, high dimensional datasets collected across multiple modalities can be organized as a higher order tensor. Low-rank tensor decomposition then arises as a powerful and widely used tool to discover simple low dimensional structures underlying such data. However, we currently lack a theoretical understanding …
This work studies the linear approximation of high-dimensional dynamical systems using low-rank dynamic mode decomposition (DMD). Searching this approximation in a data-driven approach is formalised as attempting to solve a low-rank constrained optimisation problem. This problem is non-convex and state-of-the-art algor…
The paper shows how gradient flow on over-parametrized tensor decomposition behaves like deflation.
problem Understanding the training dynamics of gradient flow on tensor decomposition.
method Empirical observation and mathematical proof of gradient flow dynamics for orthogonally decomposable tensors.
result Gradient flow dynamics for orthogonally decomposable tensors follows a tensor deflation process, recovering all tensor components.
IRCUR accelerates RPCA by using CUR decomposition for efficient low rank estimation.
problem Dimension reduction in robust principal component analysis.
method IRCUR employs CUR decomposition to update the low rank component efficiently.
result IRCUR achieves significant computational efficiency compared to existing algorithms.
Develops PRPCA for smooth image recovery combining low-rank and smoothness.
problem Image matrix recovery under low-rank and smoothness assumptions.
method Projected Robust PCA (PRPCA) framework combining low-rank and smoothness.
result Explicit statistical guarantees for PRPCA, reducing matrix dimensionality.
FLAMBE tackles RL in low rank MDPs by learning features.
problem Dealing with the curse of dimensionality in RL.
method Develops FLAMBE, a method that engages in exploration and representation learning for RL in low rank transition models.
result FLAMBE efficiently learns features for RL in low rank transition models.
Fast and accurate methods for low-rank learning problems.
problem Partial singular value decomposition and numerical rank estimation of huge matrices.
method Krylov subspaces and Ritz vectors for fast and accurate solutions.
result Advantages over traditional methods in accuracy and speed.
DeepTensor uses deep networks to efficiently decompose tensors with improved performance and robustness.
problem Efficiently decomposing tensors with deep learning to capture nonlinear structures.
method Low-rank tensor decomposition using deep generative networks trained to minimize approximation error.
result DeepTensor outperforms classical methods like SVD and PCA in various applications, including image denoising and 3D MRI.
Unified framework for non-Euclidean CPD under scalable stochastic mirror descent.
problem Handling non-Euclidean losses in tensor decomposition.
method Tensor fiber sampling strategy-based stochastic mirror descent.
result Global convergence to a stationary point under reasonable conditions.
This paper is concerned with the problem of low rank plus sparse matrix decomposition for big data. Conventional algorithms for matrix decomposition use the entire data to extract the low-rank and sparse components, and are based on optimization problems with complexity that scales with the dimension of the data, which…
Dynamic Mode Decomposition (DMD) has emerged as a powerful tool for analyzing the dynamics of non-linear systems from experimental datasets. Recently, several attempts have extended DMD to the context of low-rank approximations. This extension is of particular interest for reduced-order modeling in various applicative …
Globally optimized SVD compression of LLMs via Fermi-function-based rank selection and gauge fixingcs.LG
Physics-inspired methods optimize SVD compression of LLMs.
problem Efficiently compressing large language models (LLMs) using SVD.
method FermiGrad for globally optimal rank selection and PivGa for lossless compression.
result Global optimization of SVD ranks and lossless compression of low-rank factors.
Unified framework HASSLE-free decomposes large model weights into sparse and low-rank components.
problem Efficiently compress large foundation models to reduce inference costs.
method Designs a unified framework for sparse plus low-rank matrix decomposition with a local layer-wise reconstruction error objective.
result HASSLE-free framework significantly outperforms state-of-the-art methods in compression and evaluation benchmarks.
A low-rank tensor model simplifies multi-dimensional Markov chains.
problem Simplifying the dynamics of multi-dimensional Markov chains.
method Low-rank tensor decomposition for multi-dimensional state spaces.
result Our tensor model requires fewer parameters and samples than conventional methods.
CALDERA compresses large language models by approximating weight matrices with low-rank, low-precision factors.
problem The large sizes of Large Language Models (LLMs) hinder deployment on edge devices.
method CALDERA approximates weight matrices as , where and are low-rank factors quantized to low precision.
result CALDERA achieves better zero-shot performance than existing techniques, especially with low bit precision.
We propose a new framework for the analysis of low-rank tensors which lies at the intersection of spectral graph theory and signal processing. As a first step, we present a new graph based low-rank decomposition which approximates the classical low-rank SVD for matrices and multi-linear SVD for tensors. Then, building …
In tensor completion tasks, the traditional low-rank tensor decomposition models suffer from the laborious model selection problem due to their high model sensitivity. In particular, for tensor ring (TR) decomposition, the number of model possibilities grows exponentially with the tensor order, which makes it rather ch…
D-CDLF: Decomposition of Common and Distinctive Latent Factors for Multi-view High-dimensional Datastat.ML
Proposes D-CDLF for multi-view data decomposition.
problem Uncorrelatedness between common and distinctive latent factors.
method Decomposes data into common, distinctive, and noise components.
result Effective uncorrelatedness between distinctive latent factors from different views.
Method estimates joint probability density from samples using low-rank decomposition and random projections.
problem Estimating joint probability density from limited samples.
method Low-rank tensor decomposition, dictionaries, and Radon transforms.
result Algorithm outperforms previous methods in estimating synthetic probability densities.
Solves weakly supervised regression using low-rank approximations and manifold regularization.
problem Weakly supervised regression with known, unknown, and uncertain labels.
method Combines manifold regularization and low-rank matrix decomposition for optimization.
result Improves solution quality and stability for large datasets.
Robust tensor CP decomposition involves decomposing a tensor into low rank and sparse components. We propose a novel non-convex iterative algorithm with guaranteed recovery. It alternates between low-rank CP decomposition through gradient ascent (a variant of the tensor power method), and hard thresholding of the resid…
Proposes BHT-ARIMA for forecasting multiple short time series.
problem Forecasting multiple short time series with mutual correlations.
method Block Hankel tensors, Tucker decomposition, generalized tensor ARIMA.
result Improves forecasting accuracy and reduces computational cost.
We consider the problem of online subspace tracking of a partially observed high-dimensional data stream corrupted by noise, where we assume that the data lie in a low-dimensional linear subspace. This problem is cast as an online low-rank tensor completion problem. We propose a novel online tensor subspace tracking al…
New method decomposes corrupted data matrices into sparse and low-rank components.
problem Decomposing corrupted data matrices into sparse and low-rank components.
method Discrete optimization approach with alternating minimization, semidefinite relaxation, and branch-and-bound algorithm.
result High-quality solutions and meaningful bounds for SLR problems.
Low-rank tensor decomposition and completion have attracted significant interest from academia given the ubiquity of tensor data. However, the low-rank structure is a global property, which will not be fulfilled when the data presents complex and weak dependencies given specific graph structures. One particular applica…
3BASiL-TM decomposes LLMs into sparse and low-rank matrices for efficient compression.
problem Efficiently compressing large language models without significant performance loss.
method 3-Block ADMM method and transformer-matching refinement step for sparse plus low-rank decomposition.
result 3BASiL-TM reduces perplexity gap by over 30% and speeds up compression by 2.5x.
Paper proposes a new optimization framework for learning eigenfunctions of operators.
problem Computing eigenvalue decomposition of high-dimensional operators.
method Operator SVD with Neural Networks via Nested Low-Rank Approximation.
result Proposed method efficiently learns top-L singular values and functions in the correct order.
Tensor completion estimates missing components by exploiting the low-rank structure of multi-way data. The recently proposed methods based on tensor train (TT) and tensor ring (TR) show better performance in image recovery than classical ones. Compared with TT and TR, the projected entangled pair state (PEPS), which is…
Tensorized random projections reduce high-dimensional tensor size efficiently.
problem Efficiently reducing the dimension of very high-dimensional tensors.
method Proposes two tensorized random projection maps using TT and CP decompositions.
result TT format offers superior performance in terms of required random projection size.
RieCUR improves Robust PCA by combining Riemannian optimization and CUR decompositions.
problem Robust Principal Component Analysis (PCA) to recover low-rank and sparse matrices from their sum.
method Riemannian CUR (RieCUR) algorithm that combines Riemannian optimization and robust CUR decompositions.
result RieCUR achieves state-of-the-art performance in Robust PCA with improved robustness to outliers and comparable computational complexity.
Low-rank approximation is an effective model compression technique to not only reduce parameter storage requirements, but to also reduce computations. For convolutional neural networks (CNNs), however, well-known low-rank approximation methods, such as Tucker or CP decomposition, result in degraded model accuracy becau…
Robust PCA is a widely used statistical procedure to recover a underlying low-rank matrix with grossly corrupted observations. This work considers the problem of robust PCA as a nonconvex optimization problem on the manifold of low-rank matrices, and proposes two algorithms (for two versions of retractions) based on ma…
Paper introduces a new histogram estimator for nonparametric density estimation that improves performance.
problem Smoothness-based nonparametric density estimators are not optimal for all types of data.
method Incorporates a multi-view latent variable model into histogram-style estimators.
result A new histogram estimator converges faster to multi-view models in error.
This paper reviews methods for discovering patient subgroups from EHR data.
problem Discovering subgroups of patients and co-occurring medical conditions from EHR data.
method Low-rank data approximation methods like matrix and tensor decompositions.
result These methods provide transparent and interpretable insights into patient phenotypes.
CSD learns a common component for domain generalization, outperforming existing methods.
problem Training models to generalize across unseen domains.
method CSD decomposes the model into a common and specific component, discarding the latter.
result CSD outperforms state-of-the-art domain generalization methods.
Although the convolutional neural networks (CNNs) have become popular for various image processing and computer vision task recently, it remains a challenging problem to reduce the storage cost of the parameters for resource-limited platforms. In the previous studies, tensor decomposition (TD) has achieved promising co…
NACT improves tensor regression predictions with regularization.
problem Improving tensor regression predictions with structural information.
method Noise-Augmented regularization on Tucker decomposition.
result Achieves exact regularization on core tensor in linear and generalized linear tensor regression.
Paper proposes a novel MTL framework for personalized modeling of diverse individuals.
problem Personalized modeling of heterogeneous subpopulations with high-dimensional data.
method Low-rank tensor decomposition for multi-task learning.
result Superior performance compared to benchmarks in diverse subpopulation scenarios.
Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Modelscs.LG
LoRAs enable efficient adaptation of large models; this paper explores processing LoRA weights with machine learning.
problem Efficient processing of low-rank weight decompositions in large finetuned models.
method Developed symmetry-aware invariant and equivariant LoL models to process LoRA weights.
result LoL models can predict CLIP scores, finetuning data attributes, and accuracy on downstream tasks.
factor analysismath.OC
New method for factor analysis using nuclear and norms.
problem Finding a low-rank plus sparse decomposition from noisy covariance matrix.
method Formulated an optimization problem with nuclear norm, norm, and KL divergence. Used alternating minimization algorithm.
result Algorithm effectively decomposes covariance matrices in synthetic and real datasets.
In this article, we derive a Bayesian model to learning the sparse and low rank PARAFAC decomposition for the observed tensor with missing values via the elastic net, with property to find the true rank and sparse factor matrix which is robust to the noise. We formulate efficient block coordinate descent algorithm and …