The problem of low rank matrix completion is considered in this paper. To exploit the underlying low-rank structure of the data matrix, we propose a hierarchical Gaussian prior model, where columns of the low-rank matrix are assumed to follow a Gaussian distribution with zero mean and a common precision matrix, and a W…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “low-rank prior”
In this paper, we consider the problem of low-rank phase retrieval whose objective is to estimate a complex low-rank matrix from magnitude-only measurements. We propose a hierarchical prior model for low-rank phase retrieval, in which a Gaussian-Wishart hierarchical prior is placed on the underlying low-rank matrix to …
A new framework improves tensor completion accuracy by considering numerical priors.
problem Tensor completion accuracy loss due to ignoring numerical priors.
method Generalized CP Decomposition Tensor Completion (GCDTC) framework incorporating numerical priors.
result GCDTC framework outperforms state-of-the-arts in non-negative tensor completion.
Robust Recovery via Implicit Bias of Discrepant Learning Rates for Double Over-parameterizationcs.LG
Gradient descent recovers low-rank matrices from corrupted measurements with double over-parameterization.
problem Robust recovery of low-rank matrices from grossly corrupted measurements.
method Gradient descent with discrepant learning rates for double over-parameterized models.
result Gradient descent with discrepant learning rates provably recovers the underlying matrix without prior knowledge on rank or sparsity.
Pixel-wise classification, where each pixel is assigned to a predefined class, is one of the most important procedures in hyperspectral image (HSI) analysis. By representing a test pixel as a linear combination of a small subset of labeled pixels, a sparse representation classifier (SRC) gives rather plausible results …
A Set-Theoretic Study of the Relationships of Image Models and Priors for Restoration Problemseess.IV
The paper analyzes relationships between various image models for restoration.
problem Unclear relationships among popular image models for restoration.
method Theoretical analysis and experimental study of image models.
result Improved denoising performance by combining multiple image models.
New insights into training deep networks with low rank layers.
problem Efficiency in training deep neural networks.
method Analysis of techniques for training in low rank space.
result Falsified common beliefs in training deep networks.
We consider an online version of the robust Principle Component Analysis (PCA), which arises naturally in time-varying source separations such as video foreground-background separation. This paper proposes a compressive online robust PCA with prior information for recursively separating a sequences of frames into spars…
Algorithm recovers multiple low-rank matrices from unlabeled data.
problem Learning mixtures of low-rank models from unlabelled data.
method Three-stage meta-algorithm that copes with non-convexity and noise.
result Near-optimal sample and computational complexities under Gaussian designs.
We develop latent variable models for Bayesian learning based low-rank matrix completion and reconstruction from linear measurements. For under-determined systems, the developed methods are shown to reconstruct low-rank matrices when neither the rank nor the noise power is known a-priori. We derive relations between th…
This work provides a guaranteed tensor recovery method by combining low-rankness and smoothness priors.
problem Guaranteed tensor recovery with theoretical guarantees for low-rank and smoothness priors.
method Developed a new regularization term that combines low-rankness and smoothness priors, proving exact recovery guarantees.
result Rigorously proved exact recovery guarantees for tensor completion and tensor robust principal component analysis.
New RL algorithm maximizes CVaR in low-rank MDPs with provable efficiency.
problem Maximizing CVaR in large state spaces with function approximation.
method Upper Confidence Bound (UCB) bonus-driven algorithm for low-rank MDPs.
result Achieves sample complexity of O(H^7 A^2 d^4 / τ^2 ε^2) for ε-optimal CVaR.
LaLoRA prevents forgetting in LoRA fine-tuning.
problem Catastrophic forgetting in fine-tuned models.
method LaLoRA applies Laplace approximation to LoRA weights for regularization.
result Improved learning-forgetting trade-off with controllable regularization strength.
The paper analyzes how low-rank layers in neural networks improve generalization.
problem Understanding how low-rank layers affect generalization in neural networks.
method Applying Maurer's chain rule for Gaussian complexity to analyze rank and spectral norm constraints.
result Deep networks with low-rank layers achieve better generalization than those with full-rank layers.
The problem of low-rank matrix estimation recently received a lot of attention due to challenging applications. A lot of work has been done on rank-penalized methods and convex relaxation, both on the theoretical and applied sides. However, only a few papers considered Bayesian estimation. In this paper, we review the …
FLAMBE tackles RL in low rank MDPs by learning features.
problem Dealing with the curse of dimensionality in RL.
method Develops FLAMBE, a method that engages in exploration and representation learning for RL in low rank transition models.
result FLAMBE efficiently learns features for RL in low rank transition models.
BoRA finetunes multi-task LLMs by sharing information through hierarchical priors.
problem Limited data for some tasks in multi-task LLMs.
method Bayesian hierarchical low-rank adaption.
result BoRA outperforms individual and unified model approaches.
Study shows generative priors improve rank-one matrix recovery with optimal sample complexity.
problem Recovering a rank-one signal matrix from noisy data with additional prior information.
method Analysis of a nonlinear least squares objective with a favorable global optimization landscape.
result Established optimal sample complexity for generative priors in rank-one matrix recovery.
High resolution magnetic resonance (MR) images are desired for accurate diagnostics. In practice, image resolution is restricted by factors like hardware, cost and processing constraints. Recently, deep learning methods have been shown to produce compelling state of the art results for image super-resolution. Paying pa…
Study robust recovery of low-rank matrices from corrupted measurements without rank prior.
problem Robust recovery of low-rank matrices from corrupted Gaussian measurements with unknown rank.
method Subgradient method with diminishing stepsizes for nonconvex nonsmooth problem.
result Subgradient method converges to exact low-rank solution at sublinear rate under RDPP condition.
New algorithm for weighted low rank approximation with provable guarantees.
problem Weighted low rank approximation (WLRA) is computationally hard.
method Reweights the low rank solution using the weight matrix itself.
result Provably optimal approximation guarantees for WLRA.
Method estimates joint probability density from samples using low-rank decomposition and random projections.
problem Estimating joint probability density from limited samples.
method Low-rank tensor decomposition, dictionaries, and Radon transforms.
result Algorithm outperforms previous methods in estimating synthetic probability densities.
Low-rank Bayesian matrix completion via geodesic Hamiltonian Monte Carlo on Stiefel manifoldsstat.ML
New Bayesian matrix completion method using Stiefel manifolds.
problem Efficient Bayesian matrix completion with uncertainty quantification.
method Geodesic Hamiltonian Monte Carlo on Stiefel manifolds.
result Improved sampling performance and accuracy on real-world problems.
RNNs solve modular addition tasks using low rank and sparse Fourier structures.
problem Solving modular addition tasks with recurrent neural networks.
method Identified low rank structures and sparse Fourier representations in RNN weights.
result RNNs robust to removing individual frequencies but degrade with more ablation.
Paper develops an online EM algorithm for graph signal inference from streaming data.
problem Joint inference and clustering of graph signals with non-white excitation.
method Mixture model with low-rank plus sparse prior, online EM algorithm.
result Proposed online EM algorithm converges to MAP solution.
AIR-Net adapts low-rank regularization dynamically for better image completion.
problem Fixed low-rank regularization limits adaptability to different images.
method AIR-Net uses adaptive and implicit regularization parameterized by a dynamic Laplacian matrix.
result AIR-Net enhances implicit regularization and outperforms fixed methods in non-uniform missing data scenarios.
A new method for efficient neural network fine-tuning using queryable low-rank update atoms.
problem Rigidity of static low-rank adaptation methods when input and depth-wise computation vary.
method A shared queryable memory of low-rank update atoms, allowing dynamic and context-sensitive adaptation.
result Improves final test performance and training stability compared to standard low-rank adaptation.
New method for initializing low-rank neural networks improves performance.
problem Training low-rank neural networks efficiently and accurately.
method Inspired by function approximation, proposes a novel low-rank initialization framework.
result Demonstrates significant gap between spectral and low-rank initialization approaches.
Proposes a method to enhance multi-view learning by maximizing higher order correlations.
problem Losing intrinsic interconnections among multiple views in pairwise correlation maximization.
method Formulates multi-view data as a low rank approximation problem using higher order correlation tensor and solves it with the generating polynomial method.
result Consistently outperforms prior methods on real multi-view data.
PowerGossip compresses model differences for decentralized deep learning with low-rank linear compressors.
problem Communication bottleneck in decentralized deep learning models.
method Low-rank linear compressors applied on model differences using power iteration steps.
result Asymptotically independent of network and compression, faster convergence, and comparable performance to tuned compression algorithms.
Paper studies quantized LRMR with random dithering for correlated tasks.
problem Estimating coefficient matrix in quantized multivariate regression.
method Uniform quantization with random dithering, constrained and regularized Lasso estimators.
result Achieves minimax optimal rate with dithering, slightly worsens quantization effect.
Paper establishes limits for accurately estimating low-rank matrices from noisy, non-linear data.
problem Estimating low-rank matrices from noisy, non-linear observations.
method Proves strong universality result with equivalent Gaussian model and effective prior parameters.
result Signal-to-noise ratio requirement grows as $N^{ rac 12 (1-1/k_F)}$ for accurate reconstruction.
Novel prior for orthogonal functions improves functional component estimation.
problem Improving orthogonality in functional principal component analysis.
method Sequential adaptive priors for orthogonal functions using hierarchical conditionally normal distributions.
result Proposed prior leads to nearly orthogonal posterior estimates.
Clustering analysis by nonnegative low-rank approximations has achieved remarkable progress in the past decade. However, most approximation approaches in this direction are still restricted to matrix factorization. We propose a new low-rank learning method to improve the clustering performance, which is beyond matrix f…
LoLCATs improves linearized LLM quality with less memory and compute.
problem Linearizing large language models (LLMs) often degrades model quality and requires expensive training.
method Two-step method: attention transfer and low-rank adaptation.
result Significant improvement in linearizing quality with 20+ points on 5-shot MMLU.
Low-rank matrix estimation from incomplete measurements recently received increased attention due to the emergence of several challenging applications, such as recommender systems; see in particular the famous Netflix challenge. While the behaviour of algorithms based on nuclear norm minimization is now well understood…
Bayesian framework for sequential learning tasks with low-rank approximations.
problem Balancing knowledge retention and adaptability in sequential neural networks.
method Bayesian framework with diagonal plus low-rank approximations of the precision matrix.
result Unlocking capabilities to encode task relationships and incorporate prior knowledge from later tasks.
Deep neural networks as image priors have been recently introduced for problems such as denoising, super-resolution and inpainting with promising performance gains over hand-crafted image priors such as sparsity and low-rank. Unlike learned generative priors they do not require any training over large datasets. However…
We study a noisy tensor completion problem of broad practical interest, namely, the reconstruction of a low-rank tensor from highly incomplete and randomly corrupted observations of its entries. While a variety of prior work has been dedicated to this problem, prior algorithms either are computationally too expensive f…
3BASiL-TM decomposes LLMs into sparse and low-rank matrices for efficient compression.
problem Efficiently compressing large language models without significant performance loss.
method 3-Block ADMM method and transformer-matching refinement step for sparse plus low-rank decomposition.
result 3BASiL-TM reduces perplexity gap by over 30% and speeds up compression by 2.5x.
We develop an efficient algorithm for low-rank approximation with improved approximation guarantees.
problem Optimal low-rank approximation of matrices with norm constraints.
method Polynomial time column subset selection-based algorithm achieving -approximation.
result Improved approximation guarantees for low-rank approximation.
Low-Rank Covariance Completion for Graph Quilting with Applications to Functional Connectivitystat.ME
New method estimates neuronal connectivity from partially observed data.
problem Estimating neuronal connectivity from partially observed data.
method Two-step approach: low-rank covariance completion followed by graph structure estimation.
result Graph selection consistency demonstrated for one approach.
Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurementscs.LG
New algorithm recovers tensor factors from incomplete measurements efficiently.
problem Recovering tensor factors from incomplete measurements.
method Scaled gradient descent (ScaledGD) algorithm with spectral initializations.
result ScaledGD provably converges linearly for tensor completion and regression.
This work tackles representation learning for RL in low-rank MDPs, improving sample efficiency.
problem How to learn a compact representation for RL in low-rank MDPs efficiently.
method Proposes REP-UCB for online RL and develops an algorithm for offline RL under partial coverage.
result Significantly improved sample complexity for online RL and competitive performance for offline RL.
Unified framework HASSLE-free decomposes large model weights into sparse and low-rank components.
problem Efficiently compress large foundation models to reduce inference costs.
method Designs a unified framework for sparse plus low-rank matrix decomposition with a local layer-wise reconstruction error objective.
result HASSLE-free framework significantly outperforms state-of-the-art methods in compression and evaluation benchmarks.
DeepTensor uses deep networks to efficiently decompose tensors with improved performance and robustness.
problem Efficiently decomposing tensors with deep learning to capture nonlinear structures.
method Low-rank tensor decomposition using deep generative networks trained to minimize approximation error.
result DeepTensor outperforms classical methods like SVD and PCA in various applications, including image denoising and 3D MRI.
Unified framework for non-Euclidean CPD under scalable stochastic mirror descent.
problem Handling non-Euclidean losses in tensor decomposition.
method Tensor fiber sampling strategy-based stochastic mirror descent.
result Global convergence to a stationary point under reasonable conditions.
Algorithm for low-rank matrix bandits with heavy-tailed rewards, achieving nearly optimal regret bound.
problem Stochastic low-rank matrix bandit with heavy-tailed rewards.
method LOTUS algorithm using truncation and dynamic exploration.
result Regret bound of order $ ilde O(d^ rac{3}{2}r^ rac{1}{2}T^ rac{1}{1+δ}/ ilde{D}_{rr})$ without knowing .