Matrix completion is a problem that arises in many data-analysis settings where the input consists of a partially-observed matrix (e.g., recommender systems, traffic matrix analysis etc.). Classical approaches to matrix completion assume that the input partially-observed matrix is low rank. The success of these methods…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Low Rank Targets”
LEARNER improves low-rank matrix estimation using source population data.
problem Improving low-rank matrix estimation in target populations with diverse data sources.
method LEARNER uses similarity in latent spaces between source and target populations to enhance estimation.
result LEARNER often outperforms benchmark methods, especially with higher signal-to-noise ratios in the source population.
Implicit Regularization in Perturbed Deep Matrix Factorization: Spectral Conditions and Stabilitymath.OC
The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.
problem Stability of low-rank implicit regularization in perturbed deep matrix factorization.
method Derives spectral conditions for gradient descent to exhibit a low-rank phase in noiseless settings and analyzes perturbed dynamics.
result Gradient descent converges to a low-rank solution under perturbation, with explicit dependence on perturbation size.
LoRA enhances model adaptability without increasing parameters.
problem Theoretical understanding of LoRA's adaptability.
method Theoretical analysis of LoRA's expressive power.
result LoRA can adapt any model to a smaller target model with a specific rank.
CUR matrix decomposition is a randomized algorithm that can efficiently compute the low rank approximation for a given rectangle matrix. One limitation with the existing CUR algorithms is that they require an access to the full matrix A for computing U. In this work, we aim to alleviate this limitation. In particular, …
Low-Rank Plus Sparse Matrix Transfer Learning under Growing Representations and Ambient Dimensionscs.LG
The paper tackles transfer learning for growing matrix representations, improving estimation accuracy.
problem Structured matrix estimation under growing ambient dimensions and latent representations.
method Proposes a general transfer framework decomposing target parameters into embedded source components, low-rank innovations, and sparse edits. Develops an anchored alternating projection estimator.
result Establishes deterministic error bounds that separate target noise, representation growth, and source estimation error, yielding improved rates.
Matrix completion is a modern missing data problem where both the missing structure and the underlying parameter are high dimensional. Although missing structure is a key component to any missing data problems, existing matrix completion methods often assume a simple uniform missing mechanism. In this work, we study ma…
Language models exhibit low-rank structure, which can be used for generation.
problem Understanding the low-dimensional structure of large language models.
method Empirical demonstration and theoretical analysis of the approximate rank of language models' logits.
result Language models can generate responses using linear combinations of unrelated prompts.
This paper extends neural collapse to regression problems, revealing key features and structures.
problem Understanding the structure learned by deep neural networks in regression tasks.
method Established Neural Regression Collapse (NRC) across different models, analyzing feature and weight alignments.
result Deep neural regression models exhibit a collapsed feature space, aligning with target dimensions and covariances.
We accelerate the power method for strong low-rank approximation using fast sketching.
problem Efficiency bottleneck in power method for large target ranks.
method Developed an algorithmic and theoretical framework for accelerating the power method using fast sketching.
result Simple and provably efficient methods for singular value decomposition, low-rank factorization, and Nyström approximation.
New insights into attention mechanisms reveal dramatic trade-offs between rank and heads.
problem Dramatic trade-offs between rank and number of heads in attention mechanisms.
method Presented a simple target function and proved theoretical limits.
result Full-rank attention is necessary for long contexts, while low-rank is sufficient for short ones.
Study uncovers new phase transitions in asymmetric causal inference scenarios.
problem Understanding typical phase transitions in asymmetric causal inference.
method Combining Causal inference (C-inf) and Low-rank recovery (LRR) with Random duality - Free probability theory (RDT-FPT).
result Discovering a doubling low-rankness phenomenon in asymmetric scenarios.
Recovery of low-rank matrices has recently seen significant activity in many areas of science and engineering, motivated by recent theoretical results for exact reconstruction guarantees and interesting practical applications. A number of methods have been developed for this recovery problem. However, a principled meth…
Convex optimization method recovers low-rank matrices from rank-one projections efficiently.
problem Recovering low-rank matrices from limited rank-one projections.
method Unlifted convex optimization with subgradient method.
result The estimator succeeds with high probability if the number of measurements exceeds up to logarithmic factors.
Estimates low-rank distributional matrices from incomplete samples.
problem Matrix completion for distributional entries with limited observed data.
method Kernel mean embeddings, Tucker rank, functional unfolding operators.
result Effective estimator for distributional matrix completion established.
Develops PRPCA for smooth image recovery combining low-rank and smoothness.
problem Image matrix recovery under low-rank and smoothness assumptions.
method Projected Robust PCA (PRPCA) framework combining low-rank and smoothness.
result Explicit statistical guarantees for PRPCA, reducing matrix dimensionality.
Improves BBVI for high-dimensional Gaussian approximations by using low-rank approximations.
problem Scalability issues with BBVI for high-dimensional multivariate Gaussian approximations.
method Extends BaM framework to handle full covariance matrices by integrating patch step for low-rank parameterization.
result Shows improved efficiency and scalability on synthetic and real-world high-dimensional inference problems.
New method selects features via tensor decomposition and submodular optimization.
problem Feature selection for high-dimensional data.
method Low-rank tensor model, submodular optimization, greedy algorithm.
result Proposed method outperforms state-of-the-art feature selection.
We develop a flexible framework for low-rank matrix estimation that allows us to transform noise models into regularization schemes via a simple bootstrap algorithm. Effectively, our procedure seeks an autoencoding basis for the observed matrix that is stable with respect to the specified noise model; we call the resul…
This paper ranks pre-trained DNNs using a novel SI measure.
problem Optimizing pre-trained DNN selection for transfer learning.
method Automated ranking via Separation Index (SI) on target datasets.
result Ranked pre-trained DNNs improve classification performance.
CALDERA compresses large language models by approximating weight matrices with low-rank, low-precision factors.
problem The large sizes of Large Language Models (LLMs) hinder deployment on edge devices.
method CALDERA approximates weight matrices as , where and are low-rank factors quantized to low precision.
result CALDERA achieves better zero-shot performance than existing techniques, especially with low bit precision.
Algorithm compresses large matrices by approximating them as low rank and low precision factors.
problem Efficiently storing and processing large matrices with billions of elements.
method Randomized sketching and quantization of matrix columns to achieve low rank and low precision factorization.
result Achieves compression ratios as low as one bit per matrix coordinate while maintaining or improving performance.
Compressing DNNs is important for the real-world applications operating on resource-constrained devices. However, we typically observe drastic performance deterioration when changing model size after training is completed. Therefore, retraining is required to resume the performance of the compressed models suitable for…
We connect Causal inference and low-rank recovery via RDT and free probability theory.
problem Determining the applicability of causal inference via low-rank recovery.
method Random Duality Theory, free probability theory, and mathematical rigor.
result Exact closed-form worst case phase transitions for causal inference.
LOT improves optimal transport for large datasets.
problem Efficient optimal transport for large datasets.
method Low-rank optimal transport (LOT) restricts search to low-nonnegative rank couplings.
result LOT complements and improves upon entropic regularization.
LaLoRA prevents forgetting in LoRA fine-tuning.
problem Catastrophic forgetting in fine-tuned models.
method LaLoRA applies Laplace approximation to LoRA weights for regularization.
result Improved learning-forgetting trade-off with controllable regularization strength.
Unified framework for binary responses using AUC loss and low-rank constraint.
problem Statistical inefficiency and shared structure in fitting multiple binary responses.
method Pairwise AUC loss aggregation with low-rank constraint, scalable projected gradient descent.
result Unified framework outperforms likelihood-based approaches in challenging settings.
This paper proposes a new method for solving the well-known rank aggregation problem from pairwise comparisons using the method of low-rank matrix completion. The partial and noisy data of pairwise comparisons is transformed into a matrix form. We then use tools from matrix completion, which has served as a major compo…
Tensors play a central role in many modern machine learning and signal processing applications. In such applications, the target tensor is usually of low rank, i.e., can be expressed as a sum of a small number of rank one tensors. This motivates us to consider the problem of low rank tensor recovery from a class of lin…
We consider the decomposition of a data matrix assumed to be a superposition of a low-rank matrix and a component which is sparse in a known dictionary, using a convex demixing method. We consider two sparsity structures for the sparse factor of the dictionary sparse component, namely entry-wise and column-wise sparsit…
In this paper, we develop a relative error bound for nuclear norm regularized matrix completion, with the focus on the completion of full-rank matrices. Under the assumption that the top eigenspaces of the target matrix are incoherent, we derive a relative upper bound for recovering the best low-rank approximation of t…
New algorithm catches moving subspaces in bandit problems.
problem Adapt to changing low-dimensional latent subspaces in bandit settings.
method Piecewise-stationary low-rank linear contextual bandits with CUSUM-style boundary detection.
result Achieves intrinsic rank dynamic regret rate of .
A low-rank ensemble Kalman filter for elliptic observationsphysics.flu-dyn
A new EnKF method for elliptic PDEs reduces dimensionality for accurate state estimation.
problem Elliptic PDEs in fluid flows make traditional EnKF regularization ineffective.
method Low-rank factorization of the Kalman gain based on the Jacobian spectrum.
result Inference can be performed in a low-dimensional subspace of the state space.
A new method for decomposing non-negative tensors using energy-based modeling.
problem Challenges in traditional tensor decomposition methods, especially global optimization and rank selection.
method Energy-based modeling of tensors, considering interactions between modes for global optimization.
result Demonstrates effectiveness in tensor completion and approximation, revealing a relationship between many-body and low-rank approximations.
The paper introduces a penalized matrix estimation procedure aiming at solutions which are sparse and low-rank at the same time. Such structures arise in the context of social networks or protein interactions where underlying graphs have adjacency matrices which are block-diagonal in the appropriate basis. We introduce…
Alternating minimization represents a widely applicable and empirically successful approach for finding low-rank matrices that best fit the given data. For example, for the problem of low-rank matrix completion, this method is believed to be one of the most accurate and efficient, and formed a major component of the wi…
The Nystrom method has been popular for generating the low-rank approximation of kernel matrices that arise in many machine learning problems. The approximation quality of the Nystrom method depends crucially on the number of selected landmark points and the selection procedure. In this paper, we present a novel algori…
We study gradient compression methods to alleviate the communication bottleneck in data-parallel distributed optimization. Despite the significant attention received, current compression schemes either do not scale well or fail to achieve the target test accuracy. We propose a new low-rank gradient compressor based on …
Uncovering Locally Low-dimensional Structure in Networks by Locally Optimal Spectral Embeddingstat.ML
LASE improves local network structure visualization by targeting locally low-dimensional regions.
problem Global spectral embedding fails to capture local geometric features in sparse, transitive networks.
method Local Adjacency Spectral Embedding (LASE) using weighted spectral decomposition.
result LASE reveals locally low-dimensional structure, improving local reconstruction and visualization.
Wide adoption of complex RNN based models is hindered by their inference performance, cost and memory requirements. To address this issue, we develop AntMan, combining structured sparsity with low-rank decomposition synergistically, to reduce model computation, size and execution time of RNNs while attaining desired ac…
The paper analyzes how forgetting in LLMs is linked to simple task-upstream example associations.
problem Forgetting of upstream knowledge in fine-tuned LLMs.
method Empirical analysis of forgotten examples in upstream examples after new tasks, using low-rank matrix approximation.
result Forgetting can be predicted efficiently using matrix completion over empirical associations.
Enhances VAR model estimation using transfer learning.
problem Estimating high-dimensional VAR models with temporal dependencies.
method Transfer learning for VAR models with low-rank and sparse structures.
result Theoretical guarantees for model parameter consistency and informative set selection.
Proposes a method to correct for covariate shift in meta-analysis of randomized trials.
problem Invalidation of standard IPD meta-analysis due to covariate shift across studies.
method Placebo-anchored transport framework that treats source-trial outcomes as proxy signals and target-trial placebo outcomes as gold labels.
result Yields target-identified effect estimates in connected targets and a principled screen--then--transport procedure in disconnected targets.
The effectiveness of supervised learning techniques has made them ubiquitous in research and practice. In high-dimensional settings, supervised learning commonly relies on dimensionality reduction to improve performance and identify the most important factors in predicting outcomes. However, the economic importance of …
New compression methods handle biased input sequences for more accurate posterior summaries.
problem Handling biased input sequences for accurate posterior summaries.
method Stein kernel thinning, low-rank SKT, Stein recombination, Stein Cholesky.
result Achieves accurate posterior summaries with biased input sequences.
Rank-one measurements limit feasible sets for low-rank PSD matrices.
problem Feasibility of PSD matrices under rank-one measurements.
method Characterization of feasible sets for PSD matrices given rank-one projections.
result Radius of feasible sets determines singleton solution sets for low-rank matrices.
FedLoRU improves FL efficiency by using low-rank updates.
problem Communication inefficiency and performance reduction in Federated Learning.
method Proposes FedLoRU, a low-rank update framework for FL, which reduces communication costs while maintaining performance.
result FedLoRU achieves convergence rates similar to FedAvg and is robust to heterogeneous and large numbers of clients.
Low-rank modeling generally refers to a class of methods that solve problems by representing variables of interest as low-rank matrices. It has achieved great success in various fields including computer vision, data mining, signal processing and bioinformatics. Recently, much progress has been made in theories, algori…