arXiv research

The paper proposes using low rank assumption to improve causal structure learning in DAGs.

Matrix approximation is a common tool in machine learning for building accurate prediction models for recommendation systems, text mining, and computer vision. A prevalent assumption in constructing matrix approximations is that the partially observed matrix is of low-rank. We propose a new matrix approximation model w…

FLAMBE tackles RL in low rank MDPs by learning features.

Develops PRPCA for smooth image recovery combining low-rank and smoothness.

This work studies low-rank approximation of a positive semidefinite matrix from partial entries via nonconvex optimization. We characterized how well local-minimum based low-rank factorization approximates a fixed positive semidefinite matrix without any assumptions on the rank-matching, the condition number or eigensp…

Recovery of low-rank matrices from a small number of linear measurements is now well-known to be possible under various model assumptions on the measurements. Such results demonstrate robustness and are backed with provable theoretical guarantees. However, extensions to tensor recovery have only recently began to be st…

As opposed to manual feature engineering which is tedious and difficult to scale, network representation learning has attracted a surge of research interests as it automates the process of feature learning on graphs. The learned low-dimensional node vector representation is generalizable and eases the knowledge discove…

We provide new approximation guarantees for greedy low rank matrix estimation under standard assumptions of restricted strong convexity and smoothness. Our novel analysis also uncovers previously unknown connections between the low rank estimation and combinatorial optimization, so much so that our bounds are reminisce…

The problem of low-rank matrix estimation recently received a lot of attention due to challenging applications. A lot of work has been done on rank-penalized methods and convex relaxation, both on the theoretical and applied sides. However, only a few papers considered Bayesian estimation. In this paper, we review the …

New method solves nonsmooth low-rank matrix optimization problems efficiently.

New methods rank players using covariates and comparisons, outperforming existing algorithms.

Novel method for efficient low-rank matrix estimation and bandit algorithms.

Weight Decay induces low-rank weight matrices in neural networks, improving generalization.

Low-rank tensor regression, a new model class that learns high-order correlation from data, has recently received considerable attention. At the same time, Gaussian processes (GP) are well-studied machine learning models for structure learning. In this paper, we demonstrate interesting connections between the two, espe…

New nonconvex regularizer speeds up low-rank matrix completion.

DM2L tackles missing labels in multi-label learning by modeling local and global rank structures.

New method uses low logit rank to simplify complex language models.

This paper improves matrix completion by leveraging element importance and non-uniform sampling.

Unified approach tackles high-dimensional tensor bandits with convex optimization and weakly decomposable regularizers.

The paper establishes theoretical foundations for low-rank knowledge distillation in LLMs.

Many applications require recovering a ground truth low-rank matrix from noisy observations of the entries, which in practice is typically formulated as a weighted low-rank approximation problem and solved by non-convex optimization heuristics such as alternating minimization. In this paper, we provide provable recover…

The paper reveals low-rank structure in neural network gradients, influenced by data and model parameters.

This paper considers the problem of estimating a low-rank matrix from the observation of all or a subset of its entries in the presence of Poisson noise. When we observe all entries, this is a problem of matrix denoising; when we observe only a subset of the entries, this is a problem of matrix completion. In both case…

In this paper, we develop a novel procedure for low-rank tensor regression, namely \emph{\underline{I}mportance \underline{S}ketching \underline{L}ow-rank \underline{E}stimation for \underline{T}ensors} (ISLET). The central idea behind ISLET is \emph{importance sketching}, i.e., carefully designed sketches based on bot…

Reduced-rank method improves least-squares regression under output regularity.

Paper develops new patterns for unique matrix completions.

IMPACT optimizes LLM compression by focusing on activation importance, reducing model size up to 55.4%.

Study reward-free RL in non-linear settings, improving efficiency and removing assumptions.

New method learns shared structures in non-linear tasks.

The Nystrom method is an efficient technique used to speed up large-scale learning applications by generating low-rank approximations. Crucial to the performance of this technique is the assumption that a matrix can be well approximated by working exclusively with a subset of its columns. In this work we relate this as…

We consider a challenging multi-label classification problem where both feature matrix $\X$ and label matrix $\Y$ have missing entries. An existing method concatenated $\X$ and $\Y$ as $[\X; \Y]$ and applied a matrix completion (MC) method to fill the missing entries, under the assumption that $[\X; \Y]$ is of low-rank…

Gradient descent achieves exact linear convergence rate for symmetric matrix completion.

Estimates joint probability distribution from 1-way marginals using low-rank tensors and random projections.

Unified error analysis for low-rank approximation improves data assimilation performance.

Characterizes uncertainty in low-rank matrix completion with noisy data.

New method learns low-dimensional representations of nonlinear time series without supervision.

Matrix completion works well for smooth non-linear structures, even without low-rank assumptions.

SOLVAR efficiently analyzes cryo-EM data's structural variability.

Paper proposes efficient optimizers for large language models with fast convergence and low memory usage.

We develop an efficient algorithm for low-rank approximation with improved approximation guarantees.

This paper considers the problem of completing a matrix with many missing entries under the assumption that the columns of the matrix belong to a union of multiple low-rank subspaces. This generalizes the standard low-rank matrix completion problem to situations in which the matrix rank can be quite high or even full r…

Study compares LRMC algorithms under dependent sampling in various applications.

LoRA fine-tuning explained with gradient dynamics for low-rank perturbations.

The effectiveness of supervised learning techniques has made them ubiquitous in research and practice. In high-dimensional settings, supervised learning commonly relies on dimensionality reduction to improve performance and identify the most important factors in predicting outcomes. However, the economic importance of …

Flat minima lead to better generalization in low-rank matrix recovery models.

We propose a new method to estimate Wasserstein distances and optimal transport plans between two probability distributions from samples in high dimension. Unlike plug-in rules that simply replace the true distributions by their empirical counterparts, our method promotes couplings with low transport rank, a new struct…

New method estimates heterogeneous treatment effects with improved guarantees.

We present a novel algebraic combinatorial view on low-rank matrix completion based on studying relations between a few entries with tools from algebraic geometry and matroid theory. The intrinsic locality of the approach allows for the treatment of single entries in a closed theoretical and practical framework. More s…