arXiv research

The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.

This work improves robustness guarantees for neural networks using low rank representations.

LoRA fine-tuning explained with gradient dynamics for low-rank perturbations.

The higher order singular value decomposition (HOSVD) of tensors is a generalization of matrix SVD. The perturbation analysis of HOSVD under random noise is more delicate than its matrix counterpart. Recently, polynomial time algorithms have been proposed where statistically optimal estimates of the singular subspaces …

New algorithm improves deep learning models' robustness without sacrificing accuracy.

Proposes a low-rank PGD attack for more efficient adversarial training.

Robust tensor CP decomposition involves decomposing a tensor into low rank and sparse components. We propose a novel non-convex iterative algorithm with guaranteed recovery. It alternates between low-rank CP decomposition through gradient ascent (a variant of the tensor power method), and hard thresholding of the resid…

New method improves matrix completion accuracy, especially in noisy data.

New method estimates heterogeneous treatment effects with improved guarantees.

LSDAT reduces query efficiency for decision-based adversarial attacks.

LoRA fine-tuning causes forgetting, studied via particle system dynamics.

Gradient descent solves asymmetric low-rank matrix factorization efficiently.

We investigate the effect of the dimensionality of the representations learned in Deep Neural Networks (DNNs) on their robustness to input perturbations, both adversarial and random. To achieve low dimensionality of learned representations, we propose an easy-to-use, end-to-end trainable, low-rank regularizer (LR) that…

Algorithm learns latent simplex from perturbed points in input-sparsity time.

Efficiently compress pretrained models using RSI for improved predictive accuracy.

Study analyzes perturbations in singular subspaces under random noise.

State-of-the-art neural networks are vulnerable to adversarial examples; they can easily misclassify inputs that are imperceptibly different than their training and test data. In this work, we establish that the use of cross-entropy loss function and the low-rank features of the training data have responsibility for th…

Novel beamforming method reduces errors in wireless networks.

New CSC model extracts EEG signals with low noise sensitivity.

New technique stabilizes singular values in concatenated matrices.

Estimation of low-rank matrices is of significant interest in a range of contemporary applications. In this paper, we introduce a rank-one projection model for low-rank matrix recovery and propose a constrained nuclear norm minimization method for stable recovery of low-rank matrices in the noisy case. The procedure is…

Paper shows robustness of gradient descent in matrix sensing despite perturbations.

Matrix completion is the problem of recovering a low rank matrix by observing a small fraction of its entries. A series of recent works [KOM12,JNS13,HW14] have proposed fast non-convex optimization based iterative algorithms to solve this problem. However, the sample complexity in all these results is sub-optimal in it…

Analyzes learning dynamics of RNNs under locality constraints.

New method decomposes corrupted data matrices into sparse and low-rank components.

Novel tensor perturbation bounds for orthogonal iteration methods.

LORENZA improves LLM fine-tuning efficiency and generalization.

The Gumbel trick is a method to sample from a discrete probability distribution, or to estimate its normalizing partition function. The method relies on repeatedly applying a random perturbation to the distribution in a particular way, each time solving for the most likely configuration. We derive an entire family of r…

CSD learns a common component for domain generalization, outperforming existing methods.

New bounds for transfer learning in linear models, improving generalization.

Consider the problem of estimating a low-rank matrix when its entries are perturbed by Gaussian noise. If the empirical distribution of the entries of the spikes is known, optimal estimators that exploit this knowledge can substantially outperform simple spectral approaches. Recent work characterizes the asymptotic acc…

Active learning method for neural population dynamics using optogenetics.

TensorShield defends images from adversarial attacks using tensor decomposition.

We consider the matrix completion problem of recovering a structured matrix from noisy and partial measurements. Recent works have proposed tractable estimators with strong statistical guarantees for the case where the underlying matrix is low--rank, and the measurements consist of a subset, either of the exact individ…

Study on deep matrix factorization with Bures-Wasserstein loss, focusing on critical points and convergence.

This paper is concerned with the interplay between statistical asymmetry and spectral methods. Suppose we are interested in estimating a rank-1 and symmetric matrix $\mathbf{M}^{\star}\in \mathbb{R}^{n\times n}$, yet only a randomly perturbed version $\mathbf{M}$ is observed. The noise matrix $\mathbf{M}-\mathbf{M}^{\s…

The paper analyzes conditions for solving low-rank matrix recovery problems with noisy measurements.

Paper analyzes singular subspace estimation in noisy matrix models.

This paper studies least-square regression penalized with partly smooth convex regularizers. This class of functions is very large and versatile allowing to promote solutions conforming to some notion of low-complexity. Indeed, they force solutions of variational problems to belong to a low-dimensional manifold (the so…

Paper relaxes factor analysis for noisy data, improving robustness.

Pro-GNN defends graph neural networks from adversarial attacks by learning graph structure.

Machine learning has recently emerged as a fruitful area for finding potential quantum computational advantage. Many of the quantum enhanced machine learning algorithms critically hinge upon the ability to efficiently produce states proportional to high-dimensional data points stored in a quantum accessible memory. Eve…

New method closes certification gap for adversarially trained models.

We introduce a flexible framework for making inferences about general linear forms of a large matrix based on noisy observations of a subset of its entries. In particular, under mild regularity conditions, we develop a universal procedure to construct asymptotically normal estimators of its linear forms through double-…

New method discovers causal relationships in large-scale data.

Low-rank modeling generally refers to a class of methods that solve problems by representing variables of interest as low-rank matrices. It has achieved great success in various fields including computer vision, data mining, signal processing and bioinformatics. Recently, much progress has been made in theories, algori…

New graph convolution captures local features on non-Euclidean grids.

SGD with mini-batches can solve convex low-rank matrix problems efficiently.