arXiv research

Stochastic gradient descent on manifolds improves low-rank approximation.

Riemannian gradient descent escapes some spurious critical points on low-rank matrix manifold.

CNNs trained by gradient descent can learn intrinsic image rank robustly to background noises.

Paper presents a rank-1 approximation method for natural policy gradients in deep RL.

PLUMAGE improves large model training efficiency and stability.

We prove that if $M \longrightarrow P$ is a small cover of a compact right-angled hyperbolic polyhedron $P$ then $M$ admits a cofinal tower of finite sheeted covers with positive rank gradient. As a corollary, if $π_1(M)$ is commensurable with the reflection group of $P$, then $M$ admits a cofinal tower of finite sheet…

ScaledGD improves gradient descent for ill-conditioned low-rank matrix estimation.

We observe that stable integral simplicial volume of closed manifolds gives an upper bound for the rank gradient of the corresponding fundamental groups.

Gradient flow with infinitesimal initialization converges to Greedy Low-Rank Learning for matrix factorization.

The paper reveals low-rank structure in neural network gradients, influenced by data and model parameters.

Paper studies asymmetric matrix sensing, proving gradient descent converges to low-rank solutions.

We prove that if the fundamental group of an orientable finite volume hyperbolic 3-manifold has finite index in the reflection group of a right-angled ideal polyhedra in $\mathbb{H}^3$ then it has a co-final tower of finite sheeted covers with positive rank gradient. The manifolds we provide are also known to have co-f…

The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.

Paper proposes recycling model updates in federated learning by exploiting low-rank gradient subspaces.

SGD with mini-batches can solve convex low-rank matrix problems efficiently.

Paper develops methods for non-quadratic loss low-rank matrix recovery.

StochasticRank optimizes ranking metrics efficiently and guarantees global convergence.

Characterizes Kähler-hyperbolicity of bounded symmetric domains based on rank and genus.

ScaledGD accelerates ill-conditioned low-rank estimation.

Given a 3-manifold M with no spherical boundary components, and a primitive class φin H^1(M;Z), we show that the following are equivalent: (1) φis a fibered class, (2) the rank gradient of (M,φ) is zero, (3) the Heegaard gradient of (M,φ) is zero.

Spectral gradient methods outperform Euclidean in certain deep learning scenarios.

Gradient descent in tensor factorization favors low-rank solutions.

This study investigates how gradient-based methods bias neural networks trained on high-dimensional data.

LoRA fine-tuning explained with gradient dynamics for low-rank perturbations.

In this paper, we present GASG21 (Grassmannian Adaptive Stochastic Gradient for $L_{2,1}$ norm minimization), an adaptive stochastic gradient algorithm to robustly recover the low-rank subspace from a large matrix. In the presence of column outliers, we reformulate the batch mode matrix $L_{2,1}$ norm minimization with…

Gradient descent biases towards stable rank networks for nearly-orthogonal data.

Gradient descent solves asymmetric low-rank matrix sensing without balancing.

Deep networks learn clean structure before memorizing corrupted labels, leaving a spectral signature in gradient centered scatter.

Gradient descent recovers low-rank matrices from corrupted measurements with double over-parameterization.

We propose a generic framework based on a new stochastic variance-reduced gradient descent algorithm for accelerating nonconvex low-rank matrix recovery. Starting from an appropriate initial estimator, our proposed algorithm performs projected gradient descent based on a novel semi-stochastic gradient specifically desi…

ReLU networks implicitly favor low-rank solutions, but not as strongly as linear networks.

Gradient descent achieves exact linear convergence rate for symmetric matrix completion.

Gradient descent can find better tensor decompositions than lazy training in over-parameterized settings.

A1GM method improves efficiency in reconstructing missing data using KL divergence.

Improved convergence for overparameterized low-rank matrix sensing.

Gradient descent promotes low-rank solutions in tensor completion.

Paper proposes low-rank gradient approximation to save memory for deep neural network training.

Gradient descent solves rank-one matrix estimation problem with detailed time evolution analysis.

We propose a simple, scalable, and fast gradient descent algorithm to optimize a nonconvex objective for the rank minimization problem and a closely related family of semidefinite programs. With $O(r^3 κ^2 n \log n)$ random measurements of a positive semidefinite $n \times n$ matrix of rank $r$ and condition number $κ$…

Gradient descent with small random init mimics spectral methods for low-rank matrix recovery.

The paper shows how gradient flow on over-parametrized tensor decomposition behaves like deflation.

LORENZA improves LLM fine-tuning efficiency and generalization.

Gradient descent with preconditioning finds global optima in overparameterized nonconvex factorization.

Improved stability for matrix recovery from rank-one measurements.

IRMAE learns compact latent spaces by minimizing rank.

New approach to analyze matrix denoising using gradient flow and fixed point equations.

Proposes a cross entropy loss for better ranking algorithms.

Gradient descent solves asymmetric low-rank matrix factorization efficiently.