arXiv research

Upper bound found for rank gradient using simplicial volume.

Stochastic gradient descent on manifolds improves low-rank approximation.

Riemannian gradient descent escapes some spurious critical points on low-rank matrix manifold.

CNNs trained by gradient descent can learn intrinsic image rank robustly to background noises.

Paper presents a rank-1 approximation method for natural policy gradients in deep RL.

PLUMAGE improves large model training efficiency and stability.

ScaledGD improves gradient descent for ill-conditioned low-rank matrix estimation.

New algorithm speeds up recovery of low-rank matrices.

We prove that if $M \longrightarrow P$ is a small cover of a compact right-angled hyperbolic polyhedron $P$ then $M$ admits a cofinal tower of finite sheeted covers with positive rank gradient. As a corollary, if $π_1(M)$ is commensurable with the reflection group of $P$, then $M$ admits a cofinal tower of finite sheet…

Survey of gradient boosting algorithms and their applications.

Gradient flow with infinitesimal initialization converges to Greedy Low-Rank Learning for matrix factorization.

The paper reveals low-rank structure in neural network gradients, influenced by data and model parameters.

Paper studies asymmetric matrix sensing, proving gradient descent converges to low-rank solutions.

We prove that if the fundamental group of an orientable finite volume hyperbolic 3-manifold has finite index in the reflection group of a right-angled ideal polyhedra in $\mathbb{H}^3$ then it has a co-final tower of finite sheeted covers with positive rank gradient. The manifolds we provide are also known to have co-f…

Paper proposes recycling model updates in federated learning by exploiting low-rank gradient subspaces.

The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.

Gradient descent recovers low-rank matrices from random rank-one measurements.

SGD with mini-batches can solve convex low-rank matrix problems efficiently.

A new model for supervised learning to rank using gradient estimation.

Paper develops methods for non-quadratic loss low-rank matrix recovery.

StochasticRank optimizes ranking metrics efficiently and guarantees global convergence.

ScaledGD accelerates ill-conditioned low-rank estimation.

Spectral gradient methods outperform Euclidean in certain deep learning scenarios.

Gradient descent in tensor factorization favors low-rank solutions.

Characterizes Kähler-hyperbolicity of bounded symmetric domains based on rank and genus.

Given a 3-manifold M with no spherical boundary components, and a primitive class φin H^1(M;Z), we show that the following are equivalent: (1) φis a fibered class, (2) the rank gradient of (M,φ) is zero, (3) the Heegaard gradient of (M,φ) is zero.

This study investigates how gradient-based methods bias neural networks trained on high-dimensional data.

Gradient optimization improves functional tensor-train regression performance.

RPG improves sample efficiency in RL by learning optimal action ranks.

LoRA fine-tuning explained with gradient dynamics for low-rank perturbations.

In this paper, we present GASG21 (Grassmannian Adaptive Stochastic Gradient for $L_{2,1}$ norm minimization), an adaptive stochastic gradient algorithm to robustly recover the low-rank subspace from a large matrix. In the presence of column outliers, we reformulate the batch mode matrix $L_{2,1}$ norm minimization with…

Gradient descent solves asymmetric low-rank matrix sensing without balancing.

Gradient descent biases towards stable rank networks for nearly-orthogonal data.

Deep networks learn clean structure before memorizing corrupted labels, leaving a spectral signature in gradient centered scatter.

PowerSGD compresses gradients for faster distributed optimization.

Gradient descent recovers low-rank matrices from corrupted measurements with double over-parameterization.

Distributed algorithm finds global solutions for low-rank matrices.

Efficient algorithm estimates low-rank matrices from noisy measurements.

The study shows rank gradients of Kaehler groups are zero if and only if their kernel is finitely generated.

ReLU networks implicitly favor low-rank solutions, but not as strongly as linear networks.

Gradient descent achieves exact linear convergence rate for symmetric matrix completion.

Gradient descent can find better tensor decompositions than lazy training in over-parameterized settings.

A1GM method improves efficiency in reconstructing missing data using KL divergence.

Unified framework for nonconvex low-rank matrix estimation using gradient descent.

Improved convergence for overparameterized low-rank matrix sensing.

Gradient descent and Adam differ in their treatment of low-rank solutions due to loss symmetry.

Gradient descent promotes low-rank solutions in tensor completion.

Paper proposes low-rank gradient approximation to save memory for deep neural network training.