On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

12 results for “Clipped-SGD”

Near-Optimal Streaming Heavy-Tailed Statistical Estimation with Clipped SGDstat.ML

Improved Clipped-SGD achieves near-optimal heavy-tailed statistical estimation in streaming settings.

problem High-dimensional heavy-tailed statistical estimation in streaming with memory constraints.

method Stochastic convex optimization with Clipped-SGD, proving near-optimal sub-Gaussian statistical rates.

result Clipped-SGD achieves an error of

\sqrt{\frac{\mathsf{Tr}(Σ)+\sqrt{\mathsf{Tr}(Σ)\|Σ\|_2}\log(\frac{\log(T)}δ)}{T}}

with probability

1-δ

Clipped-SGD Heavy-tailed Streaming Stochastic Optimization Statistical Estimation

2024-10-26abs ↗pdf ↗

Robust and Fast Training via Per-Sample Clippingmath.OC

A new method improves training speed and robustness for deep learning models.

problem Training deep learning models with heavy-tailed gradient noise.

method Per-sample gradient clipping combined with SGD.

result PS-Clip-SGD achieves optimal convergence rates and high-probability guarantees.

deep learning gradient clipping SGD non-convex optimization CIFAR-100

2026-05-04abs ↗pdf ↗

Fast UCB-type algorithms for stochastic bandits with heavy and super heavy symmetric noisecs.LG

New UCB-type algorithms reduce regret bounds for stochastic bandits with heavy and super heavy noise.

problem Improving regret bounds for stochastic bandits with heavy-tailed noise.

method General convex optimization methods with an inexact oracle, Clipped-SGD-UCB algorithm.

result Achieved an

O(\log T\sqrt{KT\log T})

regret bound for symmetric noise, better than general lower bounds.

UCB algorithms stochastic bandits regret bounds heavy-tailed noise Clipped-SGD-UCB

2024-02-10abs ↗pdf ↗

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensionsstat.ML

Study on gradient clipping in SGD for high-dimensional problems.

problem Understanding and optimizing gradient clipping in high-dimensional machine learning models.

method Theoretical analysis of streaming SGD with gradient clipping in a least squares problem, focusing on large intrinsic dimensionality.

result Developed a deterministic equation to describe the loss evolution in clipped SGD, showing benefits under certain conditions.

gradient clipping SGD high-dimensional machine learning streaming

2024-06-17abs ↗pdf ↗

Clipped Gradient Methods for Nonsmooth Convex Optimization under Heavy-Tailed Noise: A Refined Analysismath.OC

Improved analysis for clipped gradient methods in nonsmooth convex optimization under heavy-tailed noise.

problem Optimization under heavy-tailed noise in nonsmooth convex problems.

method Refined analysis of Clipped Stochastic Gradient Descent (Clipped SGD) with new rates and improved utilization of Freedman's inequality.

result New rates

{\cal O}(σ_{\frak l}d_{ m eff}^{-1/2{\frak p}}\ln^{1-1/{\frak p}}(1/δ)T^{1/{\frak p}-1})

and

{\cal O}(σ_{\frak l}^2d_{ m eff}^{-1/{\frak p}}\ln^{2-2/{\frak p}}(1/δ)T^{2/{\frak p}-2})

for nonsmooth convex and strongly convex problems, respectively.

Clipped SGD Nonsmooth Convex Optimization Heavy-Tailed Noise Freedman's Inequality Generalized Effective Dimension

2025-12-29abs ↗pdf ↗

On the Convergence of DP-SGD with Adaptive Clippingcs.LG

This paper analyzes convergence of DP-SGD with adaptive quantile clipping.

problem Empirical success of adaptive clipping methods lacks theoretical understanding.

method Comprehensive convergence analysis of SGD with quantile clipping (QC-SGD).

result Establishes theoretical guarantees for DP-QC-SGD, revealing relationships between quantile selection, step size, and convergence.

differential privacy stochastic gradient descent quantile clipping convergence analysis adaptive clipping

2024-12-27abs ↗pdf ↗

Revisiting Gradient Clipping: Stochastic bias and tight convergence guaranteescs.LG

This paper improves convergence guarantees for gradient clipping in deep learning.

problem Improving convergence guarantees for gradient clipping in deep learning models.

method Analyzes and provides precise convergence guarantees for arbitrary clipping thresholds.

result Shows tight convergence guarantees for clipped stochastic gradient descent.

gradient clipping deep learning convergence guarantees stochastic gradient descent privacy

2023-05-02abs ↗pdf ↗

Online Heavy-tailed Change-point detectionstat.ML

New algorithm detects changes in heavy-tailed data streams.

problem Detecting changes in heavy-tailed data streams.

method Clipped Stochastic Gradient Descent (SGD) combined with union bound.

result First algorithm with finite-sample false-positive rate guarantees for heavy-tailed data.

change-point detection heavy-tailed data SGD false-positive rate union bound

2023-06-15abs ↗pdf ↗

Normalized/Clipped SGD with Perturbation for Differentially Private Non-Convex Optimizationcs.LG

Paper proposes DP-SGD and DP-NSGD for differentially private non-convex optimization.

problem Mitigating privacy risks in large model learning.

method Clip or normalize per-sample gradients and add noise for differential privacy.

result Achieved convergence rate of gradient norm for non-convex optimization.

differential privacy non-convex optimization gradient clipping noise addition convergence analysis

2022-06-27abs ↗pdf ↗

Why is Normalization Preferred? A Worst-Case Complexity Theory for Stochastically Preconditioned SGD under Heavy-Tailed Noisecs.LG

New theory explains why normalization is preferred in SGD under heavy-tailed noise.

problem Understanding why normalization is preferred in stochastic gradient descent (SGD) under heavy-tailed noise.

method Developed a worst-case complexity theory for stochastically preconditioned SGD and its variants.

result Normalization guarantees convergence at optimal rates, while clipping may fail in the worst case.

stochastic gradient descent heavy-tailed noise normalization clipping SGD convergence

2026-02-13abs ↗pdf ↗

A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networkscs.LG

A new algorithm reduces communication in deep learning training.

problem Slow convergence in deep neural networks due to exploding gradients.

method Communication-efficient distributed gradient clipping algorithm.

result Algorithm achieves linear speedup and reduced communication rounds.

deep learning distributed training gradient clipping SGD nonconvex optimization

2022-05-10abs ↗pdf ↗

Stochastic Weakly Convex Optimization Under Heavy-Tailed Noisesmath.OC

Paper analyzes convergence of stochastic methods under heavy-tailed noise.

problem Analyzing convergence of stochastic methods under heavy-tailed noise.

method Investigates vanilla and clipped stochastic subgradient descent methods.

result Demonstrates convergence properties under sub-Weibull and p-BCM noise assumptions.

Stochastic Optimization Weakly Convex Heavy-Tailed Noise Sub-Weibull p-BCM

2025-07-17abs ↗pdf ↗