On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

7 results for “shuffle-once”

Computing the Variance of Shuffling Stochastic Gradient Algorithms via Power Spectral Density Analysismath.OC

The paper analyzes the variance of different shuffling methods in stochastic gradient descent.

problem Understanding the variance of different shuffling methods in stochastic gradient descent.

method Power spectral density analysis to study the noise sequences of stochastic gradients.

result The stationary variances of iterates decrease in the order of SGD, SGD-RR, and SGD-SO.

stochastic gradient descent random reshuffling shuffle-once power spectral density variance analysis

2022-06-01abs ↗pdf ↗

Random Reshuffling: Simple Analysis with Vast Improvementsmath.OC

RR algorithm improves convergence rate without strong convexity assumptions.

problem Minimizing finite-sum functions efficiently.

method Iterative gradient descent with data reshuffling, removing strong convexity assumptions.

result Improved convergence rate and variance for RR, matching lower bounds.

Random Reshuffling Convergence Rate Gradient Descent Strong Convexity Non-Convex Optimization

2020-06-10abs ↗pdf ↗

On the Last-Iterate Convergence of Shuffling Gradient Methodscs.LG

New convergence rates for shuffling gradient methods without strong convexity.

problem Theoretical gap between shuffling gradient methods' empirical success and established convergence rates.

method Proved last-iterate convergence rates for shuffling gradient methods using function value gap.

result First last-iterate convergence rates for shuffling gradient methods without strong convexity.

shuffling gradient methods last-iterate convergence function value gap empirical success theoretical guarantee

2024-03-12abs ↗pdf ↗

AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithmscs.LG

Unified analysis of asynchronous-SGD algorithms for distributed learning.

problem Analyzing asynchronous-SGD in heterogeneous settings with varying speeds and data distributions.

method Unified convergence theory for non-convex smooth functions, including pure asynchronous SGD and its modifications.

result Unified convergence rates for various asynchronous algorithms, including novel methods.

asynchronous-SGD distributed learning heterogeneous settings convergence analysis non-convex smooth functions

2023-10-31abs ↗pdf ↗

Sampling without Replacement Leads to Faster Rates in Finite-Sum Minimax Optimizationmath.OC

Sampling without replacement speeds up optimization in minimax problems.

problem Optimizing minimax problems with faster convergence rates.

method Analysis of gradient descent ascent and proximal point method with two sampling strategies.

result Sampling without replacement leads to faster convergence rates in minimax optimization.

minimax optimization sampling without replacement stochastic gradient convergence rates gradient descent ascent

2022-06-07abs ↗pdf ↗

Stochastic Extragradient with Random Reshuffling: Improved Convergence for Variational Inequalitiesmath.OC

Improved convergence for VIPs with SEG-RR, a variant of SEG with random reshuffling.

problem Solving variational inequality problems (VIPs) in machine learning.

method Stochastic Extragradient with Random Reshuffling (SEG-RR).

result SEG-RR achieves faster convergence rates than with-replacement variants for certain VIP classes.

Stochastic Extragradient Random Reshuffling Variational Inequalities Machine Learning Convergence Analysis

2024-03-11abs ↗pdf ↗

How Good is SGD with Random Shuffling?cs.LG

We study the performance of stochastic gradient descent (SGD) on smooth and strongly-convex finite-sum optimization problems. In contrast to the majority of existing theoretical works, which assume that individual functions are sampled with replacement, we focus here on popular but poorly-understood heuristics, which i…

2019-07-31abs ↗pdf ↗