On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

count % of corpus

8 results for “Schedule-Free”

ScheduleFree+: Scaling Learning-Rate-Free & Schedule-Free Learning to Large Language Modelscs.LG

ScheduleFree+ improves large language model training without schedules or learning rates.

problem Scaling up Schedule-Free Learning to large language models.

method Learning-rate-free and schedule-free method for training large language models.

result ScheduleFree+ outperforms SOTA schedules by 31% at 1000 tokens per parameter.

Schedule-Free Learning Large Language Models Learning Rate Model Averaging Checkpoint Merging

2026-05-18abs ↗pdf ↗

General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimizationcs.LG

Schedule-free SGD is optimal for nonconvex optimization problems.

problem Nonconvex optimization in neural networks.

method Developed a general framework for online-to-nonconvex conversion, which converts schedule-free SGD into an effective nonconvex optimization algorithm.

result Schedule-free SGD achieves optimal iteration complexity for nonsmooth, nonconvex optimization problems.

nonconvex optimization schedule-free SGD online-to-nonconvex conversion neural networks

2024-11-11abs ↗pdf ↗

Anytime Training with Schedule-Free Spectral Optimizationcs.LG

New optimizer SF-NorMuon matches tuned AdamW across various horizons.

problem Fixed learning-rate schedules in neural network training lead to strong path dependence and costly re-tuning.

method Schedule-Free Spectral Optimization (SF-NorMuon)

result SF-NorMuon outperforms tuned AdamW on 125M and 772M parameter models across different horizons.

Neural Networks Optimization Learning Rate Schedule-Free Spectral Dynamics

2026-05-21abs ↗pdf ↗

The Road Less Scheduledcs.LG

New method schedules learning rate without stopping time, outperforming existing methods.

problem Learning rate schedules that require stopping time are outperformed.

method Schedule-Free approach that avoids stopping time and introduces no additional hyper-parameters.

result Exhibits state-of-the-art performance across various problems.

learning rate scheduling optimization deep learning schedule-free

2024-05-24abs ↗pdf ↗

Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Trainingcs.LG

New method SF-AdamW trains large models without decay phases or memory overhead.

problem Inadequate fixed compute budgets for large-scale training.

method Schedule-Free (SF) method revisited and refined.

result SF-AdamW effectively navigates loss landscape without decay phases or memory overhead.

language models training schedule-free flexibility empirical performance

2025-07-14abs ↗pdf ↗

Smoothing DiLoCo with Primal Averaging for Faster Training of LLMscs.LG

GPA improves LLM training speed by 8.71% for Llama-160M models.

problem Training Large Language Models (LLMs) with high memory overhead and slow convergence.

method Generalized Primal Averaging (GPA) extends Nesterov's method to eliminate memory-intensive two-loop structure.

result GPA achieves up to 10.13% speedup over AdamW in training Llama-1B model.

Large Language Models Training Speed Memory Overhead Nesterov's Method Optimization

2025-12-18abs ↗pdf ↗

Accelerating Neural Network Training: An Analysis of the AlgoPerf Competitioncs.LG

AlgoPerf competition evaluates neural network training speed-ups.

problem Improving neural network training speed using better algorithms.

method Compared 18 diverse submissions from 10 teams on multiple workloads.

result Schedule Free AdamW algorithm achieved best results in self-tuning ruleset.

neural networks training algorithms AlgoPerf competition hyperparameter-free non-diagonal preconditioning

2025-02-20abs ↗pdf ↗

Muon Does Not Converge on Convex Lipschitz Functionscs.LG

Muon fails to converge on convex Lipschitz functions.

problem Understanding the convergence of Muon on convex and Lipschitz functions.

method Analyzing Muon's performance on convex and Lipschitz functions without error feedback.

result Muon does not converge on convex and Lipschitz functions, regardless of learning rate schedule.

Muon Convex Lipschitz Convergence Error Feedback

2026-05-09abs ↗pdf ↗