On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

count % of corpus

← all fields·8 papers on “optimization” in Statistical ML · 1 month

Adaptive Runge-Kutta Step Control Buys Training Loss, Not Generalization: An Honest Compute-Matched Study of RK-Adam Optimizerscs.LG

A higher-order Runge-Kutta optimizer performs poorly compared to Adam when evaluated fairly.

problem Evaluating the performance of adaptive Runge-Kutta optimizers under strict conditions.

method Built and evaluated a representative Adam variant using a Bogacki-Shampine 3(2) RK pair, FSAL reuse, and local-error step control.

result The adaptive nature of the RK optimizer is illusory; it behaves like a fixed-step optimizer with gradient averaging.

Runge-Kutta optimization Adam training loss generalization

2026-07-16abs ↗pdf ↗

Thompson Sampling Is 2-Competitive for Mistakesstat.ML

Thompson Sampling is at most twice as bad as any other policy in Bayesian bandit models.

problem Optimizing selection of the best arm in Bayesian bandit models with independent latent processes.

method Thompson Sampling approach applied to models with independent latent arm processes.

result Thompson Sampling makes at most twice the expected number of mistakes compared to any other policy.

Thompson Sampling Bayesian Bandits Mistakes Independent Processes Optimization

2026-07-14abs ↗pdf ↗

Beyond Heuristic Tuning: Power-Calibrated LLM Watermarkingstat.ML

Develops a power-calibrated framework for LLM watermarking, optimizing tradeoffs between detectability and distortion.

problem The trade-off between detectability and semantic distortion in logit-based watermarking.

method Power-calibrated statistical framework for watermark hyperparameters, establishing explicit relationships.

result Derives practical parameter selection procedures achieving optimal tradeoffs under constraints.

watermarking LLM logit detectability semantic distortion

2026-07-06abs ↗pdf ↗

Curvature-Weighted Gradient Diversity: A Noise Measure for Geometry-Adaptive SGD Schedulescs.LG

CWGD measures gradient diversity weighted by curvature, improving SGD convergence.

problem Gradient noise in high-curvature directions is underestimated by standard methods.

method CWGD weights gradient diversity by the inverse square root of the Hessian.

result CWGD-Cosine reduces optimization error by up to 20% compared to standard cosine annealing.

gradient noise curvature SGD cosine annealing Hessian

2026-06-29abs ↗pdf ↗

A Stochastic--Geometric Theory of Scaling Laws in Grokkingstat.ML

Theoretical study explains grokking in neural networks.

problem Understanding the abrupt transition from fitting to generalizing in neural networks.

method Characterized a shell-core topological configuration of the solution space induced by Adam's optimization dynamics.

result Derived grokking scaling laws for learning rate, batch size, and regularization coefficient.

neural networks grokking shell-core topology Adam

2026-06-29abs ↗pdf ↗

Dead-Direction Conditioners: Gauge-Equivariant Preconditioning for Deep Networkscs.LG

A new optimizer DDC improves deep learning models by respecting symmetries.

problem Deep networks' loss is invariant to continuous symmetries, leading to optimization issues.

method DDC builds a Dead-Direction Conditioner that lifts a base optimizer into a G-equivariant one, preserving the quotient geometry.

result DDCAdam and DDCMuon outperform standard optimizers in various tasks, improving validation-train loss gaps and learning dynamics.

Deep Learning Optimization Symmetry Gauge Equivariance Adam

2026-06-28abs ↗pdf ↗

Training for the Model You Return: Improving Optimization for Iterate-Averaged Language Modelscs.LG

PACE optimizes training for averaged language models, improving performance.

problem How to optimize training for averaged language model iterates.

method Formulated as an optimal-control problem, solved for minimizing error of the average with a penalty on intervention size.

result PACE improves the limiting squared error of the iterate-average estimator by an arbitrarily large factor on some instances.

Language Models Optimization Iterate-Averaging Optimal Control AdamW

2026-06-23abs ↗pdf ↗

Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?cs.LG

Is AdamW effective under heavy-tailed noise?

problem Stochastic gradient noise in LLM pretraining is typically heavy-tailed.

method Formulate as an open problem, prove a positive weighted-metric benchmark, and give a corridor lower-bound mechanism.

result No rigorous convergence theory for AdamW established in heavy-tailed regime.

AdamW heavy-tailed noise convergence theory optimization

2026-06-22abs ↗pdf ↗