Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

count% of corpus
1223 · Oct 202519922001200920172026
8 results for “Scion”
Error Feedback for Muon and Friendscs.LG

EF21-Muon optimizes deep learning with error feedback, improving efficiency and accuracy.

problem Lack of principled distributed frameworks for non-Euclidean LMO-based optimizers.
method Introduces EF21-Muon, a communication-efficient, non-Euclidean LMO-based optimizer with convergence guarantees.
result First efficient distributed implementation of non-Euclidean LMO-based optimizers, achieving up to 7x communication savings.
error feedbackdistributed optimizationnon-Euclidean LMOsMuonScion
2025-10-01abs ↗pdf ↗
Optimal Scaling Needs Optimal Normcs.LG

Optimal scaling found to depend on operator norm across large models and datasets.

problem Lack of unifying principle for optimal hyperparameter scaling across models and datasets.
method Discovered that optimal scaling is conditioned on the operator norm of the output layer.
result The optimal learning rate/batch size pair (η∗,B∗)(η^{\ast}, B^{\ast}) consistently has the same operator norm value.
optimal scalinghyperparameter transferoperator normAdam optimizerScion optimizer
2025-10-04abs ↗pdf ↗
Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)cs.LG

Gluon optimizes LMO-based methods for large-scale tasks, improving performance and theory-practice gap.

problem LMO-based methods lack theoretical support for practical implementation and smoothness assumptions.
method Introduces Gluon, a new LMO-based method with refined smoothness model.
result Gluon's theoretical stepsizes match fine-tuned values, closing the theory-practice gap.
LMO optimizationGluonLarge-scale tasksPractical implementationSmoothness model
2025-05-19abs ↗pdf ↗
Generalized Gradient Norm Clipping & Non-Euclidean (L0,L1)(L_0,L_1)-Smoothnesscs.LG

New optimization method combines gradient clipping and non-Euclidean smoothness.

problem Improving optimization in non-Euclidean spaces for machine learning.
method Hybrid of steepest descent and conditional gradient, incorporating weight decay.
result Achieves optimal convergence rate and demonstrates effectiveness in deep learning.
Gradient ClippingNon-Euclidean SmoothnessFrank-Wolfe AlgorithmDeep LearningImage Classification
2025-06-02abs ↗pdf ↗
On the Role of Batch Size in Stochastic Conditional Gradient Methodscs.LG

New analysis reveals batch size effects on stochastic conditional gradient methods.

problem Understanding the role of batch size in stochastic conditional gradient methods.
method Deriving a new analysis focusing on momentum-based stochastic conditional gradient algorithms (e.g., Scion).
result Increasing batch size initially improves optimization accuracy but can degrade performance beyond a critical threshold.
batch sizestochastic conditional gradientKurdyka-Łojasiewicz conditionstepsizeempirical practices
2026-03-22abs ↗pdf ↗
Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Successmath.OC

Muon optimizes Transformer training with heavy-tailed data, achieving optimal sample complexity.

problem Theoretical understanding of non-Euclidean optimisation methods for heavy-tailed data in training Transformers.
method Addressing the gap in theoretical understanding, we show Muon achieves optimal sample complexity under heavy-tailed noise.
result Muon finds an ε-stationary point in nuclear norm with optimal sample complexity, absorbing heavy-tailed noise without dimension dependence.
Transformer trainingMuonHeavy-tailed dataOptimal sample complexityNuclear norm
2026-06-12abs ↗pdf ↗