On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

5 results for “AdaBelief”

AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradientscs.LG

AdaBelief optimizes deep learning models with faster convergence and better stability.

problem Combining fast convergence and stability in deep learning models.

method Adapts stepsize based on the belief in observed gradients using exponential moving average (EMA) of noisy gradients.

result AdaBelief outperforms other methods in image classification and GAN training, achieving comparable accuracy to SGD on ImageNet.

AdaBelief optimization deep learning convergence stability

2020-10-15abs ↗pdf ↗

FastAdaBelief: Improving Convergence Rate for Belief-based Adaptive Optimizers by Exploiting Strong Convexitycs.LG

FastAdaBelief improves convergence rate of AdaBelief by exploiting strong convexity.

problem Improving convergence rate of AdaBelief without sacrificing generalization ability.

method Designing FastAdaBelief that adjusts step size considering strong convexity.

result Proves

O(\log T)

regret bound for FastAdaBelief.

AdaBelief FastAdaBelief Convex Optimization Optimization Algorithms Generalization

2021-04-28abs ↗pdf ↗

A Control Theoretic Framework for Adaptive Gradient Optimizers in Machine Learningcs.LG

This paper improves Adam's performance in machine learning tasks.

problem Improving the generalization ability of adaptive gradient methods.

method Develops a control theoretic framework to propose AdamSSM, a new variant of Adam.

result AdamSSM improves generalization accuracy and convergence compared to recent adaptive gradient methods.

adaptive gradient methods control theory AdamSSM generalization convergence

2022-06-04abs ↗pdf ↗

AdamL: A fast adaptive gradient method incorporating loss functionstat.ML

AdamL optimizes deep learning models by incorporating loss function information.

problem Adaptive optimizers can suffer from poor generalization due to nonuniform gradient scaling.

method AdamL is a novel adaptive optimizer that considers loss function information for better generalization.

result AdamL achieves faster convergence or lower objective function values compared to other optimizers.

AdamL adaptive optimizer loss function generalization Polyak-Lojasiewicz inequality

2023-12-23abs ↗pdf ↗

EAdam Optimizer: How

ε

Impact Adamcs.LG

EAdam optimizer improves performance by adjusting ε in Adam.

problem Improving performance of Adam optimizer.

method Proposes EAdam optimizer by adjusting the position of ε in Adam.

result EAdam outperforms Adam on various tasks and models.

Adam optimizer EAdam optimizer ε adjustment deep learning performance improvement

2020-11-04abs ↗pdf ↗