On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

3 results for “Anti-Regularization”

Convergence Analysis and Implicit Regularization of Feedback Alignment for Deep Linear Networkscs.LG

FA algorithm provides convergence guarantees for deep linear networks.

problem Training efficiency and convergence of deep neural networks.

method Theoretical analysis of Feedback Alignment (FA) algorithm for deep linear networks.

result Certain initializations lead to implicit anti-regularization, affecting learning effectiveness.

Feedback Alignment Deep Linear Networks Convergence Analysis Implicit Regularization Anti-Regularization

2021-10-20abs ↗pdf ↗

How Memory in Optimization Algorithms Implicitly Modifies the Losscs.LG

A technique identifies memoryless algorithms approximating memory-dependent optimization methods.

problem Understanding how memory in optimization algorithms affects loss and generalization.

method Introducing a general technique to replace past iterates with the current one and adding a correction term.

result Lion does not have the same implicit anti-regularization as AdamW, explaining its better generalization performance.

optimization algorithms memory deep learning generalization Lion

2025-02-04abs ↗pdf ↗

The Effect of Mini-Batch Noise on the Implicit Bias of Adamcs.LG

Adam optimizer's bias is influenced by mini-batch size and momentum hyperparameters.

problem Understanding how Adam's implicit bias is affected by mini-batch size and momentum parameters.

method Theoretical framework to analyze mini-batch noise's impact on Adam's memory and bias.

result The magnitude of anti-regularization by memory depends on batch size and momentum hyperparameters.

Adam optimizer mini-batch noise implicit bias multi-epoch training generalization gap

2026-02-02abs ↗pdf ↗