On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

8 results for “AGOP”

Average gradient outer product as a mechanism for deep neural collapsecs.LG

AGOP mechanism explains deep neural collapse in neural networks.

problem Explaining the rigid structure of data representations in deep neural networks.

method Introducing AGOP and Deep RFM to demonstrate DNC.

result AGOP mechanism causes deep neural collapse in neural networks.

Deep Neural Collapse Deep Neural Networks Feature Learning Average Gradient Outer Product Deep Recursive Feature Machine

2024-02-21abs ↗pdf ↗

Catapults in SGD: spikes in the training loss and their impact on generalization through feature learningcs.LG

The paper explains spikes in training loss as catapults, improving feature learning and generalization.

problem Understanding and improving the training process of neural networks.

method Analysis of training loss spikes in SGD, empirical evidence of catapults, and demonstration of AGOP alignment.

result Catapults in training loss promote feature learning and better generalization.

catapults training loss SGD feature learning generalization

2023-06-07abs ↗pdf ↗

Average Gradient Outer Product in kernel regression provably recovers the central subspace for multi-index modelsstat.ML

AGOP from KRR recovers central subspace in fewer samples than needed for prediction.

problem Recovering low-dimensional structure in multi-index polynomial functions.

method Fit kernel ridge regression and compute AGOP from the fitted predictor.

result AGOP's top

r

eigenspace recovers the central subspace in

n \asymp d^{p+δ}

samples.

kernel ridge regression AGOP multi-index models central subspace sample efficiency

2026-05-14abs ↗pdf ↗

Mechanism of feature learning in convolutional neural networksstat.ML

New mechanism discovered for feature learning in CNNs.

problem Understanding how CNNs learn features from images.

method Proposed Convolutional Neural Feature Ansatz linking filter covariances to patch-based AGOPs.

result Deep ConvRFM algorithm learns features similar to deep CNNs, improving performance.

Convolutional Neural Networks Feature Learning CNNs Image Data Machine Learning

2023-09-01abs ↗pdf ↗

Linear Recursive Feature Machines provably recover low-rank matricesstat.ML

RFM reduces feature space for linear models, improving sparse recovery.

problem Sparse linear regression and low-rank matrix recovery.

method Recursive Feature Machines (RFM) that alternates between reweighting feature vectors by AGOP and learning prediction function.

result RFM generalizes IRLS and outperforms deep linear networks.

feature learning sparse recovery linear models matrix completion dimensionality reduction

2024-01-09abs ↗pdf ↗

Feature learning as alignment: a structural property of gradient descent in non-linear neural networksstat.ML

Gradient descent aligns neural feature matrices with pre-activation tangent features.

problem Understanding neural feature learning mechanisms.

method Analytical proof of alignment between weight matrices and pre-activation tangent features.

result Derivative alignment occurs almost surely in high-dimensional settings.

neural networks feature learning gradient descent alignment singular vectors

2024-02-07abs ↗pdf ↗

Emergence in non-neural models: grokking modular arithmetic via average gradient outer productstat.ML

Recursive Feature Machines show grokking in modular arithmetic without neural networks.

problem Grokking in modular arithmetic tasks.

method Recursive Feature Machines (RFM) with Average Gradient Outer Product (AGOP).

result RFM and neural networks learn block-circulant features to solve modular arithmetic.

Recursive Feature Machines Average Gradient Outer Product Grokking Modular Arithmetic Block-circulant Features

2024-07-29abs ↗pdf ↗

Breaking Data Symmetry is Needed For Generalization in Feature Learning Kernelsstat.ML

Breaking symmetry in training data is key for generalization in feature learning kernels.

problem Grokking in algebraic tasks, where models perform well on training but fail on unseen data.

method Used Recursive Feature Machine (RFM) with AGOP to learn task-relevant features, breaking symmetry in training data.

result Generalization occurs only when symmetry in the training set is broken, and RFM generalizes by recovering underlying invariance group action.

feature learning generalization symmetry breaking grokking algebraic tasks

2026-03-31abs ↗pdf ↗