On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

4 results for “Accuracy-on-the-line”

Are Domain Generalization Benchmarks with Accuracy on the Line Misspecified?cs.LG

Current OOD benchmarks overestimate model robustness to spurious correlations.

problem Spurious correlations degrade OOD performance, but benchmarks show the opposite.

method Analyze OOD datasets for spurious correlations and derive conditions for robustness.

result Current OOD benchmarks are misspecified and overestimate model robustness.

OOD benchmarks spurious correlations robustness model evaluation distribution shifts

2025-03-31abs ↗pdf ↗

Accuracy on the wrong line: On the pitfalls of noisy data for out-of-distribution generalisationcs.LG

Accuracy on in-distribution data correlates with out-of-distribution data when data is noisy or contains nuisance features.

problem Correlation between in-distribution and out-of-distribution accuracy in noisy or feature-rich data.

method Analyzes the impact of noise and nuisance features on model performance.

result Accuracy on in-distribution and out-of-distribution data can become negatively correlated in noisy or feature-rich data.

Accuracy-on-the-line Accuracy-on-the-wrong-line Noisy data Nuisance features Out-of-distribution

2024-06-27abs ↗pdf ↗

Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcs.LG

The paper shows strong correlation between in-distribution and out-of-distribution performance in various machine learning models.

problem Understanding reliability of machine learning systems in unseen environments.

method Empirical analysis of various models and distribution shifts on CIFAR-10, ImageNet, and other datasets.

result Out-of-distribution performance is strongly correlated with in-distribution performance across different models and distribution shifts.

machine learning generalization out-of-distribution in-distribution CIFAR-10

2021-07-09abs ↗pdf ↗

Deconstructing Distributions: A Pointwise Framework of Learningcs.LG

A new framework evaluates model performance on single input points, revealing insights into data and model structure.

problem Traditional evaluation methods in machine learning are insufficient for understanding model performance and data structure.

method Developed a pointwise framework to measure model performance on individual input points, analyzing the relationship between pointwise and average performance.

result Profiles of data points reveal different types of correlations between pointwise and average performance, challenging existing models of learning.

machine learning model evaluation pointwise performance data structure in-distribution

2022-02-20abs ↗pdf ↗