On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

3 results for “autorater”

Rich Insights from Cheap Signals: Efficient Evaluations via Tensor Factorizationcs.AI

Efficiently evaluate generative models at the prompt level using tensor factorization.

problem Fine-grained evaluations of generative models are costly and often misaligned with human judgment.

method Tensor factorization model that merges cheap autorater data with a small set of human gold-standard labels.

result The method provides accurate and tight confidence intervals for model performance.

tensor factorization generative models prompt evaluation autorater human gold-standard

2026-03-02abs ↗pdf ↗

Stratified Prediction-Powered Inference for Hybrid Language Model Evaluationcs.LG

StratPPI improves prediction-powered inference with stratified sampling.

problem Improving statistical estimates with limited human-labeled data.

method Combining small human-labeled data with large automatic-labeled data, stratifying data for tighter confidence intervals.

result StratPPI provides substantially tighter confidence intervals than unstratified approaches.

Prediction-powered inference Stratified sampling Confidence intervals Automatic labeling Language model evaluation

2024-06-06abs ↗pdf ↗

Valid Inference with Synthetic Data via Task Exchangeabilitystat.ME

Synthetic data can be used to ask more questions and accelerate discovery with provable validity guarantees.

problem Valid inference with synthetic data

method Task exchangeability

result Provable validity guarantees for synthetic data inference

synthetic data valid inference task exchangeability provable guarantees

2026-06-11abs ↗pdf ↗