On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

3 results for “ELECTRA”

Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscs.CL

Study recommends PLM choices for minimizing calibration error in NLP tasks.

problem Minimizing calibration error in PLM-based NLP predictions.

method Compared various options for PLM encoding, size, uncertainty quantifier, and fine-tuning loss.

result Recommendations for a well-calibrated PLM-based prediction pipeline.

uncertainty quantification pre-trained language models natural language processing calibration error ELECTRA

2022-10-10abs ↗pdf ↗

ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcs.LG

ProtTrans models predict protein features without evolutionary info.

problem Predicting protein features from amino acid sequences.

method Self-supervised deep learning on large protein datasets.

result ProtT5 embeddings outperform state-of-the-art for per-residue predictions.

protein prediction deep learning auto-regressive models protein subcellular localization protein secondary structure

2020-07-13abs ↗pdf ↗

ALIEN: Aligned Entropy Head for Improving Uncertainty Estimation of LLMscs.CL

ALIEN improves uncertainty estimation of language models by refining entropy-based methods.

problem Overconfidence in uncertainty estimation for language models, especially for difficult inputs.

method ALIEN refines entropy-based uncertainty by aligning it with prediction reliability, using a lightweight uncertainty head.

result ALIEN consistently outperforms strong baselines in detecting incorrect predictions and achieving the lowest calibration error.

uncertainty estimation language models entropy overconfidence downstream classification

2025-05-21abs ↗pdf ↗