On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,694 papers · 148 categories

Trend · papers per month

count % of corpus

3 results for “multi-teacher”

Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priorsstat.ME

Enhances model compression with multi-teacher knowledge distillation.

problem Uncertainty evaluation and diverse teacher expertise in model deployment.

method Bayesian inference and teacher-informed prior with entropy-based weighting.

result Improved predictive accuracy and robust uncertainty quantification.

knowledge distillation Bayesian inference multi-teacher teacher-informed prior entropy weighting

2026-05-27abs ↗pdf ↗

Distilling Knowledge from Ensembles of Acoustic Models for Joint CTC-Attention End-to-End Speech Recognitioncs.LG

This paper improves speech recognition by distilling knowledge from acoustic models.

problem Improving speech recognition accuracy using ensemble models.

method Proposes multi-teacher distillation strategies for joint CTC-attention end-to-end ASR systems, integrating error rate metric for optimization.

result Reports state-of-the-art error rates on various datasets and languages.

speech recognition end-to-end ASR knowledge distillation CTC-attention multi-teacher distillation

2020-05-19abs ↗pdf ↗

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directionscs.CL

Survey examines distillation methods for large language models.

problem Efficiently compress large language models while preserving their capabilities.

method Knowledge Distillation and Dataset Distillation techniques.

result Integrating KD and DD can produce more effective and scalable compression strategies.

Large Language Models Knowledge Distillation Dataset Distillation Model Compression Emergent Reasoning

2025-04-20abs ↗pdf ↗