On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,982 papers · 148 categories

Trend · papers per month

count % of corpus

3 results for “BMUF”

SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcs.LG

SlowMo improves distributed SGD by periodically synchronizing and using momentum, leading to better accuracy.

problem Reducing communication overhead in distributed training while maintaining model accuracy.

method Periodically synchronizing workers and using momentum after multiple iterations of a base optimization algorithm.

result SlowMo consistently yields improvements in optimization and generalization performance relative to the base optimizer.

distributed optimization SGD momentum communication efficiency generalization

2019-10-01abs ↗pdf ↗

Lessons from Building Acoustic Models with a Million Hours of Speechcs.LG

Report on acoustic model building from 1M hours of unlabeled speech.

problem Building acoustic models from large amounts of unlabeled speech data.

method Student/teacher training, scheduled learning, BMUF, gradient threshold compression SGD.

result Extremely large amounts of unlabeled data lead to significant relative WER improvements.

acoustic models unlabeled speech student/teacher training BMUF distributed training

2019-04-02abs ↗pdf ↗

Federated Transfer Learning with Dynamic Gradient Aggregationcs.LG

Federated Learning speeds up speech recognition training by 7x and reduces error rate by 6%.

problem Training acoustic models for speech recognition tasks in a federated manner.

method Hierarchical optimization and dynamic gradient aggregation methods.

result Significant improvement in training convergence speed and model performance.

Federated Learning Speech Recognition Acoustic Models Gradient Aggregation Hierarchical Optimization

2020-08-06abs ↗pdf ↗