On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

7 results for “2 1.6B”

Stable LM 2 1.6B Technical Reportcs.CL

StableLM 2 1.6B is a new language model series with detailed evaluations and performance metrics.

problem Improving language models with smaller sizes and better performance.

method Detailed data and training procedure for StableLM 2 1.6B, including zero- and few-shot benchmarks and multilingual evaluations.

result StableLM 2 1.6B is the state-of-the-art open model under 2B parameters.

language model StableLM 2 1.6B training procedure evaluation

2024-02-27abs ↗pdf ↗

Towards a Human-like Open-Domain Chatbotcs.CL

Meena is a chatbot trained on social media data, achieving human-like conversation quality.

problem Creating a chatbot that can have human-like conversations in an open-domain setting.

method End-to-end training of a 2.6B parameter neural network on social media data, using perplexity and a human evaluation metric (SSA) to assess quality.

result Meena achieves a high human-like conversation quality (79% SSA) when compared to existing chatbots.

chatbot open-domain end-to-end training social media perplexity

2020-01-27abs ↗pdf ↗

Prototype Language Modelscs.LG

Prototype model improves model auditing and understanding.

problem Auditing and understanding modern language models is expensive and approximate.

method Introduced a sparse, non-negative mixture of learned prototypes trained with clustering objectives.

result Prototype models either surpass or remain within 2.5 percentage points of dense baselines on downstream tasks.

language models model auditing sparse prototypes clustering objectives downstream accuracy

2026-07-01abs ↗pdf ↗

Analysis of Railway Accidents' Narratives Using Deep Learningcs.CL

Automatic understanding of domain specific texts in order to extract useful relationships for later use is a non-trivial task. One such relationship would be between railroad accidents' causes and their correspondent descriptions in reports. From 2001 to 2016 rail accidents in the U.S. cost more than $4.6B. Railroads i…

2018-10-17abs ↗pdf ↗

Logarithmic-time Schedules for Scaling Language Models with Momentumstat.ML

Logarithmic-time schedules boost large-scale language model training efficiency.

problem Improving performance and efficiency in large-scale language model training.

method Designing time-varying hyperparameters

(β_1, β_2, λ)

for AdamW, specifically logarithmic-time scheduling with damping mechanisms.

result ADANA optimizer achieves up to 40% compute efficiency compared to tuned AdamW, with gains persisting as model scale increases.

language models AdamW logarithmic-time scheduling performance gains compute efficiency

2026-02-05abs ↗pdf ↗

MiMuon: Mixed Muon Optimizer with Improved Generalization for Large Modelscs.LG

MiMuon optimizer improves generalization for large models by reducing generalization error.

problem Improving generalization of Muon optimizer for large models.

method Enhanced Muon optimizer using orthogonalization of gradient, proving lower generalization error.

result MiMuon optimizer has a lower generalization error of

O\big(\frac{1}{N}\big)

compared to Muon's

O\big(\frac{1}{Nκ^{T}}\big)

Muon optimizer generalization error large models matrix parameters algorithmic stability

2026-05-19abs ↗pdf ↗

Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routersmath.OC

New optimizer designs respect symmetry, improving deep learning models.

problem Optimizers lack respect for symmetry in neural networks.

method Introduce symmetry-compatible principle for optimizer design.

result Symmetry-compatible optimizers improve model performance.

symmetry equivariance optimization deep learning neural networks

2026-05-18abs ↗pdf ↗