OT domain adaptation improves aphasia detection across languages.
problem Detecting aphasia in low-resource languages with limited data.
method Utilized OT domain adaptation to map linguistic features across multiple languages.
result OT domain adaptation significantly improved F1 scores for French and Mandarin aphasia detection.
Global anchor method detects language shifts and domain adaptation.
problem Detecting corpus-level language shifts and domain adaptation.
method Global anchor method for comparing word embeddings.
result Global anchor method is superior to alignment method in applicability and implementation.
Real-time spell checker adapts to new languages.
problem No real-time, language-adaptable spell checkers for non-English languages.
method Used Wikipedia and subtitles data to generate dictionaries, created noisy channel datasets, compared with industry tools.
result System performs well across 24 languages, outperforming existing tools.
Structured pruning reduces large language model size and latency.
problem Reducing the size and latency of large language models.
method Parameterizing weights with low-rank factorization and adaptively removing rank-1 components.
result Structured pruning outperforms other methods at various compression levels and speeds up training and inference.
Paper introduces a conformer-based system for streaming language identification in long-form speech.
problem Language identification in long-form audio.
method Conformer layers with attentive temporal pooling and domain adaptation.
result Conformer-based models significantly outperform LSTM and transformer models.
LoRA-MCL improves language models by generating diverse sentence continuations.
problem Language models struggle with generating diverse, plausible sentence continuations.
method Low-Rank Adaptation combined with Multiple Choice Learning (MCL) to handle ambiguity.
result LoRA-MCL generates high-diversity and relevant outputs in various tasks.
New method reduces memory usage for adaptive optimization.
problem Significant memory overhead in adaptive optimizers like Adagrad and Adam.
method Memory-efficient adaptive optimization method that retains per-parameter adaptivity.
result Significantly larger models and batch sizes possible with up to 2-fold speedups.
New method improves RL/IL agents' adaptability to unseen environments.
problem Current RL/IL techniques struggle with generalizing to unseen environments.
method Zero-shot compositional policy learning with multi-modal fusion and attention mechanism.
result Language grounding enhances generalization across varied environments.
Language model benchmarks often misrepresent true understanding, revealing vulnerabilities in evaluation methods.
problem Language model benchmarks fail to accurately reflect true language understanding and adaptability.
method Systematic analysis of NLP evaluation frameworks, identifying vulnerabilities in static benchmarks, human evaluation protocols, and LLM-as-judge frameworks.
result Current evaluation methods are unreliable and need improvement to accurately assess LLM performance.
Sparse Meta Networks adapt deep neural networks incrementally for fast learning.
problem Training deep neural networks is slow and impractical for complex, changing environments.
method Sparse Meta Networks use a memory layer to learn online sequential adaptation, accumulating fast-weights incrementally.
result Sparse Meta Networks achieve strong performance in various sequential adaptation scenarios.
Paper tackles cold-start domain adaptation with language descriptions.
problem Cold-start domain adaptation failure with scarce target data.
method Leverages textual descriptions to learn preferences from LLM, integrates into EM algorithm.
result Framework guides source selection for weak target signals, improves as data accumulates.
Automates API mapping across languages with minimal manual effort.
problem Manual effort required to identify parallel program corpora and API mappings.
method Combines domain adaptation and code embedding with GANs to align vector spaces of APIs.
result Automatically identifies cross-language API mappings with less prior knowledge.
Adaptive contrastive search improves text generation quality and diversity.
problem Decoding high-quality text from language model outputs.
method Adaptive contrastive search incorporating uncertainty-guided degeneration penalty.
result Enhanced text generation quality and diversity across different models and datasets.
Geometric approach for unsupervised word embedding alignment.
problem Learning alignment between word embeddings of source and target languages.
method Formulates alignment as domain adaptation on the manifold of doubly stochastic matrices, employing Riemannian conjugate gradient algorithm.
result Empirically outperforms state-of-the-art methods on bilingual lexicon induction tasks.
Adaptive batch size schedules improve language model training efficiency and generalization.
problem Dilemma of choosing batch sizes in large-scale model training.
method General-purpose adaptive batch size schedules compatible with data and model parallelism.
result Adaptive batch size schedules outperform constant batch sizes and heuristic warmup schedules.
ConMeZO speeds up zeroth-order optimization for large language models.
problem Slow convergence in high-dimensional parameter spaces of large language models.
method Adaptive directional sampling in a cone centered around a momentum estimate.
result Achieves the same convergence rate as MeZO but up to 2X faster.
A simple modification enables a universal NMT model with language-specific parameters.
problem Creating a universal NMT model that can adapt to different languages and domains.
method Introducing a contextual parameter generator (CPG) that dynamically adjusts model parameters based on source and target language embeddings.
result The system achieves state-of-the-art performance and zero-shot translation, demonstrating the effectiveness of the CPG.
Work shows hallucination detection by LLMs is impossible without expert feedback.
problem Detecting hallucinations in LLMs is theoretically impossible without expert-labeled feedback.
method Investigated hallucination detection using a theoretical framework inspired by language identification.
result Automated hallucination detection is impossible for most language collections without expert-labeled feedback.
LoRA-One uses one-step full gradient to align adapters for efficient large model fine-tuning.
problem Fine-tuning large language models efficiently and accurately.
method Properly initializing LoRA adapters using the one-step full gradient and incorporating preconditioners.
result LoRA-One achieves significant empirical improvements over existing methods.
DS-CP improves reliability of uncertainty quantification for large language models under domain shift.
problem Overconfident and factually incorrect outputs (hallucinations) from large language models.
method Adapts conformal prediction to large language models under domain shift by reweighting calibration samples.
result DS-CP delivers more reliable coverage than standard conformal prediction, especially under substantial distribution shifts.
In this paper, we attempt to improve Statistical Machine Translation (SMT) systems on a very diverse set of language pairs (in both directions): Czech - English, Vietnamese - English, French - English and German - English. To accomplish this, we performed translation model training, created adaptations of training sett…
HS-MoE selects sparse experts using adaptive priors and data-adaptive gating.
problem Sparse expert selection in mixture-of-experts architectures.
method Combines horseshoe prior with input-dependent gating for data-adaptive sparsity.
result Data-adaptive sparsity in expert usage.
BLoB fine-tunes LLMs with Bayesian methods to improve uncertainty estimation.
problem Overconfidence in LLMs during inference for domain-specific tasks.
method Continuous Bayesian low-rank adaptation during fine-tuning.
result BLoB improves generalization and uncertainty estimation in LLMs.
FinHEAR combines LLMs with human expertise for better financial decision-making.
problem Challenges in financial decision-making for language models.
method Multi-agent framework with specialized LLMs for historical analysis, event interpretation, and expert retrieval.
result FinHEAR outperforms baselines in financial tasks with higher accuracy and risk-adjusted returns.
Generative Adapter adapts LMs with a single forward pass, reducing inference overhead.
problem Efficient adaptation of large language models for new contexts.
method Generative Adapter directly maps new contexts to low-rank LM adapters via self-supervised learning.
result Significant reduction in inference overhead with no need for fine-tuning.
Adaptive querying learns user psychometrics with AI personas.
problem Learning user psychometrics within query budgets.
method Persona-induced latent variable model with AI personas and large language model response distributions.
result Persona-based posteriors deliver accurate probabilistic predictions.
New method uses fewer parameters to match state-of-the-art performance on multiple natural language tasks.
problem Efficiently adapting BERT for multiple tasks with fewer parameters.
method PALs (projected attention layers) for shared BERT model with task-specific parameters.
result Matches state-of-the-art performance on GLUE benchmark with 7 times fewer parameters.
New study finds best language model architecture and pretraining objective for zero-shot tasks.
problem Evaluating which language model architectures and pretraining objectives best enable zero-shot generalization.
method Compared three model architectures and two pretraining objectives across 170 billion tokens, with and without finetuning.
result Causal decoder-only models trained on autoregressive language modeling exhibit strongest zero-shot generalization.
Adaptive testing segments watermarked text from LLMs.
problem Distinguishing LLM-generated text from human-written content.
method Generalized likelihood-based detection method adapted to inverse transform sampling, removing prompt estimation sensitivity.
result Effective and robust method for segmenting watermarked text.
Adaptive attention span extends Transformer's context size.
problem Limited context size in Transformers limits model performance.
method A self-attention mechanism that learns optimal attention span.
result State-of-the-art performance on character-level language modeling.
ParaDRAM automates parallel MCMC simulations across languages.
problem Efficiently sampling complex mathematical functions.
method Automated, parallel, high-performance MCMC with adaptive rejection.
result Significant memory reduction for complex simulations.
Adaptive scheduling improves multilingual neural machine translation models.
problem Training models on multiple tasks with uniform or proportional sampling leads to poor performance trade-offs.
method Exploring non-adaptive and adaptive task scheduling, including implicit schedules.
result Adaptive schedules improve model performance for low-resource tasks without negatively affecting high-resource tasks.
System learns to combine multiple model components for personalized text generation.
problem Adapting and biasing language models for personal preferences.
method Combines model-defined components, learns activation and probability combination from unlabeled text.
result Directly generates text with personalized components from unlabeled data.
Adaptive neural network adjusts layer depth based on computational needs.
problem Deciding the number of layers in deep recurrent neural networks is challenging.
method Proposes a layer flexible recurrent neural network with adaptive computation time.
result Improves performance by 7\% to 12\% on financial and language data.
Improved IVON boosts LoRA model accuracy and calibration.
problem Improving the accuracy and calibration of large language models.
method Replaced AdamW with IVON for finetuning Llama-2.
result IVON improves Llama-2 accuracy by 2.8% and calibration error by 4.6%.
The Widrow-Hoff rule simplifies language data simulation.
problem Simulating language phenomena computationally.
method Implementation and application of the Widrow-Hoff rule.
result The Widrow-Hoff rule offers new perspectives on language simulation.
BoRA finetunes multi-task LLMs by sharing information through hierarchical priors.
problem Limited data for some tasks in multi-task LLMs.
method Bayesian hierarchical low-rank adaption.
result BoRA outperforms individual and unified model approaches.
End-to-end training of automated speech recognition (ASR) systems requires massive data and compute resources. We explore transfer learning based on model adaptation as an approach for training ASR models under constrained GPU memory, throughput and training data. We conduct several systematic experiments adapting a Wa…
Improved multilingual speech recognition with low latency for nine Indic languages.
problem Imbalance in training data across languages and low latency in interactive applications.
method Conditioning on a language vector and training language-specific adapter layers.
result Lower word error rate than monolingual E2E models and conventional systems.
Improved Polish language model using subword tokenization.
problem Adapting ULMFiT for high inflection languages like Polish.
method Subword tokenization to adapt ULMFiT for Polish.
result First place in PolEval'18, 35% improvement over second best.
New method reduces gender bias in language models without harming performance.
problem Bias in language models learned from biased data.
method Causal analysis to identify problematic model components, followed by linear projection of weight matrices.
result DAMA significantly decreases bias in language models while maintaining performance.
Paper introduces NumLLM for better financial text understanding with numeric variables.
problem Poor performance of existing financial large language models in numeric financial text.
method Constructed financial corpus, fine-tuned with LoRA modules, merged into foundation model.
result NumLLM achieves best performance on financial question-answering benchmark, especially with numeric questions.
This paper benchmarks FinGPT for financial datasets using open-source large language models.
problem Challenges in integrating GPT-based models with financial datasets.
method Instruction Tuning paradigm for open-source large language models adapted for financial contexts.
result Demonstrates the effectiveness and adaptability of FinGPT in financial tasks.
Seq2seq ASR adapts to speakers, improving performance by 25%.
problem Speaker adaptation for seq2seq ASR systems to match conventional methods.
method Applied Kullback-Leibler divergence and Linear Hidden Network adaptation to seq2seq models.
result 25% relative word error rate improvement with seq2seq model adaptation.
Proposes a multilingual email segmentation benchmark and model.
problem Lack of multilingual email zoning corpora and models.
method Analysis of existing corpora, development of multilingual benchmark, introduction of OKAPI model.
result OKAPI model achieves state-of-the-art performance in English and generalizes well to unseen languages.
Learned feature representations and sub-phoneme posteriors from Deep Neural Networks (DNNs) have been used separately to produce significant performance gains for speaker and language recognition tasks. In this work we show how these gains are possible using a single DNN for both speaker and language recognition. The u…
Improved language model for French clinical reports achieves state-of-the-art performance in medical NLP tasks.
problem Lack of specialized language models for French clinical reports.
method Adapted a general pre-trained language model (CamemBERT) to French clinical reports using a corpus of 21M reports.
result Pretrained and fine-tuned models improved F1-score by 3 percentage points on APMed task.
Deep neural networks have excelled on a wide range of problems, from vision to language and game playing. Neural networks very gradually incorporate information into weights as they process data, requiring very low learning rates. If the training distribution shifts, the network is slow to adapt, and when it does adapt…