arXiv research

This paper uses LLMs and cycle consistency for better machine translation evaluation.

Paper explores why compositionality is not the right approach for emergent communication.

Expands MLM by masking token positions, improving performance and convergence.

Forward translation improves neural machine translation for sentences originally in source language.

CERT improves language understanding by contrastively learning sentence-level semantics.

This work explains how linear representations in large language models arise from training objectives and gradient descent.

Paper introduces TrufLL for language model training without labeled data.

GPT-f uses language models to find new proofs in formal math.

Improved language model for French clinical reports achieves state-of-the-art performance in medical NLP tasks.

Julia accelerates machine learning in various fields with balance of efficiency and simplicity.

Automates translating natural language to Verilog for digital design.

Study reveals neural scaling laws in random graphs and natural language models.

Recently, there has been interest in multiplicative recurrent neural networks for language modeling. Indeed, simple Recurrent Neural Networks (RNNs) encounter difficulties recovering from past mistakes when generating sequences due to high correlation between hidden states. These challenges can be mitigated by integrat…

An NMT system for Indic languages outperforms Google Translate.

Study finds phase transition in context-sensitive language model with short-range interactions.

Despite alarm over the reliance of machine learning systems on so-called spurious patterns, the term lacks coherent meaning in standard statistical frameworks. However, the language of causality offers clarity: spurious associations are due to confounding (e.g., a common cause), but not direct or indirect causal effect…

Generative Adversarial Networks (GANs) have been promising in the field of image generation, however, they have been hard to train for language generation. GANs were originally designed to output differentiable values, so discrete language generation is challenging for them which causes high levels of instability in tr…

Study builds a Japanese financial-specific LLM through continual pre-training.

ALIEN improves uncertainty estimation of language models by refining entropy-based methods.

The goal of homomorphic encryption is to encrypt data such that another party can operate on it without being explicitly exposed to the content of the original data. We introduce an idea for a privacy-preserving transformation on natural language data, inspired by homomorphic encryption. Our primary tool is {\em obfusc…

New method corrects quantization errors in LLMs using low-rank matrices.

Embodied cognition states that semantics is encoded in the brain as firing patterns of neural circuits, which are learned according to the statistical structure of human multimodal experience. However, each human brain is idiosyncratically biased, according to its subjective experience history, making this biological s…

We introduce a new sampling method for large language models that balances diversity and parallelism.

Deep learning uses alphabet frequencies to accurately classify fake news.

ResMem improves model generalization by explicitly memorizing residuals.

This work explains the structural origins of attention sinks in LLMs.

Improves drug properties using a novel LLM and reinforcement learning.

Paper addresses online alignment of large language models under uncertain preference feedback.

Recurrent neural networks trained on regular languages exhibit stable states that can recover from noise.

Method guarantees coherent factuality for language model outputs in reasoning tasks.

Paper refines cross-lingual word embeddings using Manhattan norm.

A new sparse benchmark metabench identifies key abilities from large benchmarks.

Study explains Zipf's law using geometric mechanisms from a finite alphabet.

End-to-end training of automated speech recognition (ASR) systems requires massive data and compute resources. We explore transfer learning based on model adaptation as an approach for training ASR models under constrained GPU memory, throughput and training data. We conduct several systematic experiments adapting a Wa…

BART pretrains sequence-to-sequence models by corrupting text and reconstructing it.

Tiny benchmarks reduce LLM evaluation costs by using fewer examples.

We introduce a method for using deep neural networks to amortize the cost of inference in models from the family induced by universal probabilistic programming languages, establishing a framework that combines the strengths of probabilistic programming and deep learning methods. We call what we do "compilation of infer…

The task of multi-step ahead prediction in language models is challenging considering the discrepancy between training and testing. At test time, a language model is required to make predictions given past predictions as input, instead of the past targets that are provided during training. This difference, known as exp…

TROLL improves RL for LLMs by replacing clipping with a trust region projection.

Researchers show how to secretly train models with hidden data, detect usage with high confidence.

StableLM 2 1.6B is a new language model series with detailed evaluations and performance metrics.

LLMs compress financial texts, but distort decision-making.

MARGE learns to reconstruct text by paraphrasing, achieving strong performance across multiple tasks.

Proposes RDASS for better Korean text summarization evaluation.

Layer fusion reduces deep neural network layers with minimal loss in accuracy.

ControlVAE improves VAE performance by adding a controller to tune hyperparameters.

Enhanced conformal methods improve validity of LLM outputs.

LLMs learn peaked distributions slowly due to power-law losses.