Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,051 papers · 148 categories

Trend · papers per month

4080119159 · May 202619922001200920182026
48 results for BLEU Score

EBR improves NMT by re-ranking samples drawn from MLE-trained models.

problem Discrepancy between MLE and BLEU score in neural machine translation.
method Train an energy-based model to mimic BLEU score, then use it for re-ranking.
result EBR consistently improves NMT performance by +4 BLEU points on IWSLT'14 German-English.

Paper proposes a neural network for generating better questions from text.

problem Automatic generation of relevant questions from sentences and paragraphs.
method Adaptive copying recurrent neural network model with a copying mechanism added to a bidirectional LSTM architecture.
result The model outperforms state-of-the-art methods in question generation metrics.

Paper proposes metrics to evaluate both quality and diversity in text generation models.

problem Existing metrics only evaluate quality or diversity, not both.
method Approximate the distance between generative model and real data distribution using n-gram and BERT features.
result Proposed metrics better evaluate both quality and diversity of text generation models.

Improved machine translation with INT8 hardware using a novel training method.

problem Training accurate machine translation models with limited hardware precision.
method Convert all Transformer matrix multiplications to 8-bit integer (INT8) without sacrificing accuracy.
result INT8 Transformer models achieve BLEU scores 99.3% to 100% relative to FP32 models.

FullyQT quantizes Transformer models to 8-bit precision without sacrificing translation quality.

problem Reducing computational costs of neural machine translation methods without performance loss.
method Proposes FullyQT, an all-inclusive quantization strategy for the Transformer.
result Achieves state-of-the-art quantization results, maintaining or improving translation quality.

This paper compares log-likelihood and BLEU scores for sequence generation tasks.

problem The discrepancy between density estimation and sequence generation performance.
method Comparing several density estimators on five machine translation tasks.
result The correlation between log-likelihood and BLEU varies depending on model families.

Memory-augmented neural networks improve machine translation performance.

problem Improving machine translation accuracy and flexibility.
method Evaluation of Neural Turing Machines and Differentiable Neural Computers for machine translation tasks.
result Memory-augmented neural networks perform similarly to attentional encoders on Vietnamese to English tasks but have lower BLEU scores on Romanian to English tasks.

We introduce a new sampling method for large language models that balances diversity and parallelism.

problem Balancing diversity and parallelism in decoding for large language models.
method Arithmetic sampling framework compatible with various sampling variations.
result Improves estimation of expected BLEU score reward and reduces the gap with beam search.

Auto-sizing improves machine translation efficiency by 3.9 BLEU points with fewer parameters.

problem Optimizing neural network architecture and hyperparameters for low-resource machine translation is computationally expensive.
method Auto-sizing uses regularization to dynamically adjust network size during training.
result Auto-sizing improves BLEU scores by up to 3.9 points with one-third fewer parameters.

Differentiable sampling corrects alignment issues in neural machine translation.

problem Incorrect alignment of reference words and sampled output in scheduled sampling.
method Optimizes alignment probability based on model's soft alignment prediction.
result Improves BLEU score compared to maximum likelihood and scheduled sampling.

Generative model generates realistic text without reinforcement learning.

problem Training GANs for natural language processing is challenging due to discrete sequences.
method Autoencoder learns low-dimensional sentence representation, GAN generates vectors in this space.
result Model generates realistic text with competitive BLEU scores and human ratings.

Enhances persona-based conversation model for multi-turn dialogue.

problem Improving persona-based conversation models for multi-turn dialogue.
method Introduced additional input modality into hredGAN to capture external attributes.
result Persona hredGAN (phredGANphredGAN) outperforms existing models in multi-turn dialogue corpora.

Paper tackles zero-shot translation by encouraging consistent agreement in models.

problem Challenges of generalizing multilingual translation without parallel data.
method Reformulated as probabilistic inference, introduced consistent agreement-based training.
result Agreement-based learning improves zero-shot translation by 2-3 BLEU points.

This paper tackles rare word problem in low-resource language pairs using NMT.

problem Rare word problem in neural machine translation, especially for low-resource languages.
method Three solutions: enhanced source context, morphology learning, and wordnet synonyms.
result Significant improvements in BLEU scores (+1.0 points) on English-Vietnamese and Japanese-Vietnamese.

We present power low rank ensembles (PLRE), a flexible framework for n-gram language modeling where ensembles of low rank matrices and tensors are used to obtain smoothed probability estimates of words in context. Our method can be understood as a generalization of n-gram modeling to non-integer n, and includes standar…

2013-12-26abs ↗pdf ↗

Transforms web content for better visibility in AI-driven search engines.

problem Disruption of traditional SEO by generative AI search engines.
method Fine-tunes a BART-base transformer on synthetically generated training data.
result Significant improvements in ROUGE-L and BLEU scores, and substantial visibility gains in generative search responses.

Sequence to sequence learning has recently emerged as a new paradigm in supervised learning. To date, most of its applications focused on only one task and not much work explored this framework for multiple tasks. This paper examines three multi-task learning (MTL) settings for sequence to sequence models: (a) the onet…

2015-11-19abs ↗pdf ↗

We introduce Generative Neural Machine Translation (GNMT), a latent variable architecture which is designed to model the semantics of the source and target sentences. We modify an encoder-decoder translation model by adding a latent variable as a language agnostic representation which is encouraged to learn the meaning…

2018-06-13abs ↗pdf ↗

Efficiently improves non-autoregressive sequence models for better translation performance.

problem Heavy inference latency and inconsistent output sentences in non-autoregressive models.
method Incorporates a structured inference module with an efficient CRF approximation and dynamic transition technique.
result Significantly better translation performance (BLEU score 26.80) compared to previous non-autoregressive models.

Paper explores how text generation quality and diversity metrics relate to distribution fitting.

problem Unclear relation between text generation quality and diversity metrics and distribution fitting.
method Theoretical approach to prove a linear combination of quality and diversity metrics can be a divergence metric.
result CR/NRR proposed as a better substitute for BLEU/Self-BLEU metrics.

Transformers trained with three normalization changes outperform state-of-the-art on low-resource translation tasks.

problem Improving the training of Transformers, especially on low-resource datasets.
method Three normalization changes: PreNorm, ScaleNorm, and FixNorm.
result Significant improvements in BLEU scores on low-resource translation tasks.

We describe Sockeye (version 1.12), an open-source sequence-to-sequence toolkit for Neural Machine Translation (NMT). Sockeye is a production-ready framework for training and applying models as well as an experimental platform for researchers. Written in Python and built on MXNet, the toolkit offers scalable training a…

2017-12-15abs ↗pdf ↗

Our research extends the Bilingual Evaluation Understudy (BLEU) evaluation technique for statistical machine translation to make it more adjustable and robust. We intend to adapt it to resemble human evaluation more. We perform experiments to evaluate the performance of our technique against the primary existing evalua…

2015-09-30abs ↗pdf ↗

Improved text generation using transferable rewards from related tasks.

problem Non-differentiable task-specific scores limit the use of policy gradient methods in text generation.
method Transferable Reward Learner that uses model-based rewards for sentence-level and phrase-level similarity.
result Improved performance on semantic evaluation measures in image captioning tasks.

Transformer models outperform recurrent ones in modeling hierarchical data.

problem Modeling hierarchical structure in data.
method Introducing Multiresolution Transformer Networks leveraging self-attention.
result Multiresolution Transformer Networks significantly outperform state-of-the-art models on query suggestion datasets.

Paper aims to improve education online in South Africa using NMT for Setswana.

problem Inclusion of machine translation for low-resourced local languages in online education.
method Demonstrates state-of-the-art performance on English-to-Setswana translation using the Transformer architecture.
result Current NMT techniques show promise for African languages, beating previous methods by 5.33 BLEU points.

This study examines how the size and alignment of pretraining data affect the performance of large language models on downstream tasks.

problem Understanding how the size and alignment of pretraining data impact the performance of large language models on downstream tasks.
method Investigated the scaling behavior of large language models in a transfer learning setting, focusing on machine translation tasks.
result The size of the finetuning dataset and the distribution alignment between pretraining and downstream data significantly influence the scaling behavior of downstream performance.

This paper evaluates LLMs for technical market analysis, finding GPT-4 Turbo and FinGPT outperform passive benchmarks.

problem Evaluating LLMs for technical market analysis in financial markets.
method Structured evaluation of five LLMs (GPT-4 Turbo, Claude 3 Opus, Gemini 1.5 Pro, Llama 3 70B, FinGPT) on four tasks: candlestick pattern recognition, directional signal generation, backtesting, and financial report comprehension.
result GPT-4 Turbo and FinGPT outperform passive benchmarks in simulated backtesting, with GPT-4 Turbo achieving the highest annualized return and Sharpe ratio.

We present a recurrent encoder-decoder deep neural network architecture that directly translates speech in one language into text in another. The model does not explicitly transcribe the speech into text in the source language, nor does it require supervision from the ground truth source language transcription during t…

2017-03-24abs ↗pdf ↗