Sockeye is an open-source toolkit for neural machine translation.
problem Improving Neural Machine Translation (NMT) models and techniques.
method Scalable training and inference for three NMT architectures, including attentional, self-attentional, and fully convolutional networks.
result Sockeye achieves competitive BLEU scores across different NMT architectures, including a best score for its transformer implementation.
In this paper, we study a new learning paradigm for Neural Machine Translation (NMT). Instead of maximizing the likelihood of the human translation as in previous works, we minimize the distinction between human translation and the translation given by an NMT model. To achieve this goal, inspired by the recent success …
Paper examines NMT robustness to nonsensical inputs.
problem NMT systems fail when source sentences are altered.
method Soft-attention technique to replace words in source sentences.
result Proposed technique achieves high success rate and outperforms existing methods.
Dynamic segmentation algorithm improves NMT performance by favoring character-level processing.
problem Suboptimal static segmentation choices in NMT systems.
method Adaptive Computation Time algorithm for dynamic segmentation, trainable end-to-end.
result The model prefers character-level processing when given the freedom to navigate different segmentation levels.
Study improves calibration of NMT models, especially EOS and attention uncertainty.
problem Improper functioning of beam-search inference due to miscalibrated NMT models.
method Design and apply recalibration methods based on EOS and attention uncertainty signals.
result Improved accuracy and better sequence-level calibration of NMT models.
Improves NMT with user feedback from eBay ratings and search tasks.
problem Improving neural machine translation quality with user feedback.
method Offline bandit learning of NMT parameters using real user feedback from eBay.
result Implicit task-based feedback from cross-lingual search tasks improves NMT quality.
EBR improves NMT by re-ranking samples drawn from MLE-trained models.
problem Discrepancy between MLE and BLEU score in neural machine translation.
method Train an energy-based model to mimic BLEU score, then use it for re-ranking.
result EBR consistently improves NMT performance by +4 BLEU points on IWSLT'14 German-English.
Paper uses NMT to predict solutions to stochastic optimization problems quickly.
problem Predicting solutions to stochastic discrete optimization problems under uncertainty.
method Applied a state-of-the-art NMT algorithm with minimal adaptations and hyperparameter tuning.
result NMT can produce accurate solutions in milliseconds with less variability.
Study finds optimal vocabulary size for neural machine translation.
problem Imbalanced class distribution in language data affects NMT performance.
method Casts NMT as a classification task, analyzes vocabulary sizes, and tests multiple languages.
result Certain vocabulary sizes outperform others, explaining NMT performance.
Improves NMT performance on diverse datasets without forgetting.
problem Catastrophic forgetting in NMT models on heterogeneous datasets.
method Non-parametric n-gram level retrieval combined with expressive neural network.
result Gains on all evaluation sets on a heterogeneous dataset.
Tensor Train layer improves BLEU scores in NMT models.
problem Improving Neural Machine Translation (NMT) models' performance.
method Implemented Tensor Train layer in TensorFlow for NMT training.
result Higher learning rates and more 'rectangular' core dimensions improve BLEU scores.
An NMT system for Indic languages outperforms Google Translate.
problem Challenges in translating Indic languages efficiently.
method Encoder-decoder with attention mechanism for neural machine translation.
result Outperforms Google Translate with a 6 BLEU score margin on English-Gujarati translation.
This paper proposes a curriculum learning framework for NMT to reduce training time and improve performance.
problem Slow training and need for heuristics in NMT systems.
method A curriculum learning framework that decides training samples based on estimated difficulty and model competence.
result Up to 70% decrease in training time and up to 2.2 BLEU accuracy improvements.
This paper tackles rare word problem in low-resource language pairs using NMT.
problem Rare word problem in neural machine translation, especially for low-resource languages.
method Three solutions: enhanced source context, morphology learning, and wordnet synonyms.
result Significant improvements in BLEU scores (+1.0 points) on English-Vietnamese and Japanese-Vietnamese.
Paper aims to improve education online in South Africa using NMT for Setswana.
problem Inclusion of machine translation for low-resourced local languages in online education.
method Demonstrates state-of-the-art performance on English-to-Setswana translation using the Transformer architecture.
result Current NMT techniques show promise for African languages, beating previous methods by 5.33 BLEU points.
This study improves NMT using reinforcement learning, overcoming its instability.
problem Stability issues in reinforcement learning for neural machine translation.
method Systematic study on reinforcement learning factors and a new method for monolingual data.
result Competitive results on WMT17 Chinese-English translation task, setting a state-of-the-art performance.
This paper improves neural machine translation training by selecting and denoising data.
problem Reduces negative impact of noisy data on neural machine translation training.
method Measures and selects domain data, applies denoising curriculum using online data selection.
result Significant effectiveness for training on noisy data.
Toolkit simplifies transfer learning for researchers.
problem Making transfer learning accessible to non-experts.
method Wraps 17 models, provides easy-to-use interfaces.
result Simplifies model selection for real-world applications.
CytonRL is an open-source toolkit for reinforcement learning.
problem Improving efficiency and performance in reinforcement learning.
method Implemented four deep Q-learning algorithms in C++ with GPU acceleration.
result Achieves competitive performance on Atari games.
A simple modification enables a universal NMT model with language-specific parameters.
problem Creating a universal NMT model that can adapt to different languages and domains.
method Introducing a contextual parameter generator (CPG) that dynamically adjusts model parameters based on source and target language embeddings.
result The system achieves state-of-the-art performance and zero-shot translation, demonstrating the effectiveness of the CPG.
Proposes a method to generate diverse translations by conditioning on target domain.
problem NMT models lack diversity in translations, even with search algorithms.
method Condition the decoder on a latent variable representing target domain, generated by a target encoder.
result Generated diverse translations without affecting performance or training time.
In this paper, we present Neural Phrase-based Machine Translation (NPMT). Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a recently proposed segmentation-based sequence modeling method. To mitigate the monotonic alignment requirement of SWAN, we introduce a new …
A toolkit for learning Hawkes processes simplifies research and education.
problem Analyzing asynchronous event sequences with self- and mutually-triggering patterns.
method Implementation of learning algorithms and analysis tools for Hawkes processes.
result Systematic compilation of state-of-the-art and classic Hawkes process algorithms.
Paper reviews and synthesizes methods for evaluating dimensionality reduction techniques.
problem Evaluating and comparing dimensionality reduction techniques.
method Framework and toolkit in R for exploring and evaluating dimensionality reduction quality through visual insights.
result Helps researchers compare and select dimensionality reduction techniques using visual insights.
A toolkit for path-norms enhances neural network generalization bounds.
problem Establishing generalization bounds for modern neural networks.
method Introducing a comprehensive toolkit for path-norms in ReLU networks with various operations.
result Established generalization bounds for modern neural networks that are the most widely applicable and recover/beat the sharpest known bounds.
Toolkit evaluates causal inference models for real-world data.
problem Validating causal inference models for observational data.
method Developed a comprehensive evaluation toolkit for machine learning models in causal inference.
result Improves reproducibility and ensures data extraction criteria drive model selection.
Toolkit and taxonomy for diverse AI explainability methods.
problem Diverse stakeholder needs for AI explanations.
method Open-source software toolkit with eight explainability methods and evaluation metrics.
result Taxonomy helps navigate explanation methods.
sktime toolkit benchmarks time series classification algorithms for correctness and efficiency.
problem Benchmarking correctness and efficiency of time series classification algorithms.
method Implementation and comparison of six classifiers in sktime with their tsml equivalents.
result Significant differences in accuracy and efficiency between algorithms, with one causing debugging issues.
Dynamic neural network toolkits such as PyTorch, DyNet, and Chainer offer more flexibility for implementing models that cope with data of varying dimensions and structure, relative to toolkits that operate on statically declared computations (e.g., TensorFlow, CNTK, and Theano). However, existing toolkits - both static…
We describe DyNet, a toolkit for implementing neural network models based on dynamic declaration of network structure. In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then exampl…
New toolkit for directed distances improves flexibility of OT problems.
problem Optimal transport problems with constraints.
method Directed distances between quantile functions.
result Flexibility in solving OT problems enhanced.
Jensen simplifies machine learning and optimization with an extensible toolkit.
problem Complex machine learning and optimization tasks in production environments.
method Develops a framework for convex functions and optimization algorithms, enabling easy deployment and extension.
result Jensen allows for quick model deployment and extension with minimal code, making machine learning accessible.
Improves NMT by sampling context from predicted sequence during training.
problem Error accumulation and overcorrection in NMT due to mismatched training and inference contexts.
method Samples context words from both ground truth and predicted sequences during training.
result Significant improvements on multiple datasets, including Chinese->English and WMT'14 English->German.
Flexible tree ensemble learning framework supports arbitrary loss functions and multi-task learning.
problem Limited modeling capabilities of existing tree ensemble learning toolkits.
method Differentiable tree ensembles with tensor-based formulation for efficient training.
result Our framework leads to 100x more compact and 23% more expressive tree ensembles.
CAT is a new ASR toolkit using CRF and CTC for state-of-the-art speech recognition.
problem Improving automatic speech recognition systems.
method CRF-based discriminative training with CTC-inspired state topology.
result CAT achieves state-of-the-art results with fewer parameters and is competitive with hybrid models.
CAT toolkit combines hybrid and E2E approaches for efficient speech recognition.
problem Improving speech recognition efficiency and latency.
method CTC-CRF based framework with contextualized soft forgetting.
result CAT achieves state-of-the-art results with simpler training and streaming ASR.
GluonCV and GluonNLP simplify deep learning for CV and NLP.
problem Lack of easy-to-use deep learning tools for CV and NLP.
method Developed modular APIs and pre-trained models for rapid prototyping.
result Facilitates rapid prototyping and reproducible research in CV and NLP.
Study on RL from human bandit feedback for sequence-to-sequence learning, showing reliability and learnability.
problem Reliability and learnability of human bandit feedback for RL from sequence-to-sequence learning.
method Investigated reliability of human bandit feedback, analyzed influence on reward estimator learnability, and tested improvements with regression-based reward estimator.
result Improvements of over 1 BLEU can be achieved by integrating a regression-based reward estimator trained on cardinal feedback into RL for NMT.
Python toolkit for symbolic Poisson geometry calculations.
problem Computing Poisson-Nijenhuis structures on manifolds.
method Symbolic algorithms implemented in Python.
result Examples of gauge transformations and parametric bivector fields.
CODE2SEQ generates natural language sequences from code snippets.
problem Generating natural language descriptions from code.
method CODE2SEQ represents code as AST paths and uses attention to select relevant paths.
result CODE2SEQ outperforms previous models for code-to-text tasks.
Sparklen is a Python toolkit for high-dimensional Hawkes processes.
problem Efficiently analyzing high-dimensional Hawkes processes.
method Combines Python for ease of use with C++ for performance.
result Provides state-of-the-art tools for estimating and classifying Hawkes processes.
HIVE-COTE v1.0 improves time series classification with enhanced usability.
problem Improving time series classification accuracy and usability.
method Presented a walkthrough guide and extensive experimental evaluation of HIVE-COTE v1.0.
result HIVE-COTE v1.0 outperforms three recently proposed algorithms in predictive performance and resource usage.
MOGPTK simplifies multi-channel data modeling with Gaussian processes.
problem Modeling multi-channel data efficiently and accurately.
method Python package with TensorFlow backend, supporting various GP kernels and parameter initialization strategies.
result Enables GPU-accelerated training and comprehensive GP modeling pipeline.
TensorFlow Probability MCMC toolkit improves MCMC efficiency for modern hardware.
problem Inefficient MCMC algorithms on modern hardware.
method Design and implementation of a new MCMC toolkit for TensorFlow.
result Improved MCMC efficiency on modern hardware.
HuSpaCy offers an industrial-grade Hungarian NLP toolkit.
problem Lack of suitable open-source Hungarian NLP pipelines.
method Built on spaCy, HuSpaCy includes lemmatization, morphosyntactic analysis, entity recognition, and word embeddings.
result HuSpaCy achieves high accuracy with resource-efficient prediction.
OMLT combines ML and optimization for solving complex problems.
problem Solving complex decision-making problems in computer science and engineering.
method OMLT integrates neural networks and gradient-boosted trees into optimization problems using machine learning.
result OMLT seamlessly integrates with Pyomo and solves real-world problems.
The change of two orders of magnitude in the 'new DCF' of NIST's SRE'10, relative to the 'old DCF' evaluation criterion, posed a difficult challenge for participants and evaluator alike. Initially, participants were at a loss as to how to calibrate their systems, while the evaluator underestimated the required number o…
New method improves likelihood-free parameter estimation in complex models.
problem Estimating parameters in simulation-based models with unknown likelihood.
method Nested multi-time-scale stochastic approximation (NMTS) method.
result Eliminates bias and accelerates convergence in likelihood-free inference.