Shared task on machine translation using weak user feedback.
problem Learning machine translation from unreliable user feedback.
method Bandit learning approach using real-valued quality estimates.
result Various machine translation architectures and protocols were evaluated.
Evolved Transformer improves on Transformer architecture for language tasks.
problem Improving Transformer architecture for sequence tasks.
method Evolutionary architecture search with warm starting and dynamic resource allocation.
result Evolved Transformer achieves state-of-the-art BLEU scores and reduces parameter count.
New study shows complex sparsity techniques perform inconsistently on large datasets.
problem Evaluating sparsity techniques in deep neural networks on large-scale tasks.
method Compared three state-of-the-art sparsity techniques on two large-scale learning tasks.
result Simple magnitude pruning achieves comparable or better results than complex techniques.
Sockeye is an open-source toolkit for neural machine translation.
problem Improving Neural Machine Translation (NMT) models and techniques.
method Scalable training and inference for three NMT architectures, including attentional, self-attentional, and fully convolutional networks.
result Sockeye achieves competitive BLEU scores across different NMT architectures, including a best score for its transformer implementation.
This research tackles uncertainty estimation in autoregressive structured prediction tasks.
problem Ensuring safety and robustness of AI systems through accurate uncertainty estimation.
method Develops a unified probabilistic ensemble-based framework for token-level and sequence-level uncertainty estimation.
result Provides baselines for error and out-of-domain detection on translation and speech recognition datasets.
Transformer++ improves neural machine translation BLEU scores.
problem Handling long-range dependencies in sentences.
method Proposes a new multi-head attention mechanism with context learning.
result Achieves new state-of-the-art BLEU scores on English-to-German and English-to-French translation tasks.
Tensor Train layer improves BLEU scores in NMT models.
problem Improving Neural Machine Translation (NMT) models' performance.
method Implemented Tensor Train layer in TensorFlow for NMT training.
result Higher learning rates and more 'rectangular' core dimensions improve BLEU scores.
ICML workshop on making machine learning models more understandable.
problem Making machine learning models more understandable to humans.
method Various presentations and discussions on interpretability techniques.
result Improved methods for explaining machine learning models.
Symposium on making machine learning models more understandable.
problem Making machine learning models more understandable.
method Not specified in the abstract.
result Not specified in the abstract.
Proposes a faster Transformer decoding method by truncating target-side self-attention windows.
problem Efficiency in Transformer decoding with minimal BLEU score loss.
method N-gram assumption to truncate target-side self-attention windows.
result N-gram masked self-attention model maintains BLEU score for N values from 4 to 8. We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parameters that grows only linearly with the number of languages. This is made possible by having a single attention mechanism that is …
NIPS workshop focuses on ML for developing countries.
problem Addressing machine learning challenges in developing nations.
method Not specified in the abstract.
result Not specified in the abstract.
Neural Machine Translation (MT) has reached state-of-the-art results. However, one of the main challenges that neural MT still faces is dealing with very large vocabularies and morphologically rich languages. In this paper, we propose a neural MT system using character-based embeddings in combination with convolutional…
Convolutional neural networks win SemEval-2017 for scientific relation extraction.
problem Extracting relations between scientific concepts from scholarly articles.
method Convolutional neural network model for relation extraction.
result Ranked first in SemEval-2017 Task 10 for relation extraction in scientific articles.
Tangent Works won GEFCom 2017 using automatic model building.
problem Forecasting time series with historical temperature shuffling.
method Automatic model building using Tangent Information Modeller (TIM) with historical temperature shuffling and decision on trend variable.
result Automated model building setup won the competition.
Study shows changes in information sharing between Bitcoin markets during 2017 crash.
problem Understanding information dynamics in Bitcoin markets during the 2017 crash.
method Analysis of high-frequency market-microstructure observables using information theoretic measures.
result Temporal changes in information sharing across markets, including predictability, memory, and synchronous coupling.
Amobee's system won 3rd place in Twitter sentiment classification.
problem Sentiment detection on Twitter using deep learning.
method RNN models trained on a sentiment treebank, combined with classifiers.
result 3rd place in SemEval 2017 task 4, 5-label classification.
Y. Nikonorov completes a proof in a geometry paper.
problem Completing a proof in a geometry paper.
method Completing an argument from a previous proof.
result Proof of Theorem 2.5 in JGA 27 (2017) is now complete.
Paper proposes AXE loss for non-autoregressive machine translation, improving performance.
problem Challenges in training non-autoregressive models due to lack of autoregressive factors and cross entropy loss penalties.
method Proposes aligned cross entropy (AXE) loss function using a differentiable dynamic program for better word order alignment.
result AXE-based training improves performance on major WMT benchmarks and sets a new state of the art for non-autoregressive models.
SGP combines PushSum with stochastic gradient updates for robust distributed deep learning.
problem Synchronization issues in distributed deep learning.
method Stochastic Gradient Push (SGP) using PushSum for approximate distributed averaging.
result SGP converges to a stationary point at the same rate as SGD and achieves consensus.
ES and FD gradients converge as optimization dimension grows.
problem Understanding the relationship between Evolution Strategies and Finite Differences gradients.
method Analyzing the convergence of gradients as the optimization dimension increases.
result ES and FD gradients converge as the dimension of the vector under optimization increases.
Survey of open problems in finite-dimensional integrable systems.
problem Open problems in finite-dimensional integrable systems.
method None specified; survey of existing open problems.
result Many open problems were identified from a conference.
A planning mechanism improves sequence-to-sequence model performance.
problem Improving sequence-to-sequence model performance for tasks like translation and graph analysis.
method Developed a model that plans ahead using attention and differentiable operations.
result Outperforms strong baselines on character-level translation, graph analysis, and question generation.
Two algorithms find local minima faster in finite-sum and general stochastic optimization.
problem Finding local minima in finite-sum and general stochastic nonconvex optimization.
method Stochastic Nested Variance Reduction (SNVRG) + Neon2.
result Achieves better gradient complexity for convergence to (ε,εH)-second-order stationary points. Transformers trained with three normalization changes outperform state-of-the-art on low-resource translation tasks.
problem Improving the training of Transformers, especially on low-resource datasets.
method Three normalization changes: PreNorm, ScaleNorm, and FixNorm.
result Significant improvements in BLEU scores on low-resource translation tasks.
Explains deep neural networks using new interpretation techniques.
problem Interpreting and understanding deep neural networks.
method Introduces new techniques for interpretation and practical applications.
result Efficient use of interpretation techniques on real data.
Improves NMT by sampling context from predicted sequence during training.
problem Error accumulation and overcorrection in NMT due to mismatched training and inference contexts.
method Samples context words from both ground truth and predicted sequences during training.
result Significant improvements on multiple datasets, including Chinese->English and WMT'14 English->German.
A new system detects audio replay attacks with high accuracy.
problem Detecting and preventing audio replay attacks in speaker verification systems.
method Proposes Attentive Filtering Network combining attention-based filtering and ResNet classifier.
result Achieves EER of 8.99% on ASVspoof 2017 Version 2.0 dataset.
Stable ResNet stabilizes gradients in deep networks.
problem Gradient vanishing and exploding in deep ResNet architectures.
method Introducing Stable ResNet architectures with gradient stabilization and infinite depth expressivity.
result Stable ResNet maintains gradient stability and expressivity in deep networks.
Improved neural network robustness against adversarial attacks.
problem Adversarial attacks on neural networks.
method Adversarial-trained Bayesian Neural Network (Adv-BNN).
result State-of-the-art performance improvement under strong attacks.
We introduce a new sampling method for large language models that balances diversity and parallelism.
problem Balancing diversity and parallelism in decoding for large language models.
method Arithmetic sampling framework compatible with various sampling variations.
result Improves estimation of expected BLEU score reward and reduces the gap with beam search.
Forecast predicts US recession in 2017, global economic slowdown, and eventual growth.
problem Short-term economic forecast and potential recession in developed countries.
method Analysis of log-periodic oscillations in DJIA dynamics and historical economic cycles.
result Predicts a recession in the second half of 2017 for developed countries.
Study quantifies reproducibility of machine learning papers.
problem Lack of empirical reproducibility metrics in machine learning.
method Manual implementation of 255 papers from 1984-2017, analyzing features and results.
result Manual implementation revealed discrepancies between papers and their descriptions.
Syntax-enhanced models boost machine translation and NLP performance.
problem Limited training data and complex models struggle in NLP tasks.
method Syntax information was explicitly fed into Transformer and BERT models.
result Syntax-infused models achieved significant BLEU improvements.
EWC uses quadratic penalties that may double-count earlier task data.
problem Catastrophic forgetting in neural networks.
method Extended derivation of EWC with multiple tasks.
result Quadratic penalties in EWC might double-count earlier task data.
Directly analyzes SGLD hitting times for stationary points, providing tighter bounds.
problem Analyzing the hitting time of SGLD to stationary points.
method Direct analysis using linear algebra and probability theory, avoiding complex Cheeger's constant bounds.
result Tighter bounds on hitting times compared to previous work, showing dimension-independent behavior under suitable conditions.
New estimator stabilizes higher-order influence functions for stable statistical inference.
problem Numerical instability in estimating inverse population Gram matrix.
method Proposes a new stabilized higher-order estimator without sample splitting.
result Stabilized estimator exhibits more stable performance and similar statistical guarantees.
Paper compares different spoofing detection methods for speech verification.
problem Detecting audio replay attacks in speech verification systems.
method GMM based methods, high level features extraction with simple classifier, deep learning frameworks.
result Deep learning approaches are efficient in changing acoustic conditions.
New method calibrates rough stochastic volatility models quickly.
problem Calibrating rough stochastic volatility models is expensive and time-consuming.
method Combines Levenberg-Marquardt with neural networks for fast calibration.
result Neural network approximates implied volatility map efficiently.
Deep ResNets with single-neuron hidden layers can approximate any function.
problem The challenge of universal approximation by deep neural networks.
method A ResNet architecture with one neuron per hidden layer in each module.
result ResNet with one-neuron hidden layers is a universal approximator.
The paper explores dual learning, a technique that improves machine translation and image transformation.
problem Understanding and improving dual learning's effectiveness and conditions.
method Theoretical analysis and algorithmic extension of dual learning.
result Multi-step dual learning boosts performance under mild conditions.
Insertion Transformer generates sequences with flexible insertion orders.
problem Generating sequences with arbitrary orderings.
method Iterative, partially autoregressive model with insertion operations.
result Outperforms non-autoregressive approaches in translation tasks.
Adafactor optimizes neural networks with less memory and similar performance.
problem Memory constraints in adaptive optimization methods.
method Adafactor uses row and column sums of moving averages to estimate per-parameter second moments, reducing memory usage.
result Adafactor achieves similar performance to Adam with minimal auxiliary storage.
New estimator stabilizes higher-order influence functions for bilinear forms.
problem Stability issues in estimating bilinear forms using higher-order influence functions.
method Proposes a new stabilized higher-order estimator for a class of bilinear forms without sample splitting.
result New estimator exhibits more stable finite-sample performance compared to the empirical higher-order estimator.
Sequence to sequence learning has recently emerged as a new paradigm in supervised learning. To date, most of its applications focused on only one task and not much work explored this framework for multiple tasks. This paper examines three multi-task learning (MTL) settings for sequence to sequence models: (a) the onet…
Hard to estimate L2-accurate scores without strong assumptions.
problem Estimating the score of unknown data distributions accurately.
method Reduction to generating samples and leveraging lattice-based cryptography hardness.
result Score estimation is computationally hard even with polynomial sample complexity.
Neural networks model future values in finance.
problem Modeling future values of financial portfolios.
method Deep learning with neural networks to parameterize future values, optimizing parameters.
result Obtained expected positive/negative exposures for specific financial products.
State-of-the-art Twitter sentiment analysis using CNNs and LSTMs.
problem Improving Twitter sentiment classification accuracy.
method Pre-trained word embeddings, distant supervision, fine-tuning, ensemble of CNNs and LSTMs.
result First rank on all five English subtasks of SemEval-2017.