Paper proposes a method for cross-lingual sentiment classification using distributed word representations.
problem Cross-lingual sentiment classification with strict one-to-one word mapping limitations.
method Uses distributed word representations to learn meaningful one-to-many mappings for pivot words.
result Method outperforms state-of-the-art in cross-lingual sentiment classification.
Paper proposes a method to learn word translations bidirectionally.
problem Word translation between languages.
method Jointly learns translations in both directions with minimal supervision.
result Improves accuracy of translations over previous methods.
This paper improves translation of rare words in under-resourced languages.
problem Translation of out-of-vocabulary words in low-resource languages.
method Developed neural and non-neural context models for target-language context integration.
result Neural language models with wide context are most effective in disambiguating OOV word translations.
This paper tackles rare word problem in low-resource language pairs using NMT.
problem Rare word problem in neural machine translation, especially for low-resource languages.
method Three solutions: enhanced source context, morphology learning, and wordnet synonyms.
result Significant improvements in BLEU scores (+1.0 points) on English-Vietnamese and Japanese-Vietnamese.
Improves NMT by sampling context from predicted sequence during training.
problem Error accumulation and overcorrection in NMT due to mismatched training and inference contexts.
method Samples context words from both ground truth and predicted sequences during training.
result Significant improvements on multiple datasets, including Chinese->English and WMT'14 English->German.
GNMT uses latent variables to improve machine translation, especially with missing words.
problem Improving machine translation, especially with missing words.
method GNMT adds a latent variable to an encoder-decoder model to capture sentence semantics.
result GNMT achieves competitive BLEU scores and is superior when source sentences are missing words.
Improved unsupervised word translation using adversarial autoencoder with cycle consistency and input reconstruction.
problem Challenging language pairs and lack of parallel data for unsupervised word translation.
method Adversarial autoencoder with cycle consistency and input reconstruction regularization.
result More stable and better performance than recent approaches.
Recent work on learning multilingual word representations usually relies on the use of word-level alignements (e.g. infered with the help of GIZA++) between translated sentences, in order to align the word embeddings in different languages. In this workshop paper, we investigate an autoencoder model for learning multil…
Paper proposes AXE loss for non-autoregressive machine translation, improving performance.
problem Challenges in training non-autoregressive models due to lack of autoregressive factors and cross entropy loss penalties.
method Proposes aligned cross entropy (AXE) loss function using a differentiable dynamic program for better word order alignment.
result AXE-based training improves performance on major WMT benchmarks and sets a new state of the art for non-autoregressive models.
Paper identifies and solves a 'scrambled translation' issue in UNMT models.
problem Scrambled translation in UNMT models using word shuffle noise.
method Retraining UNMT models without noise after a pre-defined number of iterations.
result Retraining strategy improves BLEU scores by 10-20% on various language pairs.
Parallel decoding improves machine translation efficiency and accuracy.
problem Efficiently generating translations from left to right.
method Conditional masked language modeling for parallel decoding.
result Improves translation performance by over 4 BLEU points.
Paper improves NART models for machine translation with faster inference.
problem Inference latency in NART models is high and translation accuracy is inferior.
method Leverages hints from hidden states and word alignments to train NART models.
result Significant improvement in translation accuracy and comparable inference speed to ART baselines.
Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates…
Transformer++ improves neural machine translation BLEU scores.
problem Handling long-range dependencies in sentences.
method Proposes a new multi-head attention mechanism with context learning.
result Achieves new state-of-the-art BLEU scores on English-to-German and English-to-French translation tasks.
West Frisian lemmatizer, POS tagger, and parser created.
problem Creating accurate lemmatization, POS tagging, and dependency parsing for West Frisian.
method Using a corpus of 44,714 words annotated according to Universal Dependency version 2. Applying Dutch POS tags and morphological/syntactic annotations to create Frisian translations.
result Significant improvement in lemma accuracy compared to default parameters.
Graph-to-Tree Neural Networks improve structured input-output translation in tasks like semantic parsing and math word problems.
problem Improving performance on tasks like semantic parsing and math word problem solving.
method Graph-to-Tree Neural Networks, consisting of a graph encoder and a hierarchical tree decoder.
result Graph2Tree model outperforms or matches state-of-the-art models on neural semantic parsing and math word problem tasks.
Unsupervised MT struggles with morphologically rich languages.
problem Limitations of unsupervised machine translation on morphologically rich languages.
method Adversarial unsupervised alignment of word embedding spaces for bilingual dictionary induction.
result A simple trick exploiting weak supervision from identical words improves unsupervised bilingual dictionary induction performance.
Proposes using Wasserstein barycenter for better multilingual alignment.
problem Finding word-to-word translations between multiple languages without parallel data.
method Uses Wasserstein barycenter as a more informative pivot language, minimizing pairwise transportation costs.
result Demonstrates state-of-the-art performances on standard benchmarks.
Anosov maps on torus curve graphs have positive integer translation lengths.
problem Understanding the translation lengths of Anosov maps on curve graphs of tori.
method Constructive proof and algorithm for calculating exact translation lengths.
result The stable translation length of an Anosov map on the curve graph is always a positive integer.
This research explores using kernels in the softmax layer for better contextual word classification.
problem Improving contextual word classification accuracy.
method Replacing the inner product in the softmax layer with various kernel functions and comparing their performance.
result Different kernel settings yield varying performance in contextual word classification tasks.
Differentiable sampling corrects alignment issues in neural machine translation.
problem Incorrect alignment of reference words and sampled output in scheduled sampling.
method Optimizes alignment probability based on model's soft alignment prediction.
result Improves BLEU score compared to maximum likelihood and scheduled sampling.
Detects out-of-distribution sentences in Neural Machine Translation.
problem Identifying sentences from a different language than the training data.
method Developed a new uncertainty measure for long sequences of words in Transformers.
result Shows ability to identify Dutch sentences as German input.
The study examines translation lengths of pseudo-Anosov maps on curve graphs.
problem Estimating translation lengths of pseudo-Anosov maps on curve graphs.
method Analyzing geodesic axes and powers of Dehn twists.
result Determining minimal translation lengths and optimizing map ratios.
New method to understand bias in word embeddings.
problem Understanding and mitigating bias in word embeddings.
method Developed a technique to trace bias origins back to training documents.
result Accurate approximations of bias reduction can be made.
Improves neural machine translation by learning better source representations with relation networks.
problem Forgetting distant information and disregarding relationship between source words in neural machine translation.
method Introduces relation networks to learn better source representations by associating source words with each other and retaining their relationships.
result Significantly improves translation performance over conventional encoder-decoder models and outperforms approaches involving supervised syntactic knowledge.
Researchers mapped clinical jargon to consumer language using embeddings alignment.
problem Mapping and translating clinical jargon to consumer language for better communication.
method Trained embeddings on clinical and consumer language corpora, aligned them using Procrustes algorithm, and refined with adversarial training.
result Procrustes algorithm effectively aligned clinical and consumer language embeddings.
POLAR framework interprets word embeddings using polar opposites.
problem Lack of interpretability in pre-trained word embeddings.
method Adopt semantic differentials and polar opposites to transform embeddings.
result Interpretable word embeddings maintain performance comparable to original embeddings.
New bounds on complexity for finding near-stationary points in stochastic convex optimization.
problem Finding near-stationary points in stochastic convex optimization.
method Joint analysis of local stochastic oracle and global oracle models; extensions of recursive regularization technique.
result Logarithmic dependence on smoothness in global oracle model for finding near-stationary points.
Paper proposes a faster neural machine translation model using election methods and Q-learning.
problem Slower inference time in complex attention models.
method Modelled attention network using election methods and Q-learning.
result Inference time is less than a standard Bahdanau translator, results comparable.
Enhanced word embeddings boost multiclass text classification accuracy.
problem Improving multiclass text classification accuracy using pre-trained embeddings.
method Proposed word-class embeddings (WCEs) to enhance pre-trained word embeddings.
result WCEs significantly improve multiclass text classification accuracy.
We describe a unified and coherent syntactic framework for supporting a semantically-informed syntactic approach to statistical machine translation. Semantically enriched syntactic tags assigned to the target-language training texts improved translation quality. The resulting system significantly outperformed a linguis…
FRAGE learns word embeddings without frequency bias, improving performance across NLP tasks.
problem Word embeddings are biased towards word frequency, affecting performance for rare words.
method Adversarial training to learn Frequency-Agnostic word Embedding (FRAGE).
result FRAGE achieves higher performance than baselines in all four NLP tasks.
Paper speeds up neural language model inference by 20x for top-k word prediction.
problem Slow inference speed of neural language models on mobile devices.
method Introduced a screening model using Gumbel softmax to approximate softmax layer.
result Achieved 20.4x speedup with 98.9% precision@1 and 99.3% precision@5 for German to English translation.
Word embeddings improve classification of research stages.
problem Classifying research stages using conventional methods.
method Used pre-trained and custom word embeddings for classification.
result Custom embeddings outperform general embeddings for research classification.
A model learns to translate natural language queries for search systems.
problem Understanding user queries for search-oriented conversational systems.
method Reinforcement learning framework for translating NL expressions to queries.
result Effectiveness of the model on TREC datasets.
Transformer models align words through attention weights, closely approximating Optimal Transport.
problem Understanding the internal mechanism of transformer models in language processing.
method Empirical evidence and theoretical analysis of attention weights and their relation to Optimal Transport.
result Transformer models can simulate gradient descent on the dual of entropy-regularized OT problem, providing a theoretical foundation for token alignment.
Neural Machine Translation (MT) has reached state-of-the-art results. However, one of the main challenges that neural MT still faces is dealing with very large vocabularies and morphologically rich languages. In this paper, we propose a neural MT system using character-based embeddings in combination with convolutional…
We introduce BilBOWA (Bilingual Bag-of-Words without Alignments), a simple and computationally-efficient model for learning bilingual distributed representations of words which can scale to large monolingual datasets and does not require word-aligned parallel training data. Instead it trains directly on monolingual dat…
Autoencoders improve sequence model representations.
problem Challenges in extracting good representations from recurrent models.
method Integrates autoencoder with sequence models to compress through discrete latent space.
result Improved latent codes correspond to words and phrases, enhancing model performance.
DKPCA improves WSD accuracy with scarce labeled data.
problem Word sense disambiguation in natural language processing.
method DKPCA combines Kernel PCA and Semantic Diffusion Kernel.
result DKPCA outperforms SVM and KPCA on SensEval data.
New method learns text generation orders without pre-specification.
problem Generating text in arbitrary orders without manual specification.
method Generates text in non-monotonic orders using a binary tree structure and imitation learning.
result Models can generate text without pre-specifying an order, achieving competitive performance.
Adaptively sparse Transformers improve interpretability and diversity in NLP.
problem Standard Transformers use dense attention, limiting interpretability and diversity.
method Introduces adaptively sparse Transformers using α-entmax for context-dependent sparsity. result Improves interpretability and diversity in NLP tasks without sacrificing accuracy.
This research explores the effects of various training settings from Polish to English Statistical Machine Translation system for spoken language. Various elements of the TED parallel text corpora for the IWSLT 2013 evaluation campaign were used as the basis for training of language models, and for development, tuning …
A new neural machine translation method learns from human feedback and reduces human effort.
problem Efficiently reducing human effort in interactive-predictive neural machine translation.
method Learning from human reinforcements, using entropy for feedback triggers, and online model updates.
result Significant improvement in translation quality with reduced human feedback requests.
Proposes RTL model for sentiment classification and key word detection in online reviews.
problem Sentiment classification and key word detection in online reviews for hospitality industry.
method Regularized Text Logistic (RTL) regression model.
result RTL model achieves satisfactory classification performance and identifies key word features.
Our research extends the Bilingual Evaluation Understudy (BLEU) evaluation technique for statistical machine translation to make it more adjustable and robust. We intend to adapt it to resemble human evaluation more. We perform experiments to evaluate the performance of our technique against the primary existing evalua…
New algorithm speeds up model optimization for time series data.
problem Slow convergence in model optimization due to continuous symmetry.
method Gauge theory-inspired optimization algorithm.
result Orders of magnitude faster convergence and interpretable representations.
Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural ma…