Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,291 papers · 148 categories
Trend · papers per month
48 results for “neural sentence encoding”
SenGen generates sentences conditioned on topics, improving topic visualization.
problem Improve topic visualization and interpretability in documents.
method Variational auto-encoder with RNN decoder conditioned on topics.
result Preliminary experiments show promise but also challenges remain.
Randomly initialized sentence encoders perform well on tasks, suggesting learning is key.
problem The role of sentence encoder architectures in language tasks.
method Random initialization and fixed architecture approach to evaluate sentence encoders.
result Priors do not leverage additional information, learning is necessary.
Develops a new dataset and model for summarizing scientific papers.
problem Lack of large datasets for summarizing scientific papers.
method Exploits author-provided summaries, uses neural sentence encoding and summarisation features.
result Models that encode sentences and their context perform best, significantly outperforming baselines.
Unified multi-view sentence representation improves downstream tasks.
problem Improving sentence representation learning from diverse views.
method Unified multi-view sentence representation learning framework using RNN and linear model, maximizing agreement with adjacent context.
result Improved representations and transferability on downstream tasks.
This work speeds up unsupervised sentence learning using paragraph coherence.
problem Training fast unsupervised sentence encoders.
method Discourse-based objective function for neural network training.
result Models trained with this method are faster and perform well.
Paper proposes continual learning for sentence encoders.
problem Optimize sentence encoders for new corpora while maintaining old corpus accuracy.
method Initialize encoders with corpus-independent features, update using Boolean operations of conceptor matrices.
result Proposed sentence encoder can continually learn features from new corpora.
Improved text summarization using neural semantic encoders with hierarchical structure.
problem Capturing long-term dependencies in text summarization.
method Proposed a novel hierarchical Neural Semantic Encoder (NSE) model augmented with lemma and PoS tags.
result Significantly outperformed state-of-the-art models in ROUGE metric.
Generative model controls text attributes for realistic sentences.
problem Challenges in generating natural language sentences with desired attributes.
method Combines variational auto-encoders and holistic attribute discriminators for semantic structure imposition.
result Effective generation of realistic sentences with desired attributes.
Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural ma…
QuASE uses QA data to improve non-QA tasks like NER.
problem Can QA data help other NLP tasks?
method Proposes QuASE framework to learn sentence encodings from QA data.
result QuASE encodings improve downstream tasks like NER.
Ranked second in fact-checking task, using DRR NN with embeddings.
problem Fact-checking questions in community forums.
method Deeply Regularized Residual Neural Network (DRR NN) with Universal Sentence Encoder embeddings, ensemble methods.
result Ranked second in fact-checking task.
GNMT uses latent variables to improve machine translation, especially with missing words.
problem Improving machine translation, especially with missing words.
method GNMT adds a latent variable to an encoder-decoder model to capture sentence semantics.
result GNMT achieves competitive BLEU scores and is superior when source sentences are missing words.
Deep learning models improved sentence similarity in medical records.
problem Improving sentence similarity in electronic medical records.
method Developed models using traditional machine learning and deep learning approaches, pre-trained sentence embeddings on biomedical corpora.
result Ensembled model achieved a Person correlation coefficient of 0.8528.
Convolutional residual model predicts diagnoses from EHR notes.
problem Predicting multiple diagnoses from medical text data.
method Convolutional Neural Network (CNN) + Deep Residual Network.
result Superior performance compared to baseline models.
Improved sentence modeling using Suffix Bidirectional LSTM.
problem Sequential bias in BiLSTMs limits long-range dependencies.
method Encodes each suffix and prefix of a sequence in both forward and reverse directions.
result SuBiLSTM improves performance in various NLP tasks.
Improved autoencoders guide latent sentence representations for better text generation and manipulation.
problem Current autoencoders struggle to maintain coherent latent spaces for meaningful text manipulations.
method Adversarial autoencoders with a denoising objective (DAAE) to guide latent space geometry.
result DAAE provides the best trade-off between generation quality and reconstruction capacity.
We present a memory augmented neural network for natural language understanding: Neural Semantic Encoders. NSE is equipped with a novel memory update rule and has a variable sized encoding memory that evolves over time and maintains the understanding of input sequences through read}, compose and write operations. NSE c…
Wasserstein autoencoder improves sentence generation in NLP.
problem Difficulty in training variational autoencoders for text.
method Proposes a stochastic Wasserstein autoencoder (WAE) for probabilistic sentence generation.
result Wasserstein autoencoder achieves higher BLEU scores for sentence reconstruction.
Two new methods improve coherence modeling without complex machine translation.
problem Improving neural coherence modeling for better sentence ordering.
method Two novel methods combining regression and context concatenation.
result Achieves state-of-the-art Kendall-tau and positional accuracy scores.
The paper presents a neural machine translation system for Wolof.
problem Low-resource conditions for translating French to Wolof.
method Developed a bidirectional LSTM-based encoder-decoder architecture with attention mechanisms.
result Best model achieved a 47% BLEU score for French-Wolof translation.
Neural network model improves sentence classification in medical abstracts.
problem Individual sentence classification misses contextual information.
method Combines ANN effectiveness with structured prediction for joint sentence classification.
result Achieves state-of-the-art results on medical abstract datasets.
DSA improves sentence embedding by dynamically attending to words.
problem Efficiently capturing the importance of words in sentences for embedding.
method DSA modifies dynamic routing from capsule networks for self-attention in sentences.
result DSA achieves state-of-the-art results in SNLI with fewer parameters.
CERT improves language understanding by contrastively learning sentence-level semantics.
problem Lack of sentence-level semantics in existing pretraining tasks.
method Contrastive self-supervised learning at the sentence level using back-translation augmentations.
result CERT outperforms BERT on 7 out of 11 GLUE benchmark tasks, achieving the same performance as BERT on 2 tasks.
SFBoW provides sentence embeddings with predefined dimensions.
problem Sentence embeddings problem at document-level.
method Refinement of Fuzzy Bag-of-Words, predefined dimension.
result Competitive performances in Semantic Textual Similarity benchmarks.
Framework learns sentence order from paragraphs using attention and transformer networks.
problem Learning to order sentences from a paragraph.
method Bidirectional sentence encoder and self-attention transformer network for ranking.
result Framework outperforms state-of-the-art methods on sentence ordering and discrimination tasks.
Paper develops neural network for Mandarin polyphone disambiguation.
problem Homograph problem in Mandarin Chinese text-to-speech.
method Bidirectional RNN for context, prediction network for mapping embeddings to pronunciations.
result Achieves 94.69% accuracy on polyphonic character dataset.
This paper assesses biases in contextualized word representations.
problem Analyzing biases in contextualized word representations.
method Proposes assessing bias at the contextual word level, capturing contextual effects of bias.
result Demonstrates evidence of bias in contextual word models, including racial bias and exacerbated effects for intersectional minorities.
Proposes a method to generate diverse translations by conditioning on target domain.
problem NMT models lack diversity in translations, even with search algorithms.
method Condition the decoder on a latent variable representing target domain, generated by a target encoder.
result Generated diverse translations without affecting performance or training time.
Generates text with specified attributes, improving content compatibility.
problem Modifying textual attributes of sentences while maintaining content compatibility.
method Introduces reconstruction and adversarial losses to generate attribute-compatible, realistic sentences.
result Demonstrates superior content compatibility and attribute control compared to prior methods.
CURE extracts relations without supervision by clustering similar entity pairs.
problem Extracting relations unsupervised without considering sentence correlations.
method CURE uses Encoder-Decoder architecture for self-supervised learning and clustering similar relations.
result CURE outperforms state-of-the-art models on NYT and UNPC datasets.
This paper explores sentence vector properties for automatic summarization.
problem Understanding the internal structure and properties of sentence vectors.
method Compositional sentence vector representations using artificial neural networks.
result Cosine similarity correlates with sentence importance and can identify gaps in summaries.
Extracts parallel sentences for machine translation.
problem Data sparsity in multilingual natural language processing.
method Bidirectional recurrent neural network approach.
result Significant improvements in machine translation performance.
New method improves sentence classification using context information.
problem Classifying sentences with limited context information.
method Context-LSTM-CNN method that considers large contexts and long-range dependencies.
result Consistently improves over previous methods on two datasets.
AUTR generates sentences using a dynamic memory and attention mechanism.
problem Generating coherent sentences without explicit training data.
method Recurrent neural network with dynamic attention and canvas memory.
result AUTR learns meaningful latent representations and achieves competitive performance.
SIVAE integrates sentences and their syntactic trees for improved text generation.
problem Improving the grammar of generated text.
method SIVAE uses two separate latent spaces for sentences and syntactic trees, optimizing a joint distribution with two encoders and two decoders.
result SIVAE generates sentences with better grammar compared to existing models.
Paper proposes a deep learning architecture for generating long stories from images.
problem Maintaining context in long event sequences for visual storytelling.
method Hierarchical deep learning architecture with encoder-decoder networks and natural language descriptions.
result Our method outperforms state-of-the-art techniques on automatic evaluation metrics.
Overcoming the Curse of Sentence Length for Neural Machine Translation using Automatic Segmentationcs.CL
The authors of (Cho et al., 2014a) have shown that the recently introduced neural network translation systems suffer from a significant drop in translation quality when translating long sentences, unlike existing phrase-based translation systems. In this paper, we propose a way to address this issue by automatically se…
Detects out-of-distribution sentences in Neural Machine Translation.
problem Identifying sentences from a different language than the training data.
method Developed a new uncertainty measure for long sequences of words in Transformers.
result Shows ability to identify Dutch sentences as German input.
Author2Vec generates user embeddings from social media data.
problem Generating useful user embeddings from noisy social media data.
method End-to-end neural network with BERT sentence representations and unsupervised pre-training.
result Author2Vec outperforms traditional methods in user classification tasks.
Paper uses LLMs to detect financial anomalies.
problem Detecting irregular financial entries.
method Non-semantic financial data encoding with LLMs embeddings, tested 3 models.
result LLMs improve anomaly detection in financial data.
Gating Mechanisms for Combining Character and Word-level Word Representations: An Empirical Studycs.CL
The study examines how character and word-level representations improve sentence-level tasks.
problem Improving the quality of word and sentence representations using character-level information.
method Feature-wise sigmoid gating mechanism for combining character and word-level representations.
result Modeling characters improves final word and sentence representations, especially for less frequent words.
Graph model detects fake news by analyzing sentence interactions.
problem Detecting fake news through online media.
method Graph Neural Network-based model for sentence interactions.
result Our model achieves state-of-the-art accuracy on fake news datasets.
Proposes a multilingual email segmentation benchmark and model.
problem Lack of multilingual email zoning corpora and models.
method Analysis of existing corpora, development of multilingual benchmark, introduction of OKAPI model.
result OKAPI model achieves state-of-the-art performance in English and generalizes well to unseen languages.
Unified model improves coherence tasks, especially local contexts.
problem Existing neural coherence models struggle with local context tasks.
method Unified neural framework integrating grammar, relations, and patterns.
result Unified model outperforms existing models significantly.
The quality of machine translation is rapidly evolving. Today one can find several machine translation systems on the web that provide reasonable translations, although the systems are not perfect. In some specific domains, the quality may decrease. A recently proposed approach to this domain is neural machine translat…
Improved text matching model using deconvolutional networks.
problem Text sequence matching challenges.
method Jointly optimizing generative and discriminative objectives with deconvolutional networks.
result Significantly outperforms sentence-encoding baselines, especially in semi-supervised settings.
Paper examines NMT robustness to nonsensical inputs.
problem NMT systems fail when source sentences are altered.
method Soft-attention technique to replace words in source sentences.
result Proposed technique achieves high success rate and outperforms existing methods.