CGMH uses Metropolis-Hastings sampling to generate sentences with complex constraints.
problem Generating sentences with specific constraints while maintaining fluency and naturalness.
method CGMH employs Metropolis-Hastings sampling for constrained sentence generation.
result CGMH outperforms previous methods in various constrained sentence generation tasks.
We propose a new generative model of sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perp…
New metrics predict human sentence comprehension across languages.
problem Predicting human sentence comprehension using computational models.
method Developed sentence-level metrics using multilingual large language models.
result Achieved high accuracy in predicting human sentence reading speeds.
Paper proposes a novel model to improve n-ary cross-sentence relation extraction by addressing noisy data and non-consecutive sentences.
problem Noisy labeled data and non-consecutive sentences in n-ary cross-sentence relation extraction.
method Two-level agent reinforcement learning model and hybrid attention mechanism/PCNN approach.
result The model reduces the impact of noisy data and achieves better performance.
We introduce the Attentive Unsupervised Text (W)riter (AUTR), which is a word level generative model for natural language. It uses a recurrent neural network with a dynamic attention and canvas memory mechanism to iteratively construct sentences. By viewing the state of the memory at intermediate stages and where the m…
DSE learns sentence embeddings from cross-attention models to speed up sentence-pair similarity computation.
problem Computing sentence-pair similarity is computationally expensive when candidate sentences are large.
method Distilled Sentence Embedding (DSE) based on knowledge distillation from cross-attentive models.
result DSE significantly outperforms ELMO variants and other sentence embedding methods, accelerating computation by several orders of magnitude.
XL-Editor improves sentence post-editing using XLNet's variable-length insertion probability.
problem Post-editing sentences to refine generated text.
method XL-Editor trains XLNet to estimate variable-length insertion probabilities and apply post-editing operations.
result XL-Editor outperforms XLNet on text insertion and deletion tasks, and achieves significant style transfer improvements.
Unsupervised scheme ranks sentences in text documents based on semantic importance.
problem Ranking sentences in text documents without labeled data.
method Extracts essential words and phrases, constructs semantic phrase and sentence graphs, applies PageRank, combines scores, and optimizes for topic diversity.
result SSR outperforms individual judges and compares favorably with combined rankings on benchmarks.
Generates text with specified attributes, improving content compatibility.
problem Modifying textual attributes of sentences while maintaining content compatibility.
method Introduces reconstruction and adversarial losses to generate attribute-compatible, realistic sentences.
result Demonstrates superior content compatibility and attribute control compared to prior methods.
SIVAE integrates sentences and their syntactic trees for improved text generation.
problem Improving the grammar of generated text.
method SIVAE uses two separate latent spaces for sentences and syntactic trees, optimizing a joint distribution with two encoders and two decoders.
result SIVAE generates sentences with better grammar compared to existing models.
We present a new topic model that generates documents by sampling a topic for one whole sentence at a time, and generating the words in the sentence using an RNN decoder that is conditioned on the topic of the sentence. We argue that this novel formalism will help us not only visualize and model the topical discourse s…
Paper uses AC-GAN to generate high-quality relational sentences for relation extraction.
problem Limited training data for relation extraction models.
method Auxiliary Classifier Generative Adversarial Networks (AC-GANs).
result Significantly improved performance of relation extraction.
This paper explores sentence vector properties for automatic summarization.
problem Understanding the internal structure and properties of sentence vectors.
method Compositional sentence vector representations using artificial neural networks.
result Cosine similarity correlates with sentence importance and can identify gaps in summaries.
Estimating the difficulty level of math word problems is an important task for many educational applications. Identification of relevant and irrelevant sentences in math word problems is an important step for calculating the difficulty levels of such problems. This paper addresses a novel application of text categoriza…
Submodular functions have many applications. Matchings have many applications. The bitext word alignment problem can be modeled as the problem of maximizing a nonnegative, monotone, submodular function constrained to matchings in a complete bipartite graph where each vertex corresponds to a word in the two input senten…
This study improves sentence embeddings from BERT models.
problem Capturing the underlying meaning of sentences using BERT models.
method Comprehensive review and testing of various sentence embedding extraction and refinement methods.
result Representation-shaping techniques significantly improve sentence embeddings from BERT-based and simple baseline models.
Existing models based on artificial neural networks (ANNs) for sentence classification often do not incorporate the context in which sentences appear, and classify sentences individually. However, traditional sentence classification approaches have been shown to greatly benefit from jointly classifying subsequent sente…
Paper proposes continual learning for sentence encoders.
problem Optimize sentence encoders for new corpora while maintaining old corpus accuracy.
method Initialize encoders with corpus-independent features, update using Boolean operations of conceptor matrices.
result Proposed sentence encoder can continually learn features from new corpora.
Proposes a new RNN for language generation capturing long-range dependencies.
problem Capturing long-range word dependencies and sentence order in text corpora.
method Recurrent Hierarchical Topic-Guided RNN with dynamic deep topic model.
result Outperforms larger-context RNN-based language models and learns interpretable topics.
CLARA generates clinical reports from raw inputs, improving accuracy and efficiency.
problem Generating accurate and detailed clinical reports from raw inputs is time-consuming and error-prone.
method Interactive method that generates reports sentence by sentence based on doctors' anchor words and partially completed sentences.
result CLARA achieves significant improvements in report generation accuracy and efficiency.
Novel CNN array for sign language recognition using wearable IMUs.
problem Efficiently recognizing sign language from wearable IMU signals.
method Two-dimensional Convolutional Neural Network array architecture for Indian sign language recognition.
result Peak classification accuracies of 94.20% for general sentences and 95.00% for interrogative sentences achieved.
PAC-Bayes analysis explains sentence vector learning from unlabeled data.
problem Understanding and improving sentence vector learning from unlabeled data.
method PAC-Bayes bound analysis for transfer learning.
result Simple heuristics and new algorithms derived from PAC-Bayes analysis.
Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP share the same learned (and static) set of filters for all input sentences. In this paper, we consider an approach of using a smal…
In the sentence classification task, context formed from sentences adjacent to the sentence being classified can provide important information for classification. This context is, however, often ignored. Where methods do make use of context, only small amounts are considered, making it difficult to scale. We present a …
Paper proposes a new method for sentence embeddings using weighted word vectors.
problem Improving sentence embeddings for natural language processing tasks.
method A simple sentence embedding method using weighted average of word vectors followed by soft projection.
result Demonstrates effectiveness on clinical semantic textual similarity task.
Bayesian EnKF improves sentence comprehension uncertainty modeling.
problem Uncertainty in human language comprehension, especially with ambiguous inputs.
method Bayesian framework using ensemble Kalman filter (EnKF) for uncertainty quantification.
result Enhanced model's ability to approximate human cognitive processing with linguistic ambiguities.
We train multi-task autoencoders on linguistic tasks and analyze the learned hidden sentence representations. The representations change significantly when translation and part-of-speech decoders are added. The more decoders a model employs, the better it clusters sentences according to their syntactic similarity, as t…
A graph model improves short text classification by integrating sentence relationships.
problem Sparse features in short text classification due to limited text length.
method PathWalk model combining graph networks and short sentences.
result PathWalk achieves state-of-the-art results on four datasets.
Generic generation and manipulation of text is challenging and has limited success compared to recent deep generative modeling in visual domain. This paper aims at generating plausible natural language sentences, whose attributes are dynamically controlled by learning disentangled latent representations with designated…
Framework learns sentence order from paragraphs using attention and transformer networks.
problem Learning to order sentences from a paragraph.
method Bidirectional sentence encoder and self-attention transformer network for ranking.
result Framework outperforms state-of-the-art methods on sentence ordering and discrimination tasks.
SFBoW provides sentence embeddings with predefined dimensions.
problem Sentence embeddings problem at document-level.
method Refinement of Fuzzy Bag-of-Words, predefined dimension.
result Competitive performances in Semantic Textual Similarity benchmarks.
System tackles indeterminacies in automated audio captioning.
problem Word selection and sentence length indeterminacies in automated audio captioning.
method Solves caption generation and sub-indeterminacy problems through multi-task learning to estimate keywords and sentence length.
result Model achieved 20.7 SPIDEr score, significantly outperforming baseline.
Paper presents token-level explanations for NLI without annotated data.
problem Generating explanations for NLI models on sentence pairs.
method Simple LSTM architecture with LIME, Anchor, and MIL methods.
result MIL-based method faster but less accurate than black-box methods.
Existing relation classification methods that rely on distant supervision assume that a bag of sentences mentioning an entity pair are all describing a relation for the entity pair. Such methods, performing classification at the bag level, cannot identify the mapping between a relation and a sentence, and largely suffe…
Case Law has a significant impact on the proceedings of legal cases. Therefore, the information that can be obtained from previous court cases is valuable to lawyers and other legal officials when performing their duties. This paper describes a methodology of applying discourse relations between sentences when processi…
Seq-CVAE learns a latent space for each word position to capture sentence intention.
problem Capturing diversity in image captioning models.
method Seq-CVAE learns a sequential latent space for each word position, mimicking future sentence summaries.
result Significantly improves diversity metrics on MSCOCO dataset compared to baselines.
Deep learning models improved sentence similarity in medical records.
problem Improving sentence similarity in electronic medical records.
method Developed models using traditional machine learning and deep learning approaches, pre-trained sentence embeddings on biomedical corpora.
result Ensembled model achieved a Person correlation coefficient of 0.8528.
The paper improves ESG taxonomy and classifies sentences as sustainable or unsustainable.
problem Improving ESG taxonomy and classifying sentences based on ESG factors.
method For ESG taxonomy, used Sentence-BERT models. For sentence classification, combined RoBERTa with a multi-layer perceptron.
result Significant performance improvement and high accuracy in classifying sentences.
Project aims to improve coherence in language generation models.
problem Models often generate inconsistent text that diverges from the prompt.
method Trained a sentence pair coherence classifier and co-trained GPT-2 with this coherence objective.
result Fine-tuned model generates coherent paragraphs without diverging.
Parallel sentences are a relatively scarce but extremely useful resource for many applications including cross-lingual retrieval and statistical machine translation. This research explores our methodology for mining such data from previously obtained comparable corpora. The task is highly practical since non-parallel m…
Detects out-of-distribution sentences in Neural Machine Translation.
problem Identifying sentences from a different language than the training data.
method Developed a new uncertainty measure for long sequences of words in Transformers.
result Shows ability to identify Dutch sentences as German input.
Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates…
Improves document summarization by combining word embeddings and n-grams.
problem Exact word matching fails to measure semantic similarity between sentences.
method Uses deep embedding features and tf-idf features to improve sentence similarity measure; builds an improved sentence similarity graph; employs a submodular objective function; develops a Transformer-based compression model.
result Outperforms tf-idf based approach and achieves state-of-the-art performance on DUC04 dataset.
We introduce Generative Neural Machine Translation (GNMT), a latent variable architecture which is designed to model the semantics of the source and target sentences. We modify an encoder-decoder translation model by adding a latent variable as a language agnostic representation which is encouraged to learn the meaning…
Text-to-Image translation has been an active area of research in the recent past. The ability for a network to learn the meaning of a sentence and generate an accurate image that depicts the sentence shows ability of the model to think more like humans. Popular methods on text to image translation make use of Generativ…
A new whole-sentence language model - neural trans-dimensional random field language model (neural TRF LM), where sentences are modeled as a collection of random fields, and the potential function is defined by a neural network, has been introduced and successfully trained by noise-contrastive estimation (NCE). In this…
Improved autoencoders guide latent sentence representations for better text generation and manipulation.
problem Current autoencoders struggle to maintain coherent latent spaces for meaningful text manipulations.
method Adversarial autoencoders with a denoising objective (DAAE) to guide latent space geometry.
result DAAE provides the best trade-off between generation quality and reconstruction capacity.
ROTS improves sentence similarity by incorporating structural information.
problem Measuring sentence similarity with theoretical insights and structural awareness.
method Recursive Optimal Transport (ROT) framework to incorporate structural information.
result ROTS outperforms weakly supervised approaches in sentence similarity tasks.