Develops a hybrid model for text summarization.
problem Summarizing long text sequences concisely.
method Extends sequence encoders with a graph component to handle long-distance relationships in text.
result Hybrid models outperform pure sequence or graph models on summarization tasks.
A new model improves text summarization by integrating topic information.
problem Improving the coherence, diversity, and informativeness of text summarization.
method Integrates topic information into ConvS2S model and uses SCST for optimization.
result The proposed model outperforms state-of-the-art methods in abstractive summarization.
Survey of seq2seq models for neural text summarization.
problem Improving fluency and quality of text summaries.
method Comprehensive review of seq2seq models for abstractive text summarization.
result Benchmarking of two models on recent datasets.
Paper develops a statistical model for summarizing event sequences.
problem Discovering frequent serial episodes from sequential data.
method Minimum Description Length (MDL) principle with modifications.
result Reduces dictionary size by more than four-fold without losing accuracy.
A new model generates summaries by conditioning on input text and latent topics.
problem Improving abstractive summarization quality.
method Conditioning decoder output on both input text and latent topics identified by LDA.
result Strongly improved ROUGE scores on CNN/Daily Mail and WikiHow datasets.
Attention models boost speaker verification accuracy.
problem Improving text-dependent speaker verification accuracy.
method Exploring attention mechanisms in sequence summarization for speaker recognition.
result Attention-based models improve EER by 14%.
Improved text summarization using neural semantic encoders with hierarchical structure.
problem Capturing long-term dependencies in text summarization.
method Proposed a novel hierarchical Neural Semantic Encoder (NSE) model augmented with lemma and PoS tags.
result Significantly outperformed state-of-the-art models in ROUGE metric.
Survey on RL for seq2seq models to solve exposure bias and inconsistency.
problem Exposure bias and inconsistency in seq2seq models.
method Combining RL methods with seq2seq models.
result Improved seq2seq performance through RL.
Representation of human actions as a sequence of human body movements or action attributes enables the development of models for human activity recognition and summarization. We present an extension of the low-rank representation (LRR) model, termed the clustering-aware structure-constrained low-rank representation (CS…
Many prediction domains, such as ad placement, recommendation, trajectory prediction, and document summarization, require predicting a set or list of options. Such lists are often evaluated using submodular reward functions that measure both quality and diversity. We propose a simple, efficient, and provably near-optim…
Self-training improves neural sequence generation by correcting incorrect predictions.
problem Improving neural sequence generation models using unlabeled data.
method Injecting pseudo-parallel data (model predictions) into the labeled dataset and using dropout as a regularizer.
result Noisy self-training significantly improves performance on machine translation and text summarization benchmarks.
Method summarizes and predicts time series data for COVID-19 cases and deaths.
problem Summarizing and predicting time series data for multiple related time series.
method Hierarchical algorithm generating shapelets for centroids, nearest neighbor search for labeling, dynamic time warping for non-uniform lengths.
result Predictive model for individual time series based on aggregated statistics.
MoChA splits sequences into chunks for efficient, online attention.
problem Efficiently decoding sequence-to-sequence models with soft attention.
method Adaptive chunking of input sequences for soft attention computation.
result MoChA models achieve state-of-the-art performance in online speech recognition and document summarization.
New method uses dendrograms for better mixture model selection and clustering.
problem Selecting the correct number of components in finite mixture models.
method Hierarchical clustering tree derived from overfitted latent mixing measures.
result Consistently selects the true number of mixing components and optimal convergence rate for parameter estimation.
This work improves sequence-to-sequence models using amortized variational inference for context vectors.
problem Improving the performance of sequence-to-sequence models, especially in hard tasks like document summarization and video captioning.
method The approach treats context vectors as latent variables and infers their posteriors using an approximate finite mixture model via amortized variational inference.
result The method yields stronger generalization capacity and improved effectiveness over state-of-the-art alternatives.
Generative Bridging Network improves sequence prediction models by penalizing confidence and smoothing language.
problem Data sparsity and overfitting in sequence prediction tasks.
method Introduces a bridge module to extend ground truth and minimize KL-divergence between bridge distribution and generator.
result Significant improvements over strong baselines in machine translation and text summarization tasks.
We give an overview of how calculus of the embedding functor can be used for the study of long knots and summarize various results connecting the calculus approach to the rational homotopy type of spaces of long knots, collapse of the Vassiliev spectral sequence, Hochschild homology of the Poisson operad, finite type k…
Attention mechanism improves various NLP tasks.
problem Improving performance in natural language processing tasks.
method Assigning importance scores to sequence elements for encoding.
result Significant improvement in various NLP tasks.
CODE2SEQ generates natural language sequences from code snippets.
problem Generating natural language descriptions from code.
method CODE2SEQ represents code as AST paths and uses attention to select relevant paths.
result CODE2SEQ outperforms previous models for code-to-text tasks.
BigBird improves transformer performance on NLP tasks with longer sequences.
problem Quadratic dependency on sequence length in transformer models.
method Sparse attention mechanism reducing quadratic dependency to linear.
result Significant improvement in performance on NLP tasks like QA and summarization.
Set-Sequence model learns cross-sectional dynamics directly from time series data.
problem Predicting large cross-sections of time series data with latent cross-sectional dynamics.
method A model that learns cross-sectional structure directly, enhancing expressivity and eliminating manual feature engineering.
result Significantly outperforms strong baselines in equity portfolio optimization and loan risk prediction.
Future autonomous systems need reliable world models and complex action sequences.
problem Current automated systems lack reliable world models and complex action sequences.
method Introduce energy-based and latent variable models combined in a hierarchical joint embedding predictive architecture (H-JEPA).
result Combining energy-based and latent variable models in H-JEPA can lead to reliable world models and complex action sequences.
TaLK Convolutions improve sequence modeling efficiency.
problem Efficiently modeling sequences with limited time complexity.
method Adaptive convolution operation that learns kernel size.
result Time complexity reduced to O(n), making sequence encoding linear. SCROLLS benchmarks long text NLP tasks, improving existing models.
problem Short NLP benchmarks ignore long texts; SCROLLS addresses this.
method Handpicked long-text datasets for summarization, QA, and inference tasks.
result Improvement potential on SCROLLS tasks, as indicated by initial baselines.
Convolutional autoencoding improves long text reconstruction.
problem Text reconstruction quality decreases with text length.
method Sequence-to-sequence, purely convolutional and deconvolutional autoencoding.
result Better at reconstructing and correcting long paragraphs.
New L2D framework allows deferring specific parts of a sequence prediction to experts.
problem Current L2D methods defer entire predictions, which is not ideal for long sequences.
method Proposes token-level and one-time rejectors to defer specific outputs of a model prediction to experts.
result Granular deferrals achieve better cost-accuracy tradeoffs than whole deferrals.
TREP learns pedestrian trajectories efficiently without needing full datasets.
problem Learning fixed-length vector representations of variable-length trajectories.
method Actor-critic sequence-to-sequence autoencoder with spatial-aware objective function.
result TREP efficiently learns trajectory representations without needing full datasets.
Unified framework connects various sequence prediction algorithms.
problem Compounding error and poor exploration efficiency in sequence prediction models.
method Formal connection between algorithms through a generalized entropy regularized policy optimization formulation.
result Unified interpretation offers systematic view of varying properties of exploration and learning efficiency.
Solution for voice conversion with limited data using hierarchical seq2seq and attention models.
problem Voice conversion between speakers with limited parallel audio pairs.
method Hierarchical sequence to sequence model with attention-based decoder, trained on single speaker dataset.
result Improved voice conversion quality using mel spectrograms and wavenet vocoder.
Image compression techniques reveal network structure for shipping box optimization.
problem Designing and planning shipping box networks between nodes.
method Image transformations and Bayesian reinforcement learning.
result Learned network structure can recommend future connectivity.
Sequence-to-sequence models predict resource usage for co-scheduled jobs in data centers.
problem Challenges in co-scheduling jobs due to resource interference and inefficiencies.
method Sequence-to-sequence models based on recurrent neural networks for workload interference prediction.
result Models accurately forecast resource usage trends from job profiles, improving scheduling decisions.
MARGE learns to reconstruct text by paraphrasing, achieving strong performance across multiple tasks.
problem Training sequence-to-sequence models with limited supervision.
method Unsupervised multi-lingual multi-document paraphrasing objective.
result Strong zero-shot performance on document translation and various tasks in multiple languages.
This study improves text summarization and fake news detection using neural models.
problem Improving text summarization and detecting fake news.
method Exploring and comparing different neural models for text summarization, including LSTM-encoder-decoder, pointer-generator networks, and transformers. Applying these models as a feature extractor for fake news detection.
result The proposed neural models enhance text summarization and improve fake news detection accuracy.
BART pretrains sequence-to-sequence models by corrupting text and reconstructing it.
problem Improving natural language generation, translation, and comprehension.
method BART uses a denoising autoencoder trained on a Transformer architecture with various noising techniques.
result BART achieves state-of-the-art performance on various NLP tasks with minimal training resources.
Improved text summarization using belief propagation on weighted bipartite graphs.
problem Text summarization from a graph theory perspective.
method Generalized belief propagation algorithm for weighted bipartite graphs.
result Our algorithm outperforms greedy methods in text summarization tasks.
SuTaT creates dialogue summaries for tete-a-tetes without labeled data.
problem Lack of high-quality paired dialogue-summary data.
method Unsupervised model for tete-a-tetes, modeling customer and agent roles separately.
result SuTaT outperforms on automatic and human evaluations.
Model creates summaries of patient notes to save time and reduce errors.
problem Improper summarization of patient notes leads to inefficiencies and errors.
method Developed an LSTM model to sequentially label topics in history of present illness notes.
result Achieved an F1 score of 0.876, indicating the model's effectiveness.
This review compares extractive and abstractive summarization methods.
problem Improving abstractive summarization in natural language processing.
method Compared various approaches including supervised and unsupervised methods, deep learning, and NLP.
result Current research uses combinations of approaches, but abstractive summarization remains unsolved.
Develops efficient algorithms for summarizing large datasets.
problem Summarizing large datasets efficiently.
method Two-stage submodular approach for linear-time optimization.
result Achieves nearly-optimal solutions for summarizing data.
We introduce Courant algebroids, providing definitions, some historical notes, and some elementary properties. Next, we summarize basic properties of graded manifolds. Then, drawing on the work of Roytenberg and others, we introduce the graded or supergraded language demonstrating a cochain complex / cohomology for (ge…
BERT helps summarize lecture content efficiently.
problem Efficiently summarize lecture content for students.
method Used BERT for text embeddings and KMeans clustering.
result Improved text summarization for lecture content.
Differentiable submodular maximization combines learning and optimization.
problem Learning and optimizing submodular functions separately.
method Interpreting greedy maximization as distributions, smoothing, and differentiating.
result The approach optimizes submodular functions with theoretical guarantees.
Mathematical framework for language models processes text and predicts next tokens.
problem Understanding and optimizing the performance of large language models.
method Describes encoding, prediction models, learning from data, and deployment of LLMs.
result Demonstrates remarkable empirical successes and provides a platform for further research.
Improves document summarization by combining word embeddings and n-grams.
problem Exact word matching fails to measure semantic similarity between sentences.
method Uses deep embedding features and tf-idf features to improve sentence similarity measure; builds an improved sentence similarity graph; employs a submodular objective function; develops a Transformer-based compression model.
result Outperforms tf-idf based approach and achieves state-of-the-art performance on DUC04 dataset.
Transformer model improves source code summarization.
problem Generating readable summaries of source code.
method Transformer model with self-attention mechanism for code representation.
result Transformer model outperforms state-of-the-art techniques.
Compact Recurrent Transformer (CRT) improves Transformer efficiency for long sequences.
problem Efficiently scaling Transformer architecture to long sequences with limited compute resources.
method Combines shallow Transformer models with recurrent neural networks and persistent memory.
result CRT achieves comparable or superior performance to full-length Transformers with shorter segments and reduced FLOPs.
This work tackles extractive compression by formulating it as tree transduction.
problem Extractive compression as a challenging natural language processing problem.
method Formulated as a parse tree transduction problem, using a deep neural model with Long Short-Term Memory extended to consider parent-child relationships.
result Achieves state-of-the-art performance on sentence compression benchmarks.
Survey of machine learning methods for spatiotemporal sequence forecasting.
problem Forecasting multi-step future of spatiotemporal systems based on past observations.
method Defined STSF problem, classified into subcategories, identified challenges, reviewed existing methods.
result No unified perspective on machine learning for STSF previously existed.