Seq2seq ASR adapts to speakers, improving performance by 25%.
problem Speaker adaptation for seq2seq ASR systems to match conventional methods.
method Applied Kullback-Leibler divergence and Linear Hidden Network adaptation to seq2seq models.
result 25% relative word error rate improvement with seq2seq model adaptation.
StackSeq2Seq improves route finding on graphs using deep neural networks.
problem Finding the shortest path between graph nodes.
method Dual Encoder Seq2Seq architecture, context vector, homotopy continuation.
result Increased accuracy in learning shortest routes on graphs.
A deep learning model predicts traffic conditions over multiple steps.
problem Multistep traffic forecasting on road networks.
method Attention Graph Convolutional Sequence-to-Sequence model (AGC-Seq2Seq) with attention mechanism.
result AGC-Seq2Seq model outperforms other models in multistep traffic prediction.
Method uses Seq2Seq learning to automatically generate recovery commands for ICT systems.
problem Manual decision-making for recovery commands is time-consuming and error-prone.
method Seq2Seq neural network model trained on past logs and commands.
result The model can estimate accurate recovery commands from new failures.
Graph2Seq converts graph inputs to sequences with attention-based neural networks.
problem Converting graph inputs to sequences for machine learning tasks.
method Graph-to-sequence neural encoder-decoder model with attention-based LSTM.
result Achieves state-of-the-art performance on various tasks.
Survey of seq2seq models for neural text summarization.
problem Improving fluency and quality of text summaries.
method Comprehensive review of seq2seq models for abstractive text summarization.
result Benchmarking of two models on recent datasets.
DeepProbe uses seq2seq models to improve query understanding and chatbot design.
problem Improving query understanding and chatbot design efficiency.
method Attention-based seq2seq recurrent neural network for information extraction and active user interactions.
result DeepProbe achieves significant improvements in query understanding and chatbot efficiency.
New model controls memory in seq2seq tasks, revealing learning regimes.
problem Understanding memory in seq2seq tasks using neural networks.
method Introducing a stochastic switching-Ornstein-Uhlenbeck (SSOU) model to control memory and a measure of non-Markovianity.
result Two learning regimes emerge from the interplay of time scales in the SSOU process.
GeoSeq2Seq integrates geometric principles into neural networks for improved predictions.
problem Improving the geometric representation of neural network embeddings.
method Proposes a new Seq2Seq network that uses Fisher information metric to encode latent embeddings as Fisher kernels of Gaussian Mixture Models.
result GeoSeq2Seq outperforms traditional methods by 10-15% in predicting shortest routes on graphs.
Survey on RL for seq2seq models to solve exposure bias and inconsistency.
problem Exposure bias and inconsistency in seq2seq models.
method Combining RL methods with seq2seq models.
result Improved seq2seq performance through RL.
Paper proposes Seq2Seq models for multimodal sentiment analysis.
problem Learning representations from multiple modalities in machine learning.
method Two unsupervised Seq2Seq models for multimodal sentiment analysis.
result Seq2Seq models improve F1 Score by twelve points in Bimodal sentiment analysis.
Improves seq2seq speech recognition by addressing overconfidence and incomplete transcriptions.
problem Overconfidence and incomplete transcriptions in seq2seq speech recognition.
method Proposed practical solutions to address overconfidence and incomplete transcriptions using a trigram language model.
result Achieved competitive speaker independent word error rates (6.7%) with a trigram language model.
Neural networks learn symbolic structure to perform compositional tasks.
problem How neural networks perform well on compositional tasks without explicit representations.
method ROLE analysis to uncover symbolic structure in recurrent neural networks.
result Neural networks converge to solutions that implicitly represent symbolic structure.
Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work…
Neural machine translation used to convert CUDA to OpenCL.
problem Translating CUDA to OpenCL programs.
method Training input set generation, pre/post processing, case study.
result Improved accuracy in translating CUDA to OpenCL.
Seq2seq models predict complex multi-physics systems' time evolution.
problem Predicting the time-evolution of complex multi-physics systems.
method Sequence-to-sequence models applied to multi-physics simulations.
result Seq2seq models accurately emulate complex systems and predict their evolution.
New method embeds RNN Seq2Seq models to visualize spatiotemporal data.
problem Visualizing and interpreting spatiotemporal data in sequence prediction tasks.
method Embedding approach to visualize and interpret RNN Seq2Seq model representations.
result Embedding space projections of RNN Seq2Seq models capture spatiotemporal dynamics.
Seq2Seq models speed up epidemic model predictions.
problem Complex epidemic models are computationally expensive.
method Used deep seq2seq models as surrogates for complex models.
result Surrogates predict scenarios up to several thousand times faster.
Models learn to transform musical scores into expressive performances.
problem Creating large volumes of paired data for sequence transformations.
method Adapted GAN techniques to sequences, creating paired data through simple transformations.
result Models can plausibly invert transformations like quantization and removing voices.
New seq2seq model can copy entire spans, outperforming simpler models in editing tasks.
problem Editing documents or source code using seq2seq models with explicit token copying.
method Extended seq2seq model capable of copying entire input spans to output in one step, new training and inference methods.
result New model consistently outperforms simpler baselines in editing tasks of natural language and source code.
Graph-to-Tree Neural Networks improve structured input-output translation in tasks like semantic parsing and math word problems.
problem Improving performance on tasks like semantic parsing and math word problem solving.
method Graph-to-Tree Neural Networks, consisting of a graph encoder and a hierarchical tree decoder.
result Graph2Tree model outperforms or matches state-of-the-art models on neural semantic parsing and math word problem tasks.
Zero Initialization improves short-term load forecasting accuracy.
problem Improving the learning speed and accuracy of neural networks for load forecasting.
method Proposed and tested Zero Initialization (ZI) for weights of a single layer network, comparing with Xavier, He, and Identity initialization.
result ZI reduces the number of epochs and improves accuracy in short-term load forecasting.
Paper proposes a new neural machine translation method for wave data.
problem Limited real-world sensor data for continuous signal waves.
method Introduces window-based representation and iterative back-translation for wave data.
result Significant performance improvements in wave translation tasks.
Sequences have become first class citizens in supervised learning thanks to the resurgence of recurrent neural networks. Many complex tasks that require mapping from or to a sequence of observations can now be formulated with the sequence-to-sequence (seq2seq) framework which employs the chain rule to efficiently repre…
Seq2Seq models perform well in generating If-Then programs from natural language.
problem Creating If-Then programs for business process automation without technical expertise.
method Modeling If-Then programs as a sequence learning task using Seq2Seq approaches.
result Seq2Seq models can effectively generate If-Then programs from natural language.
Generative model improves ECG classification with limited data.
problem Poor performance of RNNs with limited channel ECG data.
method Generative Seq2Seq model to fill missing data, followed by discriminative learning.
result Generative approach outperforms standard RNNs in disease prediction.
Enhances persona-based conversation model for multi-turn dialogue.
problem Improving persona-based conversation models for multi-turn dialogue.
method Introduced additional input modality into hredGAN to capture external attributes.
result Persona hredGAN (phredGAN) outperforms existing models in multi-turn dialogue corpora. Improves spatio-temporal forecasting by reducing errors between training and inference.
problem Accumulation of small errors in Seq2Seq models during inference due to different distributions of training and inference phases.
method Curriculum learning based on Temporal Progressive Growing Sampling to replace some ground-truth context with generated predictions.
result Better models long-term dependencies and outperforms baseline approaches on two datasets.
Study improves financial chatbot command understanding.
problem Improving chatbot command understanding for financial contexts.
method Sequence to sequence learning and Multi-Task Learning techniques.
result Enhanced performance in intent and content extraction.
Unsupervised model detects healthcare fraud from patient visit data.
problem Detecting fraudulent healthcare bills from patient visit data.
method Uses LSTM and seq2seq models for anomaly detection, normalizes scores with EDF.
result Improves anomaly detection for high class imbalance problems.
Modeling latent dynamics in high-dimensional event sequences without prior knowledge.
problem Modeling latent dynamics in high-dimensional event sequences with unknown marker relations.
method Adversarial imitation learning framework decomposed into latent structural intensity model, efficient random walk model, and seq2seq discriminator.
result Effective detection of hidden network among markers and decent prediction for future events.
Solution for voice conversion with limited data using hierarchical seq2seq and attention models.
problem Voice conversion between speakers with limited parallel audio pairs.
method Hierarchical sequence to sequence model with attention-based decoder, trained on single speaker dataset.
result Improved voice conversion quality using mel spectrograms and wavenet vocoder.
Generative Adversarial Network repairs software bugs without labeled data.
problem Automated repair of software vulnerabilities.
method Adversarial learning approach mapping between source and target domains.
result Effective at repairing software vulnerabilities, close to seq2seq approaches.
Method converts speech with attention and context preservation.
problem Voice conversion with improved stability and efficiency.
method Sequence-to-Sequence learning with attention and context preservation.
result Synthesized speech quality comparable to advanced methods.
New techniques improve channel prediction in noisy wireless systems.
problem Predicting channels in wireless communication systems from noisy observations.
method Adapted sequence-to-sequence models and transformers with reverse positional encoding and reversed encoder outputs.
result Improved robustness and relationship capture in channel prediction models.
Unsupervised learning improves clinical predictions from medical time series.
problem Improving clinical decision making through unlabeled medical data.
method Evaluation of unsupervised representation learning on medical time series using sequence-to-sequence models.
result A forecasting Seq2Seq model with an attention mechanism achieves the best performance.
A new approach to distill unnormalized EBM for energy-based seq2seq models.
problem Training unnormalized EBM for energy-based seq2seq models is challenging.
method Relating the problem to distributional RL, proposing a general distillation approach.
result General approach applicable to any sequential EBM, illustrated on GAM experiments.
Proposes models to generate more interesting story endings.
problem Generating diverse and interesting story endings for a given context.
method Trains models to focus on keyphrases and promotes non-generic words.
result Models generate more diverse and interesting story endings.
Deep learning model predicts traffic flows across entire network for multiple steps ahead.
problem Accurately forecasting future traffic flows across all network links.
method Spatial-Temporal Sequence to Sequence (STSeq2Seq) model combining seq2seq and graph convolution.
result STSeq2Seq achieves state-of-the-art performance in traffic forecasting.
Enhances dialogue model with persona attributes using adversarial learning.
problem Improving dialogue models to better capture speaker identity and topic.
method Adversarial learning framework with a dual discriminator system.
result phredGAN outperforms persona Seq2Seq model in various datasets.
GAMs combine autoregressive and log-linear components for data-efficient sequence learning.
problem Poor performance of standard autoregressive models under small-data conditions.
method Introduce Global Autoregressive Models (GAMs) combining autoregressive and log-linear components, trained in two steps.
result GAMs show a strong perplexity reduction over standard models in language modelling.
This paper uses a diffusion model to forecast electrical loads with uncertainty.
problem Uncertainties in electrical load forecasting due to renewable energy and external events.
method Diffusion-based Seq2Seq structure for epistemic uncertainty and robust additive Cauchy distribution for aleatoric uncertainty.
result Ability to separate and quantify both types of uncertainties in load forecasting.
BPE improves text-to-SQL generation by reducing training time and improving accuracy.
problem Improving text-to-SQL generation accuracy with neural models.
method Adapted Byte-Pair Encoding (BPE) for SQL generation, introduced a novel stopping criterion, and used AST BPE.
result Improved accuracy on 5 out of 6 English text-to-SQL tasks, reduced training time by 50%.
Proposes MVG-CRPS for robust multivariate forecasting.
problem Outliers in multivariate forecasting lead to significant errors.
method Integrates CRPS for MVG distributions, optimizing with MVG-CRPS.
result Improves robustness, accuracy, and uncertainty quantification.
ConvS2S-VC converts voice characteristics and pitch contour using a fully convolutional seq2seq model.
problem Voice conversion with preservation of pitch contour and duration.
method Fully convolutional seq2seq architecture with conditional batch normalization.
result ConvS2S-VC outperforms baseline methods in sound quality and speaker similarity.
The paper explores how to make neural networks extrapolate longer sequences.
problem Neural networks struggle to extrapolate beyond seen data, especially for longer sequences.
method The authors propose a model with separate content- and location-based attention mechanisms.
result Models with the proposed attention mechanisms are better at extrapolating longer sequences.
Improved text summarization using neural semantic encoders with hierarchical structure.
problem Capturing long-term dependencies in text summarization.
method Proposed a novel hierarchical Neural Semantic Encoder (NSE) model augmented with lemma and PoS tags.
result Significantly outperformed state-of-the-art models in ROUGE metric.
A new framework improves ASR alignment accuracy via optimal transport.
problem Peaky behavior and alignment inaccuracies in E2E ASR models.
method Differentiable alignment framework based on 1D optimal transport.
result Significant improvement in ASR alignment performance compared to CTC.