Convolutional model disaggregates electricity consumption data.
problem Disaggregating aggregate electricity consumption data into individual appliance usage.
method Gated linear unit convolutional layers and residual blocks refine neural network output. Partially overlapped sequences are averaged for final output.
result The proposed model outperforms existing models in disaggregating various appliance usage.
CLVSA predicts financial market trends using LSTM and attention mechanisms.
problem Predicting trends in financial markets due to complex interactions.
method Hybrid model combining LSTM, sequence-to-sequence, attention, and convolutional LSTM.
result CLVSA outperforms basic models in predicting financial market trends.
A new model improves text summarization by integrating topic information.
problem Improving the coherence, diversity, and informativeness of text summarization.
method Integrates topic information into ConvS2S model and uses SCST for optimization.
result The proposed model outperforms state-of-the-art methods in abstractive summarization.
A deep learning model predicts traffic conditions over multiple steps.
problem Multistep traffic forecasting on road networks.
method Attention Graph Convolutional Sequence-to-Sequence model (AGC-Seq2Seq) with attention mechanism.
result AGC-Seq2Seq model outperforms other models in multistep traffic prediction.
STConvS2S improves weather forecasting using only convolutional layers.
problem Predicting future weather conditions more accurately.
method Proposes a deep learning architecture combining spatiotemporal convolutional layers.
result Outperforms state-of-the-art architectures for forecasting tasks.
Deep learning model predicts traffic flows across entire network for multiple steps ahead.
problem Accurately forecasting future traffic flows across all network links.
method Spatial-Temporal Sequence to Sequence (STSeq2Seq) model combining seq2seq and graph convolution.
result STSeq2Seq achieves state-of-the-art performance in traffic forecasting.
ConvS2S-VC converts voice characteristics and pitch contour using a fully convolutional seq2seq model.
problem Voice conversion with preservation of pitch contour and duration.
method Fully convolutional seq2seq architecture with conditional batch normalization.
result ConvS2S-VC outperforms baseline methods in sound quality and speaker similarity.
CloudLSTM forecasts geospatial point-cloud data streams accurately.
problem Forecasting over geospatial point-cloud data streams.
method Introduces CloudLSTM, a recurrent neural model with a Dynamic Point-cloud Convolution (DConv) operator.
result CloudLSTM outperforms competitor models in long-term predictions for point-cloud data.
Dual-CLVSA predicts financial markets using both trading data and sentiment measurements.
problem Predicting financial markets with complex interactions and emotional influences.
method Hybrid convolutional LSTM-based variational sequence-to-sequence model with attention.
result Dual-CLVSA effectively fuses trading data and sentiment measurements, improving prediction performance.
A new framework predicts multi-step time series with stability and efficiency.
problem General probabilistic multi-step time series regression challenges.
method Sequence-to-Sequence Neural Networks, Quantile Regression, Direct Multi-Horizon Forecasting, Forking-sequences training scheme.
result Framework accommodates temporal and static covariates, learning across multiple series.
TCNs can approximate complex input-output maps with limited memory.
problem Approximating complex input-output maps with limited memory.
method Proved TCNs can approximate a wide class of input-output maps with arbitrary error tolerance.
result Deep ReLU TCNs can approximate input-output maps with finite memory to arbitrary error.
Convolutional autoencoding improves long text reconstruction.
problem Text reconstruction quality decreases with text length.
method Sequence-to-sequence, purely convolutional and deconvolutional autoencoding.
result Better at reconstructing and correcting long paragraphs.
New model controls memory in seq2seq tasks, revealing learning regimes.
problem Understanding memory in seq2seq tasks using neural networks.
method Introducing a stochastic switching-Ornstein-Uhlenbeck (SSOU) model to control memory and a measure of non-Markovianity.
result Two learning regimes emerge from the interplay of time scales in the SSOU process.
Analyzes sequence-to-sequence models for time series forecasting.
problem Choosing between sequence-to-sequence models and classical time series models.
method Theoretical analysis of sequence-to-sequence models for time series forecasting.
result Provides a quantitative guide for practitioners.
A novel sequence-to-sequence model predicts missing sensor data.
problem Missing sensor data in sequences.
method Formulated a novel sequence-to-sequence model using forward and backward RNNs.
result The model produces the lowest errors in 12% more cases than the current state-of-the-art.
Paper defines and evaluates attention mechanisms for sequence to sequence tasks.
problem Improving sequence to sequence mapping tasks.
method Mathematical definition and evaluation of attention mechanisms in sequence to sequence models.
result Models using greater attention perform better and converge faster.
Sequence learning improves query expansion in information retrieval.
problem Improving query expansion in information retrieval systems.
method Used sequence to sequence algorithms to extract keywords from sentence embeddings and trained a neural network on open datasets.
result Sequence to sequence models can capture complex query expansion relations in word embeddings.
Deep learning models improve electric load forecasting accuracy.
problem Accurate short-term electric load forecasting remains challenging.
method Comprehensive evaluation of various deep learning architectures on real-world datasets.
result Deep learning models outperform traditional methods in electric load forecasting.
Novel graph model forecasts urban traffic with reduced spatial complexity.
problem Challenges in traffic forecasting due to spatio-temporal complexity, especially in urban environments.
method MW-TGC network model that combines spatial and temporal dependencies using multi-weighted adjacency matrices and graph convolution operations.
result MW-TGC network outperforms other models in urban-core and urban-mix sites, reducing variance in heterogeneous environments.
Transformers can approximate any sequence-to-sequence function, surprising given their complexity.
problem Understanding the expressive power of Transformer models for sequence-to-sequence functions.
method Established that Transformers are universal approximators of continuous permutation equivariant sequence-to-sequence functions with compact support, and extended this to arbitrary functions using positional encodings.
result Transformers are universal approximators of arbitrary continuous sequence-to-sequence functions on a compact domain.
Parallel decoding speeds up deep autoregressive models.
problem Sequential generation limits deep autoregressive models' speed.
method Blockwise parallel decoding scheme for multiple time steps.
result Up to 7x speedup in wall-clock time with no loss in quality.
This research explores various sampling methods and probability distributions for hard alignment in sequence-to-sequence TTS synthesis.
problem Improving alignment accuracy in sequence-to-sequence text-to-speech synthesis.
method Investigated various sampling methods (greedy, beam, random) and probability distributions (Bernoulli, Concrete) for hard alignment.
result Deterministic search is more preferable than stochastic search for natural alignment transition.
Self-regulation improves sequence-to-sequence learning by choosing feedback types.
problem Different types of feedback have varying costs and effects on learning.
method Self-regulation strategies decide when to ask for different types of feedback.
result Self-regulator discovers optimal cost-quality trade-off by mixing feedback types.
A planning mechanism improves sequence-to-sequence model performance.
problem Improving sequence-to-sequence model performance for tasks like translation and graph analysis.
method Developed a model that plans ahead using attention and differentiable operations.
result Outperforms strong baselines on character-level translation, graph analysis, and question generation.
SequenceR uses seq-to-seq learning to fix bugs in code.
problem Fixing bugs in code using machine learning.
method Sequence-to-sequence learning with copy mechanism, trained on curated code samples.
result SequenceR can perfectly predict and find correct patches for bugs.
A new loss function speeds up sequence-to-sequence models for continuous outputs.
problem Slow and memory-intensive softmax layer limits vocabulary size and translation quality.
method Proposes a probabilistic loss and continuous embedding layer training/inference procedure.
result Models achieve up to 2.5x speed-up in training time with similar translation quality.
Investigates neural TTS systems for Japanese and English.
problem Improving neural TTS systems for high-quality speech synthesis.
method Comparative study of neural sequence-to-sequence TTS vs. DNN pipeline TTS, varying model architecture, parameter size, and language.
result A neural sequence-to-sequence TTS system requires sufficient model parameters and a powerful encoder for high-quality speech synthesis.
This work improves sequence-to-sequence models using amortized variational inference for context vectors.
problem Improving the performance of sequence-to-sequence models, especially in hard tasks like document summarization and video captioning.
method The approach treats context vectors as latent variables and infers their posteriors using an approximate finite mixture model via amortized variational inference.
result The method yields stronger generalization capacity and improved effectiveness over state-of-the-art alternatives.
We present a recurrent encoder-decoder deep neural network architecture that directly translates speech in one language into text in another. The model does not explicitly transcribe the speech into text in the source language, nor does it require supervision from the ground truth source language transcription during t…
Seq2seq models predict complex multi-physics systems' time evolution.
problem Predicting the time-evolution of complex multi-physics systems.
method Sequence-to-sequence models applied to multi-physics simulations.
result Seq2seq models accurately emulate complex systems and predict their evolution.
Generative models have long been the dominant approach for speech recognition. The success of these models however relies on the use of sophisticated recipes and complicated machinery that is not easily accessible to non-practitioners. Recent innovations in Deep Learning have given rise to an alternative - discriminati…
Quaternion CNNs improve speech recognition with fewer parameters.
problem Efficient end-to-end speech recognition with minimal parameters.
method Integrating quaternion algebra into CNNs for speech feature processing.
result Quaternion CNNs achieve lower phoneme error rates with fewer parameters.
Study on RL from human bandit feedback for sequence-to-sequence learning, showing reliability and learnability.
problem Reliability and learnability of human bandit feedback for RL from sequence-to-sequence learning.
method Investigated reliability of human bandit feedback, analyzed influence on reward estimator learnability, and tested improvements with regression-based reward estimator.
result Improvements of over 1 BLEU can be achieved by integrating a regression-based reward estimator trained on cardinal feedback into RL for NMT.
Model predicts variable length time series outputs.
problem Fixed-length prediction in recurrent networks.
method Sequence to sequence model with dynamic output length.
result Model balances prediction accuracy and length.
Study improves financial chatbot command understanding.
problem Improving chatbot command understanding for financial contexts.
method Sequence to sequence learning and Multi-Task Learning techniques.
result Enhanced performance in intent and content extraction.
Study improves voice conversion model with Mel-spectrogram augmentation.
problem Insufficient speech pairs data for training sequence-to-sequence voice conversion models.
method Experimented with Mel-spectrogram augmentation using SpecAugment policies and proposed new augmentation policies.
result Time axis warping policies showed better performance in training the voice conversion model.
Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work…
Paper proposes deep learning models for k-NN classification.
problem Imbalanced datasets and complex feature vectors in classification.
method Sequence to sequence model and memory network models.
result Models outperform k-NN and other state-of-the-art models.
The study forecasts, reconstructs, and selects features of ocean waves using neural networks.
problem Forecasting, reconstructing, and feature selection of ocean waves.
method Recurrent and sequence-to-sequence neural networks, Bayesian hyperparameter optimization, Elastic Net method.
result Proposed methods outperform alternatives in significant wave height reconstruction.
Study reveals how machine translation models can leak data privacy.
problem Detecting if a specific data sample was used to train machine translation models.
method Defined membership inference problem for sequence-to-sequence models, tested on machine translation models.
result Machine translation models can leak private information about training data.
We describe a fully data driven model that learns to perform a retrosynthetic reaction prediction task, which is treated as a sequence-to-sequence mapping problem. The end-to-end trained model has an encoder-decoder architecture that consists of two recurrent neural networks, which has previously shown great success in…
Survey on RL for seq2seq models to solve exposure bias and inconsistency.
problem Exposure bias and inconsistency in seq2seq models.
method Combining RL methods with seq2seq models.
result Improved seq2seq performance through RL.
Deep learning models predict solar irradiance for short-term forecasts.
problem Accurate prediction of solar irradiance for renewable energy integration.
method Sequence-to-sequence LSTM models for GHI forecasting, incorporating spatial-temporal features.
result LSTM models outperform traditional techniques in short-term GHI forecasting.
Lingvo is a flexible framework for deep learning sequence models.
problem Sequence-to-sequence modeling challenges in deep learning.
method Modular, scalable, and customizable Tensorflow framework.
result Supports distributed training and quantized inference.
Sequence-to-sequence models predict resource usage for co-scheduled jobs in data centers.
problem Challenges in co-scheduling jobs due to resource interference and inefficiencies.
method Sequence-to-sequence models based on recurrent neural networks for workload interference prediction.
result Models accurately forecast resource usage trends from job profiles, improving scheduling decisions.
Improved speech recognition with language model integration in sequence-to-sequence models.
problem Improving word error rate in speech recognition models.
method Log-linear combination of acoustic and language models with per-token renormalization.
result The proposed method shows good improvements over standard model combination on Librispeech system.
Softmax attention approximates complex functions and subsumes many known universal approximators.
problem Universal approximation of continuous sequence-to-sequence functions.
method Interpolation-based analysis of attention's internal mechanism, showing its ability to approximate ReLU functions.
result Softmax attention is a universal approximator for continuous sequence-to-sequence functions.
Sockeye is an open-source toolkit for neural machine translation.
problem Improving Neural Machine Translation (NMT) models and techniques.
method Scalable training and inference for three NMT architectures, including attentional, self-attentional, and fully convolutional networks.
result Sockeye achieves competitive BLEU scores across different NMT architectures, including a best score for its transformer implementation.