New insights into how encoder-decoder networks generate attention matrices.
problem Understanding how encoder-decoder networks use attention matrices.
method Decomposing hidden states into temporal and input-driven components.
result Attention matrices are formed based on task requirements, not architecture type.
The paper uses neural networks to forecast time series data.
problem Forecasting high-dimensional stationary processes.
method Encoder-decoder neural network structure to model past observations.
result Upper bounds for forecast error under specific assumptions.
This work uses encoder-decoder networks to denoise one-dimensional signals by aligning clean and noisy signal latent representations.
problem Noise removal in one-dimensional signals, especially in medical and motion signals.
method Encoder-decoder architecture with adversarial learning to align clean and noisy signal latent representations.
result Better performance on electrocardiogram and motion signal denoising compared to learning-based and non-learning approaches.
Unified framework explains geometric properties of CNNs.
problem Understanding why encoder-decoder CNNs perform well.
method Unified mathematical framework based on recent neural network theories.
result Encoder-decoder CNNs are related to nonlinear basis representation using combinatorial convolution frames.
Memory-augmented neural networks improve machine translation performance.
problem Improving machine translation accuracy and flexibility.
method Evaluation of Neural Turing Machines and Differentiable Neural Computers for machine translation tasks.
result Memory-augmented neural networks perform similarly to attentional encoders on Vietnamese to English tasks but have lower BLEU scores on Romanian to English tasks.
RED-SC improves deep subspace clustering efficiency.
problem Slow convergence in training deep subspace clustering models.
method Residual Encoder-Decoder network with skip-layer connections and self-expressive layer.
result Training converges much faster with RED-SC.
Although neural machine translation with the encoder-decoder framework has achieved great success recently, it still suffers drawbacks of forgetting distant information, which is an inherent disadvantage of recurrent neural network structure, and disregarding relationship between source words during encoding step. Wher…
Paper improves deep learning models for cardiac potential reconstruction.
problem Improving generalization of sequence models for cardiac potential reconstruction.
method Constrained stochasticity and global aggregation of temporal information in latent space.
result Improved generalization of inverse reconstruction networks.
In this paper, we propose a novel neural network model called RNN Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixed-length vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of t…
Paper improves multi-step chord prediction in jazz music.
problem Chord sequence prediction accuracy is poor in multi-step scenarios.
method Aggregated multi-scale encoder-decoder network architecture.
result Our model outperforms state-of-the-art methods in accuracy and perplexity.
Study efficient neural operator learning using variation spaces.
problem Operator learning using encoder-decoder neural networks.
method Introduce variation space for nonlinear operators, establish approximation bounds.
result Algebraic approximation and learning rates for polynomially decaying input and output encoding errors.
This work combines GANs and A3C for high-resolution image compression.
problem Image compression for high-resolution images without loss of quality.
method Hybrid approach using GANs and A3C for end-to-end learning.
result Improves PSNR for high-resolution images through end-to-end learning.
A novel capsule network model improves surrogate modeling and uncertainty quantification from sparse data.
problem Surrogate modeling and uncertainty quantification of systems from sparse data.
method Adapted Capsule Network (CapsNet) architecture into image-to-image regression encoder-decoder network.
result The proposed approach accurately, efficiently, and robustly predicts responses for arbitrary diffusion fields.
In "extreme" computational imaging that collects extremely undersampled or noisy measurements, obtaining an accurate image within a reasonable computing time is challenging. Incorporating image mapping convolutional neural networks (CNN) into iterative image recovery has great potential to resolve this issue. This pape…
Study clusters bank customers using LSTM and DTW.
problem Efficiently segmenting bank customers for targeted offers.
method Encoder-decoder LSTM network and Dynamic Time Warping (DTW).
result Hybrid method yields more accurate clusters.
New insights into encoder-decoder structures using information measures.
problem Understanding the role of encoder-decoder design in machine learning.
method Using information sufficiency and mutual information loss concepts.
result Characterizes the expressiveness loss in encoder-decoder designs.
Improves neural network regression performance using encoder-decoder approach.
problem Suboptimal regression results from classical methods in neural networks.
method Proposes a new method using an encoder-decoder approach for regression.
result Shows improved performance on various real-world datasets.
Image denoising is always a challenging task in the field of computer vision and image processing. In this paper, we have proposed an encoder-decoder model with direct attention, which is capable of denoising and reconstruct highly corrupted images. Our model consists of an encoder and a decoder, where the encoder is a…
In this paper, we propose a dictionary update method for Nonnegative Matrix Factorization (NMF) with high dimensional data in a spectral conversion (SC) task. Voice conversion has been widely studied due to its potential applications such as personalized speech synthesis and speech enhancement. Exemplar-based NMF (ENMF…
Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates…
A model learns stock trading rules from raw prices using encoder-decoder neural network.
problem Extracting features from long price sequences for profitable trading rules.
method Neural encoder-decoder framework combined with DRL.
result The model outperforms state-of-the-art models in dynamic environments.
CGNP embeds functional processes into latent vectors using graph neural networks.
problem Embedding and sampling functional processes over arbitrary domains.
method CGNP employs graph neural networks to embed and decode functional processes.
result CGNP effectively samples encoded functions over any domain.
Unified framework for generative models incorporating VAE and GAN.
problem Flexible incorporation of diverse measures of probability distance in generative models.
method Unified f-divergence generative model (f-GM) that incorporates both VAE and f-GAN.
result Unified f-GM enables flexible design of f-divergence functions without changing network structure.
In this paper, we compare different types of Recurrent Neural Network (RNN) Encoder-Decoders in anomaly detection viewpoint. We focused on finding the model that can learn the same data more effectively. We compared multiple models under the same conditions, such as the number of parameters, optimizer, and learning rat…
SummaryNet automates video summarisation using deep learning.
problem Creating informative video summaries from videos.
method Two-stream convolutional network for spatial and temporal features, encoder-decoder model for salient features, sigmoid regression with LSTM for frame probability.
result SummaryNet achieves comparable or better results than state-of-the-art methods on benchmark datasets.
Improved VAE models avoid posterior collapse in text modeling.
problem Posterior collapse in VAEs leads to poor data manifold parameterization.
method Coupled-VAE couples a VAE with a deterministic autoencoder to improve encoder and decoder parameterizations.
result Coupled-VAE consistently improves results in probability estimation and latent space richness.
This study improves text summarization and fake news detection using neural models.
problem Improving text summarization and detecting fake news.
method Exploring and comparing different neural models for text summarization, including LSTM-encoder-decoder, pointer-generator networks, and transformers. Applying these models as a feature extractor for fake news detection.
result The proposed neural models enhance text summarization and improve fake news detection accuracy.
An encoder-decoder model detects anomalies in manufacturing data.
problem Detecting and predicting anomalies in sequential sensor data.
method Encoder-decoder architecture for unsupervised anomaly detection.
result The model identifies anomalies and predicts future states in manufacturing processes.
Paper explores neural network approximations on sphere domains.
problem Approximating functionals on sphere domains using neural networks.
method Encoder-decoder framework with spherical harmonics for infinite-dimensional domain.
result Approximation rates of neural networks with different encoder structures.
Quaternion CNN outperforms traditional CNN in color image reconstruction.
problem Efficient image processing with small or heterogeneous datasets.
method Introducing quaternion-valued convolutional neural networks (QCNN) to learn internal and external relations.
result QCAE outperforms CAE in reconstructing unseen color images.
Streaming ASR with transformer achieves low WER.
problem Real-time ASR with speech recognition.
method Time-restricted self-attention and triggered attention mechanisms.
result 2.8% and 7.2% WER for LibriSpeech test data.
We address the problem of image translation between domains or modalities for which no direct paired data is available (i.e. zero-pair translation). We propose mix and match networks, based on multiple encoders and decoders aligned in such a way that other encoder-decoder pairs can be composed at test time to perform u…
Paper proposes a deep learning architecture for generating long stories from images.
problem Maintaining context in long event sequences for visual storytelling.
method Hierarchical deep learning architecture with encoder-decoder networks and natural language descriptions.
result Our method outperforms state-of-the-art techniques on automatic evaluation metrics.
Hybrid model combines RNNs, encoders-decoders, and Transformers for sequence tasks.
problem Sequence labelling tasks
method Combination of bidirectional RNNs, encoder-decoder, and Transformer models
result Results are close to state-of-the-art and better for some tasks
OpTorch optimizes deep learning for resource-limited environments.
problem Resource constraints in deep learning training.
method Optimized deep learning pipelines in training time and memory.
result Achieved similar accuracy to existing libraries with reduced memory usage.
Paper proposes GANs for generating business process suffixes and remaining times.
problem Generating accurate suffixes and remaining times for business process models.
method Encoder-decoder GAN architecture with Gumbel-Softmax distribution for categorical sequences.
result The approach outperforms baselines in suffix and remaining time prediction accuracy.
Survey reviews recent advances in learning graph representations over time.
problem Challenges in learning and inference for evolving graphs.
method Categorizes models from encoder-decoder perspective, analyzes techniques.
result Highlights directions for future research in dynamic graph representation learning.
A novel feedbackward approach for semantic segmentation reduces parameter count.
problem Semantic segmentation challenges in terms of model complexity and performance.
method Reverse-direction encoder-decoder architecture using the same network for both encoding and decoding.
result Significantly outperforms other models using only 13 VGG-16 layers, achieving higher IoU scores.
A new model trains prior and encoder/decoder networks simultaneously for efficient generation.
problem Complex autoregressive prior in VQ-VAE models leads to slow generation.
method Builds a diffusion bridge between continuous and non-informative prior distributions.
result Model is competitive and efficient in optimization and sampling.
Improved online AED models with multi-stage training and multi-task learning.
problem Enhance performance of online attention-based encoder-decoder models.
method Three-stage training with character encoder, BPE encoder, and attention decoder; multi-task learning at character and BPE levels; transfer learning from bidirectional encoder.
result 35% and 10% relative improvement over baselines for smaller and bigger models, respectively.
Encoder-decoder GANs architectures (e.g., BiGAN and ALI) seek to add an inference mechanism to the GANs setup, consisting of a small encoder deep net that maps data-points to their succinct encodings. The intuition is that being forced to train an encoder alongside the usual generator forces the system to learn meaning…
Convolutional network converts speaker voices without text.
problem Speaker conversion without text-based methods.
method Fully convolutional wav-to-wav network with ASR pre-training.
result Successfully converts TTS robot's voice to narrated audiobook voices.
Transmeter quickly measures transferability between different datasets.
problem Measuring transferability between heterogeneous multivariate datasets.
method Pre-trained source model, adversarial network, and encoder-decoder architecture.
result Transmeter provides the most accurate transferability measurement up to 10.3 times faster.
Graph U-Nets improve graph representation learning.
problem Challenges in learning representations for graph data.
method Proposed novel graph pooling and unpooling operations in a graph U-Nets architecture.
result Graph U-Nets achieve better performance on node and graph classification tasks.
Study improves calibration of NMT models, especially EOS and attention uncertainty.
problem Improper functioning of beam-search inference due to miscalibrated NMT models.
method Design and apply recalibration methods based on EOS and attention uncertainty signals.
result Improved accuracy and better sequence-level calibration of NMT models.
This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows si…
We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder-decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural netwo…
CRRN detects printer defects in SPI data of SMT boards.
problem Detecting defects in solder paste inspection data.
method Convolutional Recurrent Reconstructive Network (CRRN) with CSTMs and ST-Attention.
result CRRN outperforms conventional models in anomaly detection.