Study efficient neural operator learning using variation spaces.
problem Operator learning using encoder-decoder neural networks.
method Introduce variation space for nonlinear operators, establish approximation bounds.
result Algebraic approximation and learning rates for polynomially decaying input and output encoding errors.
Improved VAE models avoid posterior collapse in text modeling.
problem Posterior collapse in VAEs leads to poor data manifold parameterization.
method Coupled-VAE couples a VAE with a deterministic autoencoder to improve encoder and decoder parameterizations.
result Coupled-VAE consistently improves results in probability estimation and latent space richness.
Learning in the latent variable model is challenging in the presence of the complex data structure or the intractable latent variable. Previous variational autoencoders can be low effective due to the straightforward encoder-decoder structure. In this paper, we propose a variational composite autoencoder to sidestep th…
VED framework learns low-dimensional latent representations of physical systems.
problem Learning latent representations of complex physical systems.
method Variational Encoder-Decoder (VED) framework with KL divergence and covariance regularization.
result VED achieves lower-dimensional latent representations with improved feature disentanglement.
A new model trains prior and encoder/decoder networks simultaneously for efficient generation.
problem Complex autoregressive prior in VQ-VAE models leads to slow generation.
method Builds a diffusion bridge between continuous and non-informative prior distributions.
result Model is competitive and efficient in optimization and sampling.
In "extreme" computational imaging that collects extremely undersampled or noisy measurements, obtaining an accurate image within a reasonable computing time is challenging. Incorporating image mapping convolutional neural networks (CNN) into iterative image recovery has great potential to resolve this issue. This pape…
New model improves diversity in question generation over text.
problem Inadequate evaluation metrics for diverse questions.
method Proposed a variational encoder-decoder model and evaluation scheme.
result Improved diversity in generated questions without loss of quality.
Bidirectional VAE reduces parameters and improves image tasks.
problem Improving image reconstruction, classification, interpolation, and generation.
method Uses a single neural network for both encoding and decoding in both forward and backward directions.
result Bidirectional VAEs reduce parameters by almost 50% and slightly outperform unidirectional VAEs.
New insights into encoder-decoder structures using information measures.
problem Understanding the role of encoder-decoder design in machine learning.
method Using information sufficiency and mutual information loss concepts.
result Characterizes the expressiveness loss in encoder-decoder designs.
AutoBayes automates Bayesian graph exploration for robust machine learning.
problem Learning representations invariant to nuisance variations in machine learning.
method Automated Bayesian inference framework exploring different graphical models.
result Significant performance improvement with nuisance-invariant machine learning pipelines.
New insights into how encoder-decoder networks generate attention matrices.
problem Understanding how encoder-decoder networks use attention matrices.
method Decomposing hidden states into temporal and input-driven components.
result Attention matrices are formed based on task requirements, not architecture type.
An encoder-decoder model detects anomalies in manufacturing data.
problem Detecting and predicting anomalies in sequential sensor data.
method Encoder-decoder architecture for unsupervised anomaly detection.
result The model identifies anomalies and predicts future states in manufacturing processes.
Bayesian variational inference improves medical image segmentation confidence.
problem Improving interpretability and confidence in deep learning models for medical image segmentation.
method Encoder-decoder architecture based on variational inference for segmenting brain tumor images.
result The model segments brain tumors with both aleatoric and epistemic uncertainty.
Paper tackles skill transfer in RL for morphologically different agents.
problem Transfer skills between morphologically different reinforcement learning agents.
method Proposes a paired variational encoder-decoder model (PVED) for subspace learning.
result Demonstrates improved skill transfer efficiency compared to state-of-the-art methods.
Encoder-decoder networks using convolutional neural network (CNN) architecture have been extensively used in deep learning literatures thanks to its excellent performance for various inverse problems. However, it is still difficult to obtain coherent geometric view why such an architecture gives the desired performance…
Batch normalization with regularization turns deterministic autoencoders into generative models.
problem Creating generative models from deterministic autoencoders.
method Using batch normalization as a source of non-determinism and adding entropic regularization.
result Deterministic autoencoders can be transformed into generative models with similar performance to variational autoencoders.
The paper uses neural networks to forecast time series data.
problem Forecasting high-dimensional stationary processes.
method Encoder-decoder neural network structure to model past observations.
result Upper bounds for forecast error under specific assumptions.
Paper introduces a new method for generating diverse human motion predictions.
problem Stochastic human motion prediction with limited flexibility.
method Stochastically combines root variations with previous pose information in a recurrent network.
result Model generates more diverse motion sequences than existing techniques.
MIM learns useful representations with high mutual information.
problem Learning useful representations for downstream tasks.
method Symmetric Jensen-Shannon divergence and mutual information regularizer in an encoder/decoder framework.
result MIM learns high mutual information representations without posterior collapse.
IntroVAC learns interpretable latent subspaces for better image quality.
problem Difficulties in interpreting latent spaces and limitations in image generation.
method Introspective Variational Classifier (IntroVAC) using additional labels and adversarial training.
result Improved image quality and meaningful latent directions for fine-grained manipulation.
Hybrid model combines RNNs, encoders-decoders, and Transformers for sequence tasks.
problem Sequence labelling tasks
method Combination of bidirectional RNNs, encoder-decoder, and Transformer models
result Results are close to state-of-the-art and better for some tasks
TIMeSynC combines financial service interactions for intent prediction.
problem Aligning and learning from multi-domain, multi-resolution sequences for accurate intent prediction.
method An encoder-decoder transformer model addressing sequence alignment, temporal dynamics, and dynamic/static sequence combination.
result Significant improvement in intent prediction over existing methods.
Survey of factor analysis, PCA, variational inference, and VAE.
problem Dimensionality reduction and generative modeling of data.
method Variational inference, factor analysis, probabilistic PCA, and VAE.
result Derivation and explanation of ELBO, EM, and closed-form solutions.
RED-SC improves deep subspace clustering efficiency.
problem Slow convergence in training deep subspace clustering models.
method Residual Encoder-Decoder network with skip-layer connections and self-expressive layer.
result Training converges much faster with RED-SC.
Generating versatile and appropriate synthetic speech requires control over the output expression separate from the spoken text. Important non-textual speech variation is seldom annotated, in which case output control must be learned in an unsupervised fashion. In this paper, we perform an in-depth study of methods for…
CardiCat generates synthetic data for high-cardinality tabular datasets.
problem Learning complexities of high-cardinality categorical features in tabular data.
method Substitutes one-hot encoding with regularized dual encoder-decoder embedding layers.
result Generates high-quality synthetic data with a smaller parameter space.
Improved online AED models with multi-stage training and multi-task learning.
problem Enhance performance of online attention-based encoder-decoder models.
method Three-stage training with character encoder, BPE encoder, and attention decoder; multi-task learning at character and BPE levels; transfer learning from bidirectional encoder.
result 35% and 10% relative improvement over baselines for smaller and bigger models, respectively.
Encoder-decoder GANs architectures (e.g., BiGAN and ALI) seek to add an inference mechanism to the GANs setup, consisting of a small encoder deep net that maps data-points to their succinct encodings. The intuition is that being forced to train an encoder alongside the usual generator forces the system to learn meaning…
Memory-augmented neural networks improve machine translation performance.
problem Improving machine translation accuracy and flexibility.
method Evaluation of Neural Turing Machines and Differentiable Neural Computers for machine translation tasks.
result Memory-augmented neural networks perform similarly to attentional encoders on Vietnamese to English tasks but have lower BLEU scores on Romanian to English tasks.
Unified empirical and variational Bayes for unnormalized densities.
problem Approximating unnormalized densities using latent variable models.
method Formulate a latent variable model for Y=X+N(0,σ2Id), use ELBO as parametrization of Y's energy function, and estimate X with empirical Bayes least-squares. result UVB has higher capacity to approximate energy functions than MLPs in DEEN.
This paper analyzes VAE approximation errors in conditional exponential families.
problem Posterior collapse and approximation errors in VAEs.
method Analysis of ELBO objective and conditional exponential families.
result The ELBO optimizer pulls away from the likelihood optimizer towards a consistent subset of models.
SUMO provides unbiased log marginal likelihood estimation for latent variable models.
problem Biased estimates of log marginal likelihood in latent variable models.
method Randomized truncation of infinite series for unbiased estimation.
result Models trained with SUMO give better test-set likelihoods than standard methods.
A model learns stock trading rules from raw prices using encoder-decoder neural network.
problem Extracting features from long price sequences for profitable trading rules.
method Neural encoder-decoder framework combined with DRL.
result The model outperforms state-of-the-art models in dynamic environments.
In this paper, we propose a novel neural network model called RNN Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixed-length vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of t…
Deep learning models have shown state-of-the-art performance in many inverse reconstruction problems. However, it is not well understood what properties of the latent representation may improve the generalization ability of the network. Furthermore, limited models have been presented for inverse reconstructions over ti…
New approach uses text generation to boost AI agent development.
problem Lack of training data hinders AI agent development.
method Used encoder-decoder generative models, focusing on conditional variational auto-encoders.
result Significantly improved AI agent performance in low-resource cases.
We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Aut…
This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows si…
Proposes a variational autoencoder for long-term customer revenue forecasting.
problem Predicting long-term customer revenue from sparse and irregular transaction data.
method Variational Autoencoder (VAE) with flexible latent representation.
result Improves upon latest benchmarks in multiple real-world datasets.
Although neural machine translation with the encoder-decoder framework has achieved great success recently, it still suffers drawbacks of forgetting distant information, which is an inherent disadvantage of recurrent neural network structure, and disregarding relationship between source words during encoding step. Wher…
Image compression is an essential approach for decreasing the size in bytes of the image without deteriorating the quality of it. Typically, classic algorithms are used but recently deep-learning has been successfully applied. In this work, is presented a deep super-resolution work-flow for image compression that maps …
A new inference network learns undirected models from data.
problem Learning undirected models with hidden variables is hard.
method Train a single stochastic inference network from data, using a cost function over queries.
result The network can perform inference tasks like an undirected model without partition function.
The presence of noise is common in signal processing regardless the signal type. Deep neural networks have shown good performance in noise removal, especially on the image domain. In this work, we consider deep neural networks as a denoising tool where our focus is on one dimensional signals. We introduce an encoder-de…
Paper improves multi-step chord prediction in jazz music.
problem Chord sequence prediction accuracy is poor in multi-step scenarios.
method Aggregated multi-scale encoder-decoder network architecture.
result Our model outperforms state-of-the-art methods in accuracy and perplexity.
Paper tackles missing data in irregularly-sampled time series.
problem Modeling irregularly-sampled time series data.
method Encoder-decoder framework based on variational autoencoders and generative adversarial networks.
result Models achieve competitive or better classification results on irregularly-sampled multivariate time series.
BF-VAE estimates uncertainty from LF and HF QoI samples.
problem Balancing computational efficiency and numerical accuracy in uncertainty quantification.
method Bi-fidelity formulation of VAEs in latent space.
result BF-VAE improves accuracy with limited HF data.
Study clusters bank customers using LSTM and DTW.
problem Efficiently segmenting bank customers for targeted offers.
method Encoder-decoder LSTM network and Dynamic Time Warping (DTW).
result Hybrid method yields more accurate clusters.
We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder-decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural netwo…