NAOMI improves imputation accuracy for long-range sequences.
problem Missing value imputation in spatiotemporal data.
method Non-autoregressive deep generative model exploiting multiresolution structure.
result Significant improvement in imputation accuracy (60% reduction in average prediction error).
BiGG model efficiently generates sparse graphs with reduced complexity.
problem Challenges in scalable deep learning for sparse graphs.
method BiGG model, an autoregressive model that leverages graph sparsity.
result Graph generation time complexity reduced from O(n2) to O((n+m)logn). This review compares various deep generative models.
problem Training deep neural networks to model data distributions.
method Comprehensive comparison of VAEs, GANs, flows, energy models, and autoregressives.
result Trade-offs and interrelationships among different models.
Paper explores variable skipping to speed up range density estimation.
problem Efficiently estimating range densities over high-dimensional data.
method Variable skipping technique to accelerate range density estimation.
result 10-100x efficiency improvements in challenging high-quantile error metrics.
Deep state space model generates text without autoregressive feedback, avoiding biases.
problem Text generation biases and forgetting local nuances.
method Non-autoregressive deep state space model with independent noise and deterministic transition.
result Generative model on par with auto-regressive models, interpretable and without biases.
Deep neural network solves complex groundwater contaminant source identification.
problem Identifying groundwater contaminant sources in highly heterogeneous media.
method Deep autoregressive neural network surrogate for forward model, ILUES for inversion.
result Deep autoregressive neural network provides accurate approximation for high-dimensional model.
Paper proposes a forecasting model combining autoregressive models with spectral attention.
problem Time series forecasting across various domains.
method Combines deep autoregressive models with Spectral Attention (SA) module.
result SAAM consistently demonstrates improved forecasting accuracy compared to state-of-the-art approaches.
Parallel decoding speeds up deep autoregressive models.
problem Sequential generation limits deep autoregressive models' speed.
method Blockwise parallel decoding scheme for multiple time steps.
result Up to 7x speedup in wall-clock time with no loss in quality.
ARTree uses deep learning to infer tree topologies efficiently.
problem Efficient phylogenetic inference from tree topologies.
method Deep autoregressive model based on graph neural networks (GNNs).
result ARTree provides a flexible family of distributions over tree topologies.
Efficient autoregressive entity linking with correction for faster, more accurate results.
problem High computational cost and non-parallelizable decoding in autoregressive entity linking.
method Parallelizes autoregressive linking across all mentions, uses a shallow decoder, and adds a discriminative correction term.
result 70 times faster and more accurate than previous methods, outperforming state-of-the-art approaches.
Method uses autoregressive models to interpret neural network representations.
problem Understanding and quantifying information preserved in neural network layers.
method Trains autoregressive models to invert model representations and estimate mutual information.
result Mutual information between inputs and network layers decreases over training.
New model handles missing data effectively in autoregressive models.
problem Handling missing data in autoregressive models.
method Reinterpret existing models through missing data lens, introduce principled framework for incomplete datasets, active information acquisition.
result MO-ARM consistently outperforms imputation baselines across real-world benchmarks.
This paper introduces methods to handle discrete data by dequantization.
problem Handling discrete data in deep learning models.
method Dequantization framework, including importance-weighted and Rényi dequantization objectives, and autoregressive dequantization.
result Improved performance on uniform dequantization distributions and state-of-the-art negative log-likelihood on CIFAR10.
LMConv improves autoregressive models for image generation and completion.
problem Limited generation order in autoregressive models restricts their applicability.
method Introduces LMConv, a modified 2D convolution that allows arbitrary masks to be applied to weights.
result LMConv achieves improved performance on image density estimation and coherent completions.
We introduce a deep, generative autoencoder capable of learning hierarchies of distributed representations from data. Successive deep stochastic hidden layers are equipped with autoregressive connections, which enable the model to be sampled from quickly and exactly via ancestral sampling. We derive an efficient approx…
New method uses neural networks to solve statistical mechanics problems.
problem Statistical mechanics of systems with finite size.
method Variational autoregressive neural networks with reinforcement learning.
result Directly computes free energy, entropy, magnetizations, and correlations.
CARRNN tackles deep learning for sporadic data, improving prediction errors in healthcare.
problem Challenges in learning temporal patterns from sporadic multivariate longitudinal data.
method Developed a novel deep learning architecture combining RNN and CAR models, using a generalized discrete-time autoregressive model.
result CARRNN achieves the lowest prediction errors in multivariate time-series regression tasks.
Proposes a neural density estimator for anomaly detection using labeled data.
problem Improving anomaly detection performance with limited labeled data.
method Uses deep autoregressive neural density estimators trained with anomaly labels to maximize normal likelihood and minimize anomalous likelihood.
result Significantly improves anomaly detection performance with few labeled instances compared to existing methods.
Single autoregressive model outperforms ensemble methods in offline reinforcement learning.
problem Offline reinforcement learning with limited data and model errors.
method Infer system dynamics from data and optimize policies on model rollouts, using a single autoregressive model.
result Single autoregressive model achieves better performance than ensembles on the D4RL benchmark.
Neural networks improve gravitational-wave parameter estimation.
problem Estimating parameters of binary black hole systems from gravitational-wave data.
method Autoregressive normalizing flows for likelihood-free inference.
result Performance comparable to current best deep-learning approaches, with fast sampling.
NVAE improves VAE performance on large image datasets.
problem Improving variational autoencoder performance for large image datasets.
method Deep hierarchical VAE with depth-wise separable convolutions and batch normalization, residual parameterization of Normal distributions, and spectral regularization.
result NVAE achieves state-of-the-art results on MNIST, CIFAR-10, CelebA 64, and CelebA HQ datasets.
ARMA cell simplifies neural autoregressive modeling for time series.
problem Complex RNN cells are not always necessary and can be inferior.
method Introduces ARMA cell, a simpler, modular approach for neural time series modeling.
result The ARMA cell is competitive with popular alternatives in performance.
Paper offers fast deep learning approach for SSMs.
problem Approximate Bayesian inference in SSMs.
method Deep learning and variational inference.
result Fast approach for SSMs.
The paper compares DL models to WP curve modeling for forecasting with irregular shutdowns.
problem Forecasting wind power with irregular shutdowns due to redispatching.
method Compared autoregressive DL models to WP curve modeling.
result WP curve modeling achieves lower forecasting errors and is more computationally efficient.
This paper proposes CF-NADE, a neural autoregressive architecture for collaborative filtering (CF) tasks, which is inspired by the Restricted Boltzmann Machine (RBM) based CF model and the Neural Autoregressive Distribution Estimator (NADE). We first describe the basic CF-NADE model for CF tasks. Then we propose to imp…
A new drift detection method based on autoregressive models.
problem Concept drift in real-world data leads to decreased model performance.
method Autoregressive based drift detection method (ADDM).
result ADDM outperforms state-of-the-art drift detection methods.
PARNN improves ARNN with ARIMA feedback for accurate long-range forecasting.
problem Accurate long-range forecasting of complex time series data.
method Improves ARNN using ARIMA feedback, providing uncertainty quantification.
result PARNN outperforms state-of-the-art forecasters across various horizons.
GraphAF generates chemically valid molecules efficiently and accurately.
problem Generating chemically valid molecular structures while optimizing chemical properties.
method Flow-based autoregressive model combining autoregressive and flow-based approaches.
result GraphAF generates 68% chemically valid molecules without chemical knowledge rules and 100% with rules, achieving state-of-the-art performance.
Behavior cloning training instabilities amplified by SGD noise over long horizons.
problem Training instabilities in behavior cloning with deep neural networks.
method Empirical dissection of minibatch SGD updates and their effects on long-horizon rewards.
result Exponential moving average (EMA) of iterates effectively mitigates gradient variance amplification (GVA).
Deep learning models outperform traditional methods in stock price prediction.
problem Improving stock price prediction accuracy using deep learning.
method Comparative analysis of deep learning models (LSTM, GRU) and traditional methods (ARIMA, ARMA) on historical data.
result Deep learning models, particularly LSTM, outperform traditional methods in predicting stock prices across different time horizons.
Improved time series forecasting with multivariate probabilistic models.
problem Improving accuracy in forecasting time series with statistical dependencies.
method Conditioned Normalizing Flows for autoregressive deep learning models.
result Improved performance over state-of-the-art models on real-world data sets.
Non-autoregressive transformer improves speech recognition speed and accuracy.
problem Reducing inference computation cost in speech recognition.
method Two non-autoregressive transformer structures (A-CMLM and A-FMLM) trained with masked tokens and iterative prediction during inference.
result Non-autoregressive transformer can match autoregressive transformer performance with 7x speedup.
AR CI framework handles complex confounders and sequential actions.
problem Low-dimensional confounders and singleton actions in causal inference.
method Sequencification to transform data into sequences, enabling CI with complex confounders and sequential actions.
result AR model can estimate multiple causal quantities using a single model, simplifying inference and improving outcome prediction.
Efficiently improves non-autoregressive sequence models for better translation performance.
problem Heavy inference latency and inconsistent output sentences in non-autoregressive models.
method Incorporates a structured inference module with an efficient CRF approximation and dynamic transition technique.
result Significantly better translation performance (BLEU score 26.80) compared to previous non-autoregressive models.
Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling th…
New deep learning method solves TSP faster and more efficiently.
problem Approximately solving the Travelling Salesman Problem on 2D Euclidean graphs.
method Uses Graph Convolutional Networks for efficient TSP graph representations and non-autoregressive beam search.
result Significantly reduces optimality gap for large problem instances.
Paper combines latent state space with CRF for improved autoregressive text generation.
problem Autoregressive models expose hidden state trajectory to biases.
method Combines latent state space model with CRF observation model.
result Improved performance on unconditional sentence generation compared to RNN and GAN baselines.
APLC-XLNet improves XMTC by clustering labels and reducing computational time.
problem Efficiently tagging texts with many labels from a large set.
method Fine-tunes XLNet with APLC to approximate cross entropy loss.
result Achieved state-of-the-art results on XMTC benchmarks.
Sparse deep learning improves prediction uncertainty for time series data.
problem Uncertainty quantification for dependent data like time series.
method Sparse recurrent neural networks (RNNs) for time series data.
result Sparse deep learning can consistently estimate and predict time series data with correct uncertainty quantification.
This work proposes an efficient autoregressive model for text generation.
problem The challenge of generating high-quality text with autoregressive models.
method Introduces a cascaded decoding approach using Markov transformers to achieve sub-linear parallel time generation.
result Shows competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
Deep learning predicts currency volatility accurately.
problem Predicting future volatility in Forex trading.
method Constructed a deep-learning network using multiscale LSTM with multi-currency pairs.
result Multiscale LSTM model outperforms conventional models.
Alternative sampling method for autoregressive models using Langevin dynamics.
problem Efficiently sampling from autoregressive models.
method Initialize sequences with white noise and follow Langevin dynamics on global log-likelihood.
result Parallelizes and generalizes sampling process for autoregressive models.
Bayesian method for multivariate autoregressive models with exogenous inputs.
problem Estimating uncertainties in autoregressive models with exogenous inputs.
method Recursive Bayesian estimation via message passing in a factor graph.
result Produces full posterior distributions for autoregressive coefficients and noise precision.
GAMs combine autoregressive and log-linear components for data-efficient sequence learning.
problem Poor performance of standard autoregressive models under small-data conditions.
method Introduce Global Autoregressive Models (GAMs) combining autoregressive and log-linear components, trained in two steps.
result GAMs show a strong perplexity reduction over standard models in language modelling.
Optimal attack against autoregressive models by manipulating environment states.
problem Manipulating autoregressive forecasts to track a target trajectory.
method Linear Quadratic Regulator (LQR) for linear models, Model Predictive Control (MPC) for nonlinear models.
result Optimal attack formulations for both white-box and black-box settings.
Paper proposes AXE loss for non-autoregressive machine translation, improving performance.
problem Challenges in training non-autoregressive models due to lack of autoregressive factors and cross entropy loss penalties.
method Proposes aligned cross entropy (AXE) loss function using a differentiable dynamic program for better word order alignment.
result AXE-based training improves performance on major WMT benchmarks and sets a new state of the art for non-autoregressive models.
Generative model predicts financial market order flow with high accuracy.
problem Creating realistic order flow models for financial markets.
method Token-level autoregressive generative model using deep state space layers.
result Model generates high-quality order flow data with low perplexity.
Non-autoregressive model speeds up sequence generation tasks.
problem Efficiency in sequence generation tasks.
method Iterative refinement based on latent variable models and denoising autoencoders.
result Significant speedup in decoding with comparable quality.