NAF combines neural networks with autoregressive models for better density estimation.
problem Improving density estimation and speech synthesis speed.
method Generalizes autoregressive models using neural networks for invertible transformations.
result NAF is a universal approximator for continuous probability distributions and outperforms IAF.
Non-autoregressive model speeds up sequence generation tasks.
problem Efficiency in sequence generation tasks.
method Iterative refinement based on latent variable models and denoising autoencoders.
result Significant speedup in decoding with comparable quality.
A new neural network for efficient density estimation.
problem Efficient density estimation for high-dimensional data.
method Triangular neural network implementation of neural autoregressive flow (NAF).
result Achieves state-of-the-art bits-per-dimension indices on MNIST and CIFAR-10.
IAF improves variational inference by scaling to high-dimensional spaces.
problem Flexible variational inference of posteriors over latent variables.
method Inverse autoregressive flow (IAF) using invertible transformations based on autoregressive neural networks.
result IAF significantly improves upon diagonal Gaussian approximate posteriors.
HCNAF models complex conditional distributions for probabilistic occupancy forecasting.
problem Modeling complex conditional probability density functions for occupancy forecasting.
method Hyper-Conditioned Neural Autoregressive Flow (HCNAF) combining AF and hyper-network.
result HCNAF achieves state-of-the-art performance in self-driving datasets.
Estimates causal effects using neural autoregressive density estimators.
problem Estimating causal effects in non-linear systems.
method Neural autoregressive density estimators within Pearl's do-calculus framework.
result Retrieves causal effects from non-linear systems without explicit modeling.
B-NAF is a more compact flow for density estimation and inference.
problem Efficiently modeling complex density functions with fewer parameters.
method Directly models a bijection using a single feed-forward network with block matrices.
result B-NAF uses orders of magnitude fewer parameters while being competitive.
ARCNPs improve CNPs by autoregressively modeling dependencies.
problem CNPs struggle with modeling dependencies in predictions.
method Autoregressive deployment of factorized Gaussian CNPs.
result ARCNPs significantly outperform non-AR CNPs in various tasks.
ARMA cell simplifies neural autoregressive modeling for time series.
problem Complex RNN cells are not always necessary and can be inferior.
method Introduces ARMA cell, a simpler, modular approach for neural time series modeling.
result The ARMA cell is competitive with popular alternatives in performance.
Improves autoregressive models for better density estimation.
problem Limited flexibility of autoregressive models.
method Stacked autoregressive models with random number modeling.
result Achieves state-of-the-art performance in density estimation.
SNL trains autoregressive flows on simulated data to learn likelihood for Bayesian inference.
problem Intractable likelihood in simulator models.
method Trains autoregressive flow on simulated data to model likelihood.
result SNL is more robust, accurate, and requires less tuning than related methods.
A new model estimates complex densities without explicit normalizing constants.
problem Accurately estimating the normalizing constant of high-dimensional energy functions.
method Autoregressive Energy Machine (AEM) learns an unnormalized density and an importance-sampling estimate of the normalizing constant.
result Achieves state-of-the-art performance on density-estimation tasks.
A new training method improves autoregressive data completion efficiency.
problem Efficiently completing missing data in autoregressive models.
method Proposed an alternative training procedure (OA++) that reduces overfitting and leverages prior knowledge.
result OA++ achieves better performance with fewer computations and less overfitting.
New method makes machine learning approximations unbiased and efficient.
problem Efficient sampling of complex probability distributions.
method Uses autoregressive neural networks with cluster updates and physical symmetries.
result Shows unbiased and low-variance approximations for phase transitions.
Study finds optimal vocabulary size for neural machine translation.
problem Imbalanced class distribution in language data affects NMT performance.
method Casts NMT as a classification task, analyzes vocabulary sizes, and tests multiple languages.
result Certain vocabulary sizes outperform others, explaining NMT performance.
LogitBoost improves autoregressive models for binary data.
problem Efficiently modeling multivariate binary distributions.
method Training separate probability estimators for each dimension using LogitBoost.
result Separate probability estimators can achieve state-of-the-art performance.
A new neural model improves collaborative filtering performance.
problem Improving recommendation systems for better user satisfaction.
method Integrates neural autoregressive distribution estimation with collaborative filtering, sharing parameters, and considering ordinal preferences.
result CF-NADE outperforms previous methods on various datasets.
New method estimates spin system mutual information using neural networks.
problem Estimating mutual information in spin systems.
method Monte Carlo sampling enhanced by autoregressive neural networks.
result Area law satisfied for temperatures away from critical temperature.
This work proposes an efficient autoregressive model for text generation.
problem The challenge of generating high-quality text with autoregressive models.
method Introduces a cascaded decoding approach using Markov transformers to achieve sub-linear parallel time generation.
result Shows competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
Deep neural network solves complex groundwater contaminant source identification.
problem Identifying groundwater contaminant sources in highly heterogeneous media.
method Deep autoregressive neural network surrogate for forward model, ILUES for inversion.
result Deep autoregressive neural network provides accurate approximation for high-dimensional model.
Method uses autoregressive models to interpret neural network representations.
problem Understanding and quantifying information preserved in neural network layers.
method Trains autoregressive models to invert model representations and estimate mutual information.
result Mutual information between inputs and network layers decreases over training.
NeLLoC improves image compression with parallel decoding.
problem Image compression with OOD generalization.
method Local autoregressive model with parallel decoding.
result Significant gains in compression runtime.
Paper proposes AR model for graph sequences.
problem Predicting sequences of graphs with variable topology.
method Formalizes AR model for graphs using GNN to learn and predict next graph.
result Significantly better performance on synthetic graph generation problems.
New method uses neural networks to solve statistical mechanics problems.
problem Statistical mechanics of systems with finite size.
method Variational autoregressive neural networks with reinforcement learning.
result Directly computes free energy, entropy, magnetizations, and correlations.
Paper proposes a neural network method for fast, interpretable AR model estimation.
problem Computational inefficiency and convergence issues in conventional AR model estimation.
method Embeds autoregressive structure into a feedforward neural network for coefficient estimation via backpropagation.
result Neural network method consistently recovers AR model coefficients, converging in all cases and providing reliable estimates.
UMNNs improve density estimation and variational inference without constraints.
problem Creating expressive invertible transformations without constraints.
method Proposed UMNN architecture enforcing monotonicity with a free-form neural network.
result UMNNs enhance autoregressive flows for density estimation and variational inference.
CTRNNs improve blood glucose forecasting in ICU, outperforming traditional models.
problem Forecasting blood glucose in ICU with irregular measurements.
method Continuous time autoregressive recurrent neural networks (CTRNNs) using neural ODE or neural flow layers.
result CTRNNs generally outperform traditional autoregressive models in probabilistic forecasting of blood glucose.
Study reveals issues with neural autoregressive models and proposes mode recovery cost.
problem Unreasonable affinity of neural autoregressive models to short and long sequences.
method Investigates modes of ground-truth, empirical, and decoding-induced distributions via mode recovery cost.
result Mode recovery cost varies depending on ground-truth distribution and impacts decoding-induced distribution.
The paper develops a learning theory for neural network-based CHARME models.
problem Developing a learning theory for CHARME models using neural networks.
method Proves the stationarity and ergodicity of CHARME models under weak conditions, then applies neural networks to derive strong consistency and asymptotic normality of estimators.
result Strong consistency and asymptotic normality of NN-based estimators of CHARME model weights and biases under weak conditions.
BiGG model efficiently generates sparse graphs with reduced complexity.
problem Challenges in scalable deep learning for sparse graphs.
method BiGG model, an autoregressive model that leverages graph sparsity.
result Graph generation time complexity reduced from O(n2) to O((n+m)logn). New model predicts univariate and multivariate time series with improved accuracy.
problem Complex patterns in univariate and multivariate time series forecasting.
method Uses autoregressive convolutional recurrent neural network with feature extraction and recurrent encoder.
result Outperforms existing architectures in multivariate time series datasets.
Proposes a neural density estimator for anomaly detection using labeled data.
problem Improving anomaly detection performance with limited labeled data.
method Uses deep autoregressive neural density estimators trained with anomaly labels to maximize normal likelihood and minimize anomalous likelihood.
result Significantly improves anomaly detection performance with few labeled instances compared to existing methods.
Thermalizer stabilizes autoregressive models for long-term predictions in chaotic systems.
problem Long-term predictions in chaotic spatiotemporal systems are unreliable due to trajectory divergence.
method Diffusion models are used to implicitly estimate the score of an invariant measure, which stabilizes autoregressive emulators by applying denoising during inference.
result Thermalization extends the time horizon of stable predictions by an order of magnitude in chaotic systems.
Paper uses surprisal to dynamically allocate computation between fast and slow models.
problem Dynamic allocation of computation in neural networks.
method Surprisal-based dynamic model selection.
result Model can match baseline performance with 15% fewer FLOPs.
This work maps Boltzmann distributions to ARNNs for better physics-based model approximations.
problem Approximating Boltzmann distributions of binary systems.
method Exact mapping of Boltzmann distribution to autoregressive neural network architecture.
result New ARNN architectures derived from physical models show superior performance.
Efficiently combines autoregressive and set-based models for joint distributions.
problem Joint distributions over multiple predictions from set-based models.
method Causal autoregressive buffer that caches context and captures dependencies.
result Up to 20x faster joint sampling and density evaluation, up to 7x lower memory usage.
Neural networks improve gravitational-wave parameter estimation.
problem Estimating parameters of binary black hole systems from gravitational-wave data.
method Autoregressive normalizing flows for likelihood-free inference.
result Performance comparable to current best deep-learning approaches, with fast sampling.
EventFlow forecasts event sequences without autoregression, improving accuracy.
problem Forecasting errors in autoregressive models for event sequences.
method EventFlow uses flow matching to learn joint distributions over event times directly.
result EventFlow reduces forecast error by 20%-53% compared to baselines.
Generative neural samplers estimate quantum spin system properties.
problem Estimating observables for quantum spin systems.
method Autoregressive models using Suzuki-Trotter transformation.
result Results for energy, specific heat, and susceptibility are in good agreement with Monte Carlo methods.
Improved lattice field theory simulations with local-Autoregressive Conditional Normalizing Flow.
problem Efficiently sampling lattice field theories with computational challenges.
method Integrates locality into autoregressive conditional normalizing flows.
result Autocorrelation times improved by orders of magnitude for φ4 theory on a 2D lattice. Improved simulation of phase transitions using hierarchical autoregressive networks.
problem Simulating phase transitions in complex systems.
method Hierarchical Autoregressive Neural (HAN) network sampling algorithm.
result Significant improvement in statistical uncertainty compared to the Wolff cluster algorithm.
Neural spline flows enhance flow models with rational-quadratic splines.
problem Improving flexibility and density estimation in flow models.
method Proposes a new differentiable module based on monotonic rational-quadratic splines.
result Demonstrates improved performance in density estimation, variational inference, and generative modeling of images.
PARNN improves ARNN with ARIMA feedback for accurate long-range forecasting.
problem Accurate long-range forecasting of complex time series data.
method Improves ARNN using ARIMA feedback, providing uncertainty quantification.
result PARNN outperforms state-of-the-art forecasters across various horizons.
The paper addresses statistical guarantees for autoregressive models in non-Gaussian settings.
problem Statistical guarantees for autoregressive models in non-Gaussian settings.
method Sparsity-regularized maximum likelihood estimator, martingale concentration inequalities, and modern empirical process techniques.
result Sample complexity bounds derived for autoregressive generalized linear models.
LMConv improves autoregressive models for image generation and completion.
problem Limited generation order in autoregressive models restricts their applicability.
method Introduces LMConv, a modified 2D convolution that allows arbitrary masks to be applied to weights.
result LMConv achieves improved performance on image density estimation and coherent completions.
mLSTM improves sequence modeling with better autoregressive density estimation.
problem Improving autoregressive density estimation in sequence modeling.
method Introduces mLSTM, a recurrent neural network combining LSTM and multiplicative recurrent networks.
result mLSTM outperforms standard LSTM and its variants in character-level language modeling tasks.
Improved language generation with faster sampling speed.
problem Speed and coherence issues in autoregressive language models.
method Introduces Neural Flow Diffusion Models (NFDM) for discrete state spaces.
result Substantially reduces likelihood gap with autoregressive models.
Transformers become faster by linearizing self-attention.
problem Quadratic complexity of transformers makes them slow for long sequences.
method Expressed self-attention as a linear dot-product and used matrix product associativity to reduce complexity.
result Linear transformers are up to 4000x faster on long sequences.