arXiv research

Efficiently combines autoregressive and set-based models for joint distributions.

AdaCat improves density estimation and planning in autoregressive models.

Paper proposes a self-supervised method to denoise autoregressive signals with heavy-tailed noise.

Autoregressive models struggle with hard-to-compute distributions, alternatives like energy-based and latent-variable models solve this.

While normalizing flows have led to significant advances in modeling high-dimensional continuous distributions, their applicability to discrete distributions remains unknown. In this paper, we show that flows can in fact be extended to discrete events---and under a simple change-of-variables formula not requiring log-d…

Neural autoregressive models are explicit density estimators that achieve state-of-the-art likelihoods for generative modeling. The D-dimensional data distribution is factorized into an autoregressive product of one-dimensional conditional distributions according to the chain rule. Data completion is a more involved ta…

Normalizing flows and autoregressive models have been successfully combined to produce state-of-the-art results in density estimation, via Masked Autoregressive Flows (MAF), and to accelerate state-of-the-art WaveNet-based speech synthesis to 20x faster than real-time, via Inverse Autoregressive Flows (IAF). We unify a…

Optimized variable orderings improve autoregressive model performance.

Bayesian method for multivariate autoregressive models with exogenous inputs.

Transformers encode latent distributions in text, improving performance in out-of-distribution cases.

Study reveals issues with neural autoregressive models and proposes mode recovery cost.

AR-CSM models use derivatives of univariate log-conditionals to estimate joint distributions efficiently.

New method makes machine learning approximations unbiased and efficient.

Latent Block-Diffusion Temporal Point Processes (LBDTPP) is a semi-autoregressive framework for generating asynchronous event sequences.

Conditional Autoregressive Value-at-Risk and Conditional Autoregressive Expectile have become two popular approaches for direct measurement of market risk. Since their introduction several improvements both in the Bayesian and in the classical framework have been proposed to better account for asymmetry and local non-l…

A new autoregressive model learns the order of graph generation tasks.

Linear autoregressive models serve as basic representations of discrete time stochastic processes. Different attempts have been made to provide non-linear versions of the basic autoregressive process, including different versions based on kernel methods. Motivated by the powerful framework of Hilbert space embeddings o…

Study finds optimal vocabulary size for neural machine translation.

ARCNPs improve CNPs by autoregressively modeling dependencies.

Paper uses GMM and MAF for probabilistic classification, outperforming simpler models.

We introduce the use of autoregressive normalizing flows for rapid likelihood-free inference of binary black hole system parameters from gravitational-wave data with deep neural networks. A normalizing flow is an invertible mapping on a sample space that can be used to induce a transformation from a simple probability …

Standard autoregressive seq2seq models are easily trained by max-likelihood, but tend to show poor results under small-data conditions. We introduce a class of seq2seq models, GAMs (Global Autoregressive Models), which combine an autoregressive component with a log-linear component, allowing the use of global \textit{a…

Efficient methods for answering complex probabilistic queries in sequential data.

Paper proposes GAS-ALD model for financial risk prediction.

This work maps Boltzmann distributions to ARNNs for better physics-based model approximations.

Normalizing flows are a powerful class of generative models for continuous random variables, showing both strong model flexibility and the potential for non-autoregressive generation. These benefits are also desired when modeling discrete random variables such as text, but directly applying normalizing flows to discret…

EventFlow forecasts event sequences without autoregression, improving accuracy.

This study compares different types of normalizing flows for generating complex distributions.

Flow-based generative models are a family of exact log-likelihood models with tractable sampling and latent-variable inference, hence conceptually attractive for modeling complex distributions. However, flow-based models are limited by density estimation performance issues as compared to state-of-the-art autoregressive…

New method uses quantum annealing and VAN for better statistical mechanics calculations.

Neural density estimators are flexible families of parametric models which have seen widespread use in unsupervised machine learning in recent years. Maximum-likelihood training typically dictates that these models be constrained to specify an explicit density. However, this limitation can be overcome by instead using …

The paper proposes autoregressive models for better offline RL.

We introduce Hyper-Conditioned Neural Autoregressive Flow (HCNAF); a powerful universal distribution approximator designed to model arbitrarily complex conditional probability density functions. HCNAF consists of a neural-net based conditional autoregressive flow (AF) and a hyper-network that can take large conditions …

In this work, we consider the class of multi-state autoregressive processes that can be used to model non-stationary time-series of interest. In order to capture different autoregressive (AR) states underlying an observed time series, it is crucial to select the appropriate number of states. We propose a new model sele…

New method uses G-expectation for financial risk measurement.

We consider high-dimensional distribution estimation through autoregressive networks. By combining the concepts of sparsity, mixtures and parameter sharing we obtain a simple model which is fast to train and which achieves state-of-the-art or better results on several standard benchmark datasets. Specifically, we use a…

LMConv improves autoregressive models for image generation and completion.

Transformers learn a mesa-optimizer to implement in-context learning.

Agent uses message passing to optimize robot navigation, balancing exploration and exploitation.

NeLLoC improves image compression with parallel decoding.

Thermalizer stabilizes autoregressive models for long-term predictions in chaotic systems.

We present a distributionally robust formulation of a stochastic optimization problem for non-i.i.d vector autoregressive data. We use the Wasserstein distance to define robustness in the space of distributions and we show, using duality theory, that the problem is equivalent to a finite convex-concave saddle point pro…

Multivariate binary distributions can be decomposed into products of univariate conditional distributions. Recently popular approaches have modeled these conditionals through neural networks with sophisticated weight-sharing structures. It is shown that state-of-the-art performance on several standard benchmark dataset…

Discriminator guidance improves autoregressive diffusion models for generating molecular graphs.

Media is generally stored digitally and is therefore discrete. Many successful deep distribution models in deep learning learn a density, i.e., the distribution of a continuous random variable. Naïve optimization on discrete data leads to arbitrarily high likelihoods, and instead, it has become standard practice to add…

This review compares various deep generative models.

The fundamental task of general density estimation $p(x)$ has been of keen interest to machine learning. In this work, we attempt to systematically characterize methods for density estimation. Broadly speaking, most of the existing methods can be categorized into either using: \textit{a}) autoregressive models to estim…

Bayesian model predicts interest rates with short-term accuracy and long-term stability.