Autoregressive sequence models achieve state-of-the-art performance in domains like machine translation. However, due to the autoregressive factorization nature, these models suffer from heavy latency during inference. Recently, non-autoregressive sequence models were proposed to reduce the inference time. However, the…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “autoregressive inference”
Non-autoregressive method speeds up protein folding prediction 23 times.
problem Generating protein sequences with higher order interactions.
method Discrete diffusion conditioned on 3D structure using ProteinMPNN.
result 23 times speed up in inference without performance loss.
Efficiently combines autoregressive and set-based models for joint distributions.
problem Joint distributions over multiple predictions from set-based models.
method Causal autoregressive buffer that caches context and captures dependencies.
result Up to 20x faster joint sampling and density evaluation, up to 7x lower memory usage.
Due to the unparallelizable nature of the autoregressive factorization, AutoRegressive Translation (ART) models have to generate tokens sequentially during decoding and thus suffer from high inference latency. Non-AutoRegressive Translation (NART) models were proposed to reduce the inference time, but could only achiev…
Agent uses message passing to optimize robot navigation, balancing exploration and exploitation.
problem Optimizing robot navigation in continuous-valued spaces with uncertainty.
method Autoregressive active inference agent using message passing on a factor graph.
result Agent modulates action based on predictive uncertainty, leading to better model of dynamics.
Autoregressive flow models can perform causal discovery and inference tasks.
problem Causal inference tasks such as causal discovery and interventional predictions.
method Using autoregressive flow models to estimate causal directions and make predictions.
result Autoregressive flows can accurately perform causal inference tasks without restrictive assumptions.
Standard autoregressive seq2seq models are easily trained by max-likelihood, but tend to show poor results under small-data conditions. We introduce a class of seq2seq models, GAMs (Global Autoregressive Models), which combine an autoregressive component with a log-linear component, allowing the use of global \textit{a…
The framework of normalizing flows provides a general strategy for flexible variational inference of posteriors over latent variables. We propose a new type of normalizing flow, inverse autoregressive flow (IAF), that, in contrast to earlier published flows, scales well to high-dimensional latent spaces. The proposed f…
Neural autoregressive models are explicit density estimators that achieve state-of-the-art likelihoods for generative modeling. The D-dimensional data distribution is factorized into an autoregressive product of one-dimensional conditional distributions according to the chain rule. Data completion is a more involved ta…
Estimates causal effects using neural autoregressive density estimators.
problem Estimating causal effects in non-linear systems.
method Neural autoregressive density estimators within Pearl's do-calculus framework.
result Retrieves causal effects from non-linear systems without explicit modeling.
Modified asymmetric hidden Markov models for time series with autoregressive components.
problem Dynamic relationships between variables in time series data.
method Introducing an asymmetric autoregressive component to recent asymmetric hidden Markov models.
result The model can choose the optimal autoregressive order for better likelihood.
ARTree uses deep learning to infer tree topologies efficiently.
problem Efficient phylogenetic inference from tree topologies.
method Deep autoregressive model based on graph neural networks (GNNs).
result ARTree provides a flexible family of distributions over tree topologies.
Causal Autoregressive Flowsstat.ML
Causal autoregressive flows enable accurate causal inference and prediction.
problem Causal discovery and interventional predictions in machine learning.
method Autoregressive normalizing flows with fixed variable orderings.
result Causal models derived from autoregressive flows are identifiable and allow for accurate interventional and counterfactual predictions.
A new multivariate stochastic volatility estimation procedure for financial time series is proposed. A Wishart autoregressive process is considered for the volatility precision covariance matrix, for the estimation of which a two step procedure is adopted. The first step is the conditional inference on the autoregressi…
Improves sampling speed of ARM models without losing accuracy.
problem Slow sampling from ARMs.
method Predictive sampling algorithm using ARM fixed-point iteration and learned forecasting modules.
result Significant improvements in sampling speed and efficiency.
LTMs use latent vectors for efficient autoregressive generation.
problem Efficient autoregressive generation in language models.
method Dual-rate optimization in variational Bayes framework.
result LTMs achieve superior sample and parameter efficiency.
Non-autoregressive translation (NAT) models remove the dependence on previous target tokens and generate all target tokens in parallel, resulting in significant inference speedup but at the cost of inferior translation accuracy compared to autoregressive translation (AT) models. Considering that AT models have higher a…
AR CI framework handles complex confounders and sequential actions.
problem Low-dimensional confounders and singleton actions in causal inference.
method Sequencification to transform data into sequences, enabling CI with complex confounders and sequential actions.
result AR model can estimate multiple causal quantities using a single model, simplifying inference and improving outcome prediction.
Paper explores variable skipping to speed up range density estimation.
problem Efficiently estimating range densities over high-dimensional data.
method Variable skipping technique to accelerate range density estimation.
result 10-100x efficiency improvements in challenging high-quantile error metrics.
Vector autoregressive models characterize a variety of time series in which linear combinations of current and past observations can be used to accurately predict future observations. For instance, each element of an observation vector could correspond to a different node in a network, and the parameters of an autoregr…
This paper proposes an EM approach to reduce inference latency in NAR sequence generation.
problem High inference latency in NAR models due to multi-modality in sequence generation.
method A unified EM framework that jointly optimizes AR and NAR models, with iterative refinement.
result The proposed approach achieves competitive performance with existing NAR models and significantly reduces inference latency.
Improved flow-based models capture dependencies better with multi-scale autoregressive priors.
problem Limited expressiveness of flow-based models for long-range data dependencies.
method Introducing channel-wise dependencies through multi-scale autoregressive priors (mAR) in split coupling flow layers (mAR-SCF).
result Achieves state-of-the-art density estimation results on MNIST, CIFAR-10, and ImageNet.
Estimating hidden processes from non-linear noisy observations is particularly difficult when the parameters of these processes are not known. This paper adopts a machine learning approach to devise variational Bayesian inference for such scenarios. In particular, a random process generated by the autoregressive moving…
New approaches improve uncertainty quantification in autoregressive models for sequence data.
problem Uncertainty quantification in autoregressive models for exchangeable sequences.
method Study of inferential and architectural biases for autoregressive models, focusing on multi-step inference.
result Custom architectures are necessary for multi-step inference to ensure exchangeability.
New method for estimating and testing impulse responses in high-dimensional VAR systems.
problem Statistical inference for impulse responses in sparse, high-dimensional vector autoregressions.
method Local projection equations and de-sparsified estimators combined with a non-regularized contemporaneous impact matrix.
result Valid inference procedures for structural impulse responses in high-dimensional systems.
New MIF architecture improves posterior approximations in Bayesian models.
problem Challenges in variational inference for complex hierarchical models.
method Combines VIP and autoregressive flow with prior information and hierarchical ordering.
result Empirically, MIF delivers tighter posterior approximations and state-of-the-art performance.
Training of the neural autoregressive density estimator (NADE) can be viewed as doing one step of probabilistic inference on missing values in data. We propose a new model that extends this inference scheme to multiple steps, arguing that it is easier to learn to improve a reconstruction in steps rather than to lea…
Self-reflective VAE improves inference and generative modeling without complex components.
problem Limitations of typical VAEs in inference and generative modeling.
method Introduces self-reflective inference, a new hierarchical structure that matches variational posterior to exact posterior.
result Self-reflective inference achieves state-of-the-art performance on binarized MNIST without autoregressive layers.
Optimized variable orderings improve autoregressive model performance.
problem Challenges in variable ordering affect autoregressive model efficiency.
method Learn graphical model structure to inform optimal variable orderings.
result Graph-informed orderings yield higher-fidelity samples.
Agents learn and control complex mechanical systems through shared memories.
problem Controlling multi-joint dynamical systems.
method Coupled autoregressive active inference agents using Bayesian filtering and minimizing expected free energy.
result Demonstrated learning and control of a double mass-spring-damper system.
Improved lattice field theory simulations with local-Autoregressive Conditional Normalizing Flow.
problem Efficiently sampling lattice field theories with computational challenges.
method Integrates locality into autoregressive conditional normalizing flows.
result Autocorrelation times improved by orders of magnitude for theory on a 2D lattice.
We present Sequential Neural Likelihood (SNL), a new method for Bayesian inference in simulator models, where the likelihood is intractable but simulating data from the model is possible. SNL trains an autoregressive flow on simulated data in order to learn a model of the likelihood in the region of high posterior dens…
New model handles missing data effectively in autoregressive models.
problem Handling missing data in autoregressive models.
method Reinterpret existing models through missing data lens, introduce principled framework for incomplete datasets, active information acquisition.
result MO-ARM consistently outperforms imputation baselines across real-world benchmarks.
Flow-based generative models are a family of exact log-likelihood models with tractable sampling and latent-variable inference, hence conceptually attractive for modeling complex distributions. However, flow-based models are limited by density estimation performance issues as compared to state-of-the-art autoregressive…
Thermalizer stabilizes autoregressive models for long-term predictions in chaotic systems.
problem Long-term predictions in chaotic spatiotemporal systems are unreliable due to trajectory divergence.
method Diffusion models are used to implicitly estimate the score of an invariant measure, which stabilizes autoregressive emulators by applying denoising during inference.
result Thermalization extends the time horizon of stable predictions by an order of magnitude in chaotic systems.
Conditional Autoregressive Value-at-Risk and Conditional Autoregressive Expectile have become two popular approaches for direct measurement of market risk. Since their introduction several improvements both in the Bayesian and in the classical framework have been proposed to better account for asymmetry and local non-l…
New method for inferring network topology from partial data.
problem Inferring network topology from limited node data.
method Vector autoregressive model and Gaussian mixture algorithm.
result The proposed method converges to the network combination matrix in probability.
Proposes DCNAR for dynamic causal inference from neural time series.
problem Uncertainty and evolution of causal structure in real-world domains.
method Two-stage neural causal modeling integrating discovery and inference.
result Dynamic causal inferences are more stable and meaningful than alternatives.
We introduce Hyper-Conditioned Neural Autoregressive Flow (HCNAF); a powerful universal distribution approximator designed to model arbitrarily complex conditional probability density functions. HCNAF consists of a neural-net based conditional autoregressive flow (AF) and a hyper-network that can take large conditions …
NeLLoC improves image compression with parallel decoding.
problem Image compression with OOD generalization.
method Local autoregressive model with parallel decoding.
result Significant gains in compression runtime.
Paper proposes energy objective for training normalizing flows without determinants.
problem Challenges in training normalizing flows due to Jacobian determinants.
method Introduces energy objective based on proper scoring rules, determinant-free.
result Energy objective supports novel model families and competitive performance.
A new autoregressive model learns the order of graph generation tasks.
problem Generating graphs in a meaningful order when the canonical order is not obvious.
method Introduces a variant of autoregressive models that dynamically decides the autoregressive order based on data.
result Achieves state-of-the-art results on molecular graph generation benchmarks.
Listen and Fill in the Missing Letters: Non-Autoregressive Transformer for Speech Recognitioneess.AS
Recently very deep transformers have outperformed conventional bi-directional long short-term memory networks by a large margin in speech recognition. However, to put it into production usage, inference computation cost is still a serious concern in real scenarios. In this paper, we study two different non-autoregressi…
State-space models (SSMs) provide a flexible framework for modelling time-series data. Consequently, SSMs are ubiquitously applied in areas such as engineering, econometrics and epidemiology. In this paper we provide a fast approach for approximate Bayesian inference in SSMs using the tools of deep learning and variati…
WaveGrad generates high-fidelity audio using gradient estimation.
problem Generating high-fidelity audio efficiently.
method Conditional model using score matching and diffusion models, iteratively refining a Gaussian white noise signal.
result WaveGrad can generate high-fidelity audio samples using as few as six iterations.
Normalising flows (NFS) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to hand-crafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions. Their fl…
Automatically learns summary features from time series data for likelihood-free inference.
problem Necessity of hand-tailored summary features for time series data in likelihood-free inference.
method Data-driven approach to automatically learn summary features.
result Learning summary features from data can outperform hand-crafted values in likelihood-free inference.
Variational Causal Networks approximate Bayesian inference over causal structures.
problem Quantifying uncertainty in causal structure inference from finite data.
method Parametric variational family over DAGs, using Evidence Lower Bound (ELBO) for tractable learning.
result Approximation of the true posterior over DAGs is demonstrated to be good.