Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling th…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “autoregression”
This work proposes an efficient autoregressive model for text generation.
problem The challenge of generating high-quality text with autoregressive models.
method Introduces a cascaded decoding approach using Markov transformers to achieve sub-linear parallel time generation.
result Shows competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
Autoregressive sequence models achieve state-of-the-art performance in domains like machine translation. However, due to the autoregressive factorization nature, these models suffer from heavy latency during inference. Recently, non-autoregressive sequence models were proposed to reduce the inference time. However, the…
Online Bayesian system identification in multivariate autoregressive models via message passingeess.SP
Bayesian method for multivariate autoregressive models with exogenous inputs.
problem Estimating uncertainties in autoregressive models with exogenous inputs.
method Recursive Bayesian estimation via message passing in a factor graph.
result Produces full posterior distributions for autoregressive coefficients and noise precision.
Autoregressive state transitions, where predictions are conditioned on past predictions, are the predominant choice for both deterministic and stochastic sequential models. However, autoregressive feedback exposes the evolution of the hidden state trajectory to potential biases from well-known train-test discrepancies.…
Alternative sampling method for autoregressive models using Langevin dynamics.
problem Efficiently sampling from autoregressive models.
method Initialize sequences with white noise and follow Langevin dynamics on global log-likelihood.
result Parallelizes and generalizes sampling process for autoregressive models.
Paper proposes AXE loss for non-autoregressive machine translation, improving performance.
problem Challenges in training non-autoregressive models due to lack of autoregressive factors and cross entropy loss penalties.
method Proposes aligned cross entropy (AXE) loss function using a differentiable dynamic program for better word order alignment.
result AXE-based training improves performance on major WMT benchmarks and sets a new state of the art for non-autoregressive models.
Linear attention in Transformers can be interpreted as dynamic VAR models.
problem Misalignment between Transformers and autoregressive forecasting objectives.
method Interpreting linear attention as VAR, rearranging MLP, attention, and flow.
result SAMoVAR improves performance, interpretability, and efficiency.
Paper proposes a self-supervised method to denoise autoregressive signals with heavy-tailed noise.
problem Denoising autoregressive signals corrupted by heavy-tailed noise.
method Self-supervised learning approach without requiring full noise distribution knowledge.
result Strong denoising performance compared to baseline methods, especially for impulsive noise.
Parallelizes autoregressive generation using VSSM.
problem Autoregressive models' inability to parallelize generation.
method Variational SSM (VSSM) with parallelizable sampling and decoding.
result Parallel generation possible with VSSM.
We develop methods to estimate lag and parameters for multiple stable autoregressive processes.
problem Estimating lag and parameters for multiple stable autoregressive processes with unknown lag.
method Use convex programming to simultaneously select lag and estimate parameters across multiple processes.
result The estimated process is stable, and forecasting errors can outperform known rates.
Efficiently combines autoregressive and set-based models for joint distributions.
problem Joint distributions over multiple predictions from set-based models.
method Causal autoregressive buffer that caches context and captures dependencies.
result Up to 20x faster joint sampling and density evaluation, up to 7x lower memory usage.
New method for identifying autoregressive systems on manifolds.
problem Identifying autoregressive systems on Stiefel and Grassmann manifolds.
method Defining parameters as orthogonal group elements, averaging over observations, conjugate gradient descent on manifolds.
result System parameters can be estimated efficiently using the proposed algorithm.
A new clustering method for vector time series using autoregressive dynamics.
problem Clustering of vector time series based on their dynamics is challenging.
method System identification approach using mixture autoregressive models.
result Developed a computationally manageable algorithm k-LMVAR for clustering vector time series.
Standard autoregressive seq2seq models are easily trained by max-likelihood, but tend to show poor results under small-data conditions. We introduce a class of seq2seq models, GAMs (Global Autoregressive Models), which combine an autoregressive component with a log-linear component, allowing the use of global \textit{a…
The recently proposed mask-predict decoding algorithm has narrowed the performance gap between semi-autoregressive machine translation models and the traditional left-to-right approach. We introduce a new training method for conditional masked language models, SMART, which mimics the semi-autoregressive behavior of mas…
Linear autoregressive models serve as basic representations of discrete time stochastic processes. Different attempts have been made to provide non-linear versions of the basic autoregressive process, including different versions based on kernel methods. Motivated by the powerful framework of Hilbert space embeddings o…
Latent Block-Diffusion Temporal Point Processes (LBDTPP) is a semi-autoregressive framework for generating asynchronous event sequences.
problem Generating asynchronous event sequences
method Latent Block-Diffusion Temporal Point Processes
result Outperforms state-of-the-art TPP baselines in both unconditional and conditional generation tasks
Normalizing flows and autoregressive models have been successfully combined to produce state-of-the-art results in density estimation, via Masked Autoregressive Flows (MAF), and to accelerate state-of-the-art WaveNet-based speech synthesis to 20x faster than real-time, via Inverse Autoregressive Flows (IAF). We unify a…
A new online learning setting for autoregressive processes with sublinear regret.
problem Sequential decision-making with temporal dependence in autoregressive processes.
method Autoregressive Bandits (ARBs) and AutoRegressive Upper Confidence Bound (AR-UCB) algorithm.
result Sublinear regret of order $\widetilde{\mathcal{O}} \left( \frac{(k+1)^{3/2}\sqrt{nT}}{(1-Γ)^2}
ight)$ for optimal policy.
Bayesian method estimates Kronecker graphical models from autoregressive processes.
problem Estimating Kronecker graphical models from autoregressive Gaussian processes.
method Bayesian approach to estimate Kronecker graphical models.
result Effectiveness demonstrated through numerical experiments and real-world data application.
Autoregressive models struggle with hard-to-compute distributions, alternatives like energy-based and latent-variable models solve this.
problem Autoregressive models struggle with distributions whose next-symbol probability is hard to compute.
method Alternatives include energy-based models and latent-variable autoregressive models.
result Alternatives to autoregressive models can escape limitations of hard-to-compute distributions.
We describe an optimal adversarial attack formulation against autoregressive time series forecast using Linear Quadratic Regulator (LQR). In this threat model, the environment evolves according to a dynamical system; an autoregressive model observes the current environment state and predicts its future values; an attac…
Missing value imputation is a fundamental problem in spatiotemporal modeling, from motion tracking to the dynamics of physical systems. Deep autoregressive models suffer from error propagation which becomes catastrophic for imputing long-range sequences. In this paper, we take a non-autoregressive approach and propose …
EventFlow forecasts event sequences without autoregression, improving accuracy.
problem Forecasting errors in autoregressive models for event sequences.
method EventFlow uses flow matching to learn joint distributions over event times directly.
result EventFlow reduces forecast error by 20%-53% compared to baselines.
We propose a new class of models specifically tailored for spatio-temporal data analysis. To this end, we generalize the spatial autoregressive model with autoregressive and heteroskedastic disturbances, i.e. SARAR(1,1), by exploiting the recent advancements in Score Driven (SD) models typically used in time series eco…
Non-autoregressive method speeds up protein folding prediction 23 times.
problem Generating protein sequences with higher order interactions.
method Discrete diffusion conditioned on 3D structure using ProteinMPNN.
result 23 times speed up in inference without performance loss.
Study improves dividend discount model using VAR process.
problem Improving dividend discount models for better predictions.
method Introduced a Gordon growth model based on Vector Autoregressive Process (VAR).
result Two Propositions related to the new model.
The paper proposes autoregressive models for better offline RL.
problem Offline RL policy evaluation and optimization challenges.
method Autoregressive dynamics models for sequential state and reward prediction.
result Autoregressive models outperform standard methods in log-likelihood and RL tasks.
Proposes a non-autoregressive Transformer for time series forecasting.
problem Autoregressive errors and spatial-temporal dependencies in time series forecasting.
method Introduces a Non-Autoregressive Transformer with a learned temporal influence map.
result Demonstrates state-of-the-art performance on time series forecasting datasets.
While normalizing flows have led to significant advances in modeling high-dimensional continuous distributions, their applicability to discrete distributions remains unknown. In this paper, we show that flows can in fact be extended to discrete events---and under a simple change-of-variables formula not requiring log-d…
New method forecasts multilinear data using tensor autoregression.
problem Forecasting 2D data in big data.
method L-Transform Tensor autoregressive (L-TAR) method.
result Statistical independence achieved through invertible discrete linear transforms.
Efficient autoregressive entity linking with correction for faster, more accurate results.
problem High computational cost and non-parallelizable decoding in autoregressive entity linking.
method Parallelizes autoregressive linking across all mentions, uses a shallow decoder, and adds a discriminative correction term.
result 70 times faster and more accurate than previous methods, outperforming state-of-the-art approaches.
Optimizes Gaussian process hyperparameters using Bayesian autoregression.
problem Optimizing hyperparameters for Matérn kernel temporal Gaussian processes.
method Recursive Bayesian estimation for autoregressive parameters.
result Outperforms traditional optimization methods in runtime and accuracy.
We consider stationary autoregressive processes with coefficients restricted to an ellipsoid, which includes autoregressive processes with absolutely summable coefficients. We provide consistency results under different norms for the estimation of such processes using constrained and penalized estimators. As an applica…
Improved lattice field theory simulations with local-Autoregressive Conditional Normalizing Flow.
problem Efficiently sampling lattice field theories with computational challenges.
method Integrates locality into autoregressive conditional normalizing flows.
result Autocorrelation times improved by orders of magnitude for theory on a 2D lattice.
Autoregressive generative models of images tend to be biased towards capturing local structure, and as a result they often produce samples which are lacking in terms of large-scale coherence. To address this, we propose two methods to learn discrete representations of images which abstract away local detail. We show th…
Due to the unparallelizable nature of the autoregressive factorization, AutoRegressive Translation (ART) models have to generate tokens sequentially during decoding and thus suffer from high inference latency. Non-AutoRegressive Translation (NART) models were proposed to reduce the inference time, but could only achiev…
A new autoregressive model learns the order of graph generation tasks.
problem Generating graphs in a meaningful order when the canonical order is not obvious.
method Introduces a variant of autoregressive models that dynamically decides the autoregressive order based on data.
result Achieves state-of-the-art results on molecular graph generation benchmarks.
Modified asymmetric hidden Markov models for time series with autoregressive components.
problem Dynamic relationships between variables in time series data.
method Introducing an asymmetric autoregressive component to recent asymmetric hidden Markov models.
result The model can choose the optimal autoregressive order for better likelihood.
New model handles missing data effectively in autoregressive models.
problem Handling missing data in autoregressive models.
method Reinterpret existing models through missing data lens, introduce principled framework for incomplete datasets, active information acquisition.
result MO-ARM consistently outperforms imputation baselines across real-world benchmarks.
Flow-based generative models are a family of exact log-likelihood models with tractable sampling and latent-variable inference, hence conceptually attractive for modeling complex distributions. However, flow-based models are limited by density estimation performance issues as compared to state-of-the-art autoregressive…
The framework of normalizing flows provides a general strategy for flexible variational inference of posteriors over latent variables. We propose a new type of normalizing flow, inverse autoregressive flow (IAF), that, in contrast to earlier published flows, scales well to high-dimensional latent spaces. The proposed f…
Paper proposes energy objective for training normalizing flows without determinants.
problem Challenges in training normalizing flows due to Jacobian determinants.
method Introduces energy objective based on proper scoring rules, determinant-free.
result Energy objective supports novel model families and competitive performance.
WAVE improves time series forecasting by integrating AR and MA components.
problem Time series forecasting challenges.
method WAVE attention mechanism with AR and MA components.
result WAVE attention consistently improves TSF performance.
Neural autoregressive models are explicit density estimators that achieve state-of-the-art likelihoods for generative modeling. The D-dimensional data distribution is factorized into an autoregressive product of one-dimensional conditional distributions according to the chain rule. Data completion is a more involved ta…
Autoregressive networks can achieve promising performance in many sequence modeling tasks with short-range dependence. However, when handling high-dimensional inputs and outputs, the huge amount of parameters in the network lead to expensive computational cost and low learning efficiency. The problem can be alleviated …
Optimized variable orderings improve autoregressive model performance.
problem Challenges in variable ordering affect autoregressive model efficiency.
method Learn graphical model structure to inform optimal variable orderings.
result Graph-informed orderings yield higher-fidelity samples.