Improved lattice field theory simulations with local-Autoregressive Conditional Normalizing Flow.
problem Efficiently sampling lattice field theories with computational challenges.
method Integrates locality into autoregressive conditional normalizing flows.
result Autocorrelation times improved by orders of magnitude for φ4 theory on a 2D lattice. NeLLoC improves image compression with parallel decoding.
problem Image compression with OOD generalization.
method Local autoregressive model with parallel decoding.
result Significant gains in compression runtime.
Autoregressive state transitions, where predictions are conditioned on past predictions, are the predominant choice for both deterministic and stochastic sequential models. However, autoregressive feedback exposes the evolution of the hidden state trajectory to potential biases from well-known train-test discrepancies.…
Autoregressive generative models of images tend to be biased towards capturing local structure, and as a result they often produce samples which are lacking in terms of large-scale coherence. To address this, we propose two methods to learn discrete representations of images which abstract away local detail. We show th…
Paper proposes a forecasting model combining autoregressive models with spectral attention.
problem Time series forecasting across various domains.
method Combines deep autoregressive models with Spectral Attention (SA) module.
result SAAM consistently demonstrates improved forecasting accuracy compared to state-of-the-art approaches.
WAVE improves time series forecasting by integrating AR and MA components.
problem Time series forecasting challenges.
method WAVE attention mechanism with AR and MA components.
result WAVE attention consistently improves TSF performance.
Study local sensitivity of HDD and CDD temperature derivatives prices.
problem Understanding how temperature derivatives prices change with small temperature changes.
method Analyzes sensitivity of HDD and CDD futures and options prices to temperature perturbations using a CAR process.
result Identifies the order of the CAR process and its impact on temperature derivatives prices.
Proposes a non-autoregressive Transformer for time series forecasting.
problem Autoregressive errors and spatial-temporal dependencies in time series forecasting.
method Introduces a Non-Autoregressive Transformer with a learned temporal influence map.
result Demonstrates state-of-the-art performance on time series forecasting datasets.
Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence. Autoregressive models, such as WaveNet, model local structure at the expense of global latent structure and slow iterative sampling, while Generative A…
DiAMoNDBack models protein backmapping from coarse-grained Cα traces.
problem Restoring all-atom details from coarse-grained protein representations.
method Autoregressive denoising diffusion model for residue-by-residue backmapping.
result Achieves state-of-the-art reconstruction performance in diverse applications.
Paper proposes LATC for multivariate time series prediction and missing data imputation.
problem Large-scale, incomplete, and corrupted multivariate time series data.
method Transforms multivariate time series into a tensor structure, models global and local trends, and uses autoregressive norm.
result Integration of global and local trends improves missing data imputation and rolling prediction.
Conditional Autoregressive Value-at-Risk and Conditional Autoregressive Expectile have become two popular approaches for direct measurement of market risk. Since their introduction several improvements both in the Bayesian and in the classical framework have been proposed to better account for asymmetry and local non-l…
Latent Block-Diffusion Temporal Point Processes (LBDTPP) is a semi-autoregressive framework for generating asynchronous event sequences.
problem Generating asynchronous event sequences
method Latent Block-Diffusion Temporal Point Processes
result Outperforms state-of-the-art TPP baselines in both unconditional and conditional generation tasks
Autoregressive feedback is considered a necessity for successful unconditional text generation using stochastic sequence models. However, such feedback is known to introduce systematic biases into the training process and it obscures a principle of generation: committing to global information and forgetting local nuanc…
Study compares local and global models for hierarchical forecasting accuracy.
problem Challenges in hierarchical time series forecasting, especially in accuracy and information utilisation.
method Developed and evaluated local and global forecasting models (GFMs) to exploit cross-series and cross-hierarchies information.
result Global Forecasting Models (GFMs) outperform local models in hierarchical forecasting accuracy and computational efficiency.
Optimal sample complexity for autoregressive chain-of-thought learning proven.
problem Determining the minimum number of samples needed for accurate autoregressive chain-of-thought learning.
method Proved upper bound on sample complexity using Daniely-Shalev-Shwartz dimension and roll-out stable parity dimension.
result The sample complexity is bounded by the local next-token class rate, with no dependence on rollout length.
LMConv improves autoregressive models for image generation and completion.
problem Limited generation order in autoregressive models restricts their applicability.
method Introduces LMConv, a modified 2D convolution that allows arbitrary masks to be applied to weights.
result LMConv achieves improved performance on image density estimation and coherent completions.
Calculates local Granger causality for Gaussian and nonlinear systems.
problem Understanding causal influence in complex systems.
method Vector autoregression and information-theoretic approach.
result Local Granger causality offers a robust and fast method for time-directed information transfer.
A new method for filling in missing traffic data improves accuracy over existing techniques.
problem Incomplete spatiotemporal traffic data.
method Low-rank autoregressive tensor completion (LATC) framework.
result LATC framework better captures spatiotemporal consistency and local consistency.
LTMs use latent vectors for efficient autoregressive generation.
problem Efficient autoregressive generation in language models.
method Dual-rate optimization in variational Bayes framework.
result LTMs achieve superior sample and parameter efficiency.
Realistic music generation is a challenging task. When building generative models of music that are learnt from data, typically high-level representations such as scores or MIDI are used that abstract away the idiosyncrasies of a particular performance. But these nuances are very important for our perception of musical…
Identification of a groundwater contaminant source simultaneously with the hydraulic conductivity in highly-heterogeneous media often results in a high-dimensional inverse problem. In this study, a deep autoregressive neural network-based surrogate method is developed for the forward model to allow us to solve efficien…
Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling th…
This work proposes an efficient autoregressive model for text generation.
problem The challenge of generating high-quality text with autoregressive models.
method Introduces a cascaded decoding approach using Markov transformers to achieve sub-linear parallel time generation.
result Shows competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
Autoregressive sequence models achieve state-of-the-art performance in domains like machine translation. However, due to the autoregressive factorization nature, these models suffer from heavy latency during inference. Recently, non-autoregressive sequence models were proposed to reduce the inference time. However, the…
Alternative sampling method for autoregressive models using Langevin dynamics.
problem Efficiently sampling from autoregressive models.
method Initialize sequences with white noise and follow Langevin dynamics on global log-likelihood.
result Parallelizes and generalizes sampling process for autoregressive models.
Bayesian method for multivariate autoregressive models with exogenous inputs.
problem Estimating uncertainties in autoregressive models with exogenous inputs.
method Recursive Bayesian estimation via message passing in a factor graph.
result Produces full posterior distributions for autoregressive coefficients and noise precision.
Study reveals issues with neural autoregressive models and proposes mode recovery cost.
problem Unreasonable affinity of neural autoregressive models to short and long sequences.
method Investigates modes of ground-truth, empirical, and decoding-induced distributions via mode recovery cost.
result Mode recovery cost varies depending on ground-truth distribution and impacts decoding-induced distribution.
Flow-based generative models, conceptually attractive due to tractability of both the exact log-likelihood computation and latent-variable inference, and efficiency of both training and sampling, has led to a number of impressive empirical successes and spawned many advanced variants and theoretical investigations. Des…
Sparse Tucker decomposition with graph regularization improves time series forecasting accuracy.
problem High-dimensional time series forecasting with over-parameterization issue.
method Sparse Tucker decomposition and graph regularization for tensor-based model.
result Non-asymptotic error bound and superior performance in numerical experiments.
Paper proposes AXE loss for non-autoregressive machine translation, improving performance.
problem Challenges in training non-autoregressive models due to lack of autoregressive factors and cross entropy loss penalties.
method Proposes aligned cross entropy (AXE) loss function using a differentiable dynamic program for better word order alignment.
result AXE-based training improves performance on major WMT benchmarks and sets a new state of the art for non-autoregressive models.
Proposes a method to generate text that adheres to logical constraints.
problem Generating text that respects logical constraints is hard for autoregressive models.
method Bayesian conditioning to draw samples subject to a constraint, considering the entire sequence and inducing a local, factorized distribution.
result Our approach generates samples that closely approximate the target distribution and are guaranteed to satisfy the constraints.
SXL embeds spatial autocorrelation into neural networks for better geographic data learning.
problem Difficulties in learning spatial effects for neural networks in geographic data.
method SXL uses auxiliary tasks and autoregressive embeddings to learn spatial autocorrelation.
result SXL improves neural network training in unsupervised and supervised learning tasks.
Efficient method classifies locally stationary time series based on second-order characteristics.
problem Classifying locally stationary time series for various applications.
method Autoregressive approximation, ensemble aggregation, distance-based threshold.
result Zero misclassification error rate asymptotically for mildly differing second-order characteristics.
Global information is essential for dense prediction problems, whose goal is to compute a discrete or continuous label for each pixel in the images. Traditional convolutional layers in neural networks, initially designed for image classification, are restrictive in these problems since the filter size limits their rece…
Parallelizes autoregressive generation using VSSM.
problem Autoregressive models' inability to parallelize generation.
method Variational SSM (VSSM) with parallelizable sampling and decoding.
result Parallel generation possible with VSSM.
Linear attention in Transformers can be interpreted as dynamic VAR models.
problem Misalignment between Transformers and autoregressive forecasting objectives.
method Interpreting linear attention as VAR, rearranging MLP, attention, and flow.
result SAMoVAR improves performance, interpretability, and efficiency.
Global methods outperform local in forecasting groups of time series, even in heterogeneous datasets.
problem Forecasting groups of time series, especially in heterogeneous datasets.
method Local methods consider each series separately, global methods fit a single model to all series.
result Global methods can outperform local methods in forecasting groups of time series, even in heterogeneous datasets.
Efficiently combines autoregressive and set-based models for joint distributions.
problem Joint distributions over multiple predictions from set-based models.
method Causal autoregressive buffer that caches context and captures dependencies.
result Up to 20x faster joint sampling and density evaluation, up to 7x lower memory usage.
Flow-based deep generative models learn data distributions by transforming a simple base distribution into a complex distribution via a set of invertible transformations. Due to the invertibility, such models can score unseen data samples by computing their exact likelihood under the learned distribution. This makes fl…
Standard autoregressive seq2seq models are easily trained by max-likelihood, but tend to show poor results under small-data conditions. We introduce a class of seq2seq models, GAMs (Global Autoregressive Models), which combine an autoregressive component with a log-linear component, allowing the use of global \textit{a…
Paper proposes a self-supervised method to denoise autoregressive signals with heavy-tailed noise.
problem Denoising autoregressive signals corrupted by heavy-tailed noise.
method Self-supervised learning approach without requiring full noise distribution knowledge.
result Strong denoising performance compared to baseline methods, especially for impulsive noise.
The recently proposed mask-predict decoding algorithm has narrowed the performance gap between semi-autoregressive machine translation models and the traditional left-to-right approach. We introduce a new training method for conditional masked language models, SMART, which mimics the semi-autoregressive behavior of mas…
Autoregressive models struggle with hard-to-compute distributions, alternatives like energy-based and latent-variable models solve this.
problem Autoregressive models struggle with distributions whose next-symbol probability is hard to compute.
method Alternatives include energy-based models and latent-variable autoregressive models.
result Alternatives to autoregressive models can escape limitations of hard-to-compute distributions.
New method for estimating and testing impulse responses in high-dimensional VAR systems.
problem Statistical inference for impulse responses in sparse, high-dimensional vector autoregressions.
method Local projection equations and de-sparsified estimators combined with a non-regularized contemporaneous impact matrix.
result Valid inference procedures for structural impulse responses in high-dimensional systems.
Study improves dividend discount model using VAR process.
problem Improving dividend discount models for better predictions.
method Introduced a Gordon growth model based on Vector Autoregressive Process (VAR).
result Two Propositions related to the new model.
We describe an optimal adversarial attack formulation against autoregressive time series forecast using Linear Quadratic Regulator (LQR). In this threat model, the environment evolves according to a dynamical system; an autoregressive model observes the current environment state and predicts its future values; an attac…
Bayesian method estimates Kronecker graphical models from autoregressive processes.
problem Estimating Kronecker graphical models from autoregressive Gaussian processes.
method Bayesian approach to estimate Kronecker graphical models.
result Effectiveness demonstrated through numerical experiments and real-world data application.