We consider stationary autoregressive processes with coefficients restricted to an ellipsoid, which includes autoregressive processes with absolutely summable coefficients. We provide consistency results under different norms for the estimation of such processes using constrained and penalized estimators. As an applica…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “autoregressive norm”
Paper proposes LATC for multivariate time series prediction and missing data imputation.
problem Large-scale, incomplete, and corrupted multivariate time series data.
method Transforms multivariate time series into a tensor structure, models global and local trends, and uses autoregressive norm.
result Integration of global and local trends improves missing data imputation and rolling prediction.
The optimal predictor for a linear dynamical system (with hidden state and Gaussian noise) takes the form of an autoregressive linear filter, namely the Kalman filter. However, a fundamental problem in reinforcement learning and control theory is to make optimal predictions in an unknown dynamical system. To this end, …
We study sparse principal component analysis for high dimensional vector autoregressive time series under a doubly asymptotic framework, which allows the dimension to scale with the series length . We treat the transition matrix of time series as a nuisance parameter and directly apply sparse principal component…
This study compares different types of normalizing flows for generating complex distributions.
problem Comparing different types of normalizing flows for generating complex distributions.
method Real-valued non-Volume preserving (RealNVP), masked autoregressive flow (MAF), coupling rational quadratic spline (C-RQS), and autoregressive rational quadratic spline (A-RQS) were compared using statistical tests.
result A-RQS algorithm outperforms others in terms of accuracy and training speed.
Kernel adaptive filters, a class of adaptive nonlinear time-series models, are known by their ability to learn expressive autoregressive patterns from sequential data. However, for trivial monotonic signals, they struggle to perform accurate predictions and at the same time keep computational complexity within desired …
While considerable advances have been made in estimating high-dimensional structured models from independent data using Lasso-type models, limited progress has been made for settings when the samples are dependent. We consider estimating structured VAR (vector auto-regressive models), where the structure can be capture…
Causal normalizing flows recover causal models from observational data.
problem Recovering causal models from observational data.
method Use autoregressive normalizing flows and analyze design choices.
result Causal normalizing flows can capture causal data-generating processes.
Paper proposes a new sparsity scheme for high-dimensional VAR models.
problem Estimation of high-dimensional VAR models with sparsity assumptions.
method Regularized estimation procedures for sparse VAR models.
result Threholding extends consistency properties of regularized estimators.
Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling th…
This work proposes an efficient autoregressive model for text generation.
problem The challenge of generating high-quality text with autoregressive models.
method Introduces a cascaded decoding approach using Markov transformers to achieve sub-linear parallel time generation.
result Shows competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
Autoregressive sequence models achieve state-of-the-art performance in domains like machine translation. However, due to the autoregressive factorization nature, these models suffer from heavy latency during inference. Recently, non-autoregressive sequence models were proposed to reduce the inference time. However, the…
Online Bayesian system identification in multivariate autoregressive models via message passingeess.SP
Bayesian method for multivariate autoregressive models with exogenous inputs.
problem Estimating uncertainties in autoregressive models with exogenous inputs.
method Recursive Bayesian estimation via message passing in a factor graph.
result Produces full posterior distributions for autoregressive coefficients and noise precision.
Autoregressive state transitions, where predictions are conditioned on past predictions, are the predominant choice for both deterministic and stochastic sequential models. However, autoregressive feedback exposes the evolution of the hidden state trajectory to potential biases from well-known train-test discrepancies.…
Alternative sampling method for autoregressive models using Langevin dynamics.
problem Efficiently sampling from autoregressive models.
method Initialize sequences with white noise and follow Langevin dynamics on global log-likelihood.
result Parallelizes and generalizes sampling process for autoregressive models.
Paper proposes AXE loss for non-autoregressive machine translation, improving performance.
problem Challenges in training non-autoregressive models due to lack of autoregressive factors and cross entropy loss penalties.
method Proposes aligned cross entropy (AXE) loss function using a differentiable dynamic program for better word order alignment.
result AXE-based training improves performance on major WMT benchmarks and sets a new state of the art for non-autoregressive models.
In this work, we study the problem of aggregating a finite number of predictors for nonstationary sub-linear processes. We provide oracle inequalities relying essentially on three ingredients: (1) a uniform bound of the norm of the time varying sub-linear coefficients, (2) a Lipschitz assumption on the predict…
Linear attention in Transformers can be interpreted as dynamic VAR models.
problem Misalignment between Transformers and autoregressive forecasting objectives.
method Interpreting linear attention as VAR, rearranging MLP, attention, and flow.
result SAMoVAR improves performance, interpretability, and efficiency.
Paper proposes a self-supervised method to denoise autoregressive signals with heavy-tailed noise.
problem Denoising autoregressive signals corrupted by heavy-tailed noise.
method Self-supervised learning approach without requiring full noise distribution knowledge.
result Strong denoising performance compared to baseline methods, especially for impulsive noise.
Parallelizes autoregressive generation using VSSM.
problem Autoregressive models' inability to parallelize generation.
method Variational SSM (VSSM) with parallelizable sampling and decoding.
result Parallel generation possible with VSSM.
We develop methods to estimate lag and parameters for multiple stable autoregressive processes.
problem Estimating lag and parameters for multiple stable autoregressive processes with unknown lag.
method Use convex programming to simultaneously select lag and estimate parameters across multiple processes.
result The estimated process is stable, and forecasting errors can outperform known rates.
Efficiently combines autoregressive and set-based models for joint distributions.
problem Joint distributions over multiple predictions from set-based models.
method Causal autoregressive buffer that caches context and captures dependencies.
result Up to 20x faster joint sampling and density evaluation, up to 7x lower memory usage.
GAS-Norm improves deep learning time series forecasting in non-stationary settings.
problem Deep learning models struggle with non-stationary time series data.
method Combines GAS model for adaptive normalization with deep neural networks.
result Improves deep learning performance in 21 out of 25 settings.
New method for identifying autoregressive systems on manifolds.
problem Identifying autoregressive systems on Stiefel and Grassmann manifolds.
method Defining parameters as orthogonal group elements, averaging over observations, conjugate gradient descent on manifolds.
result System parameters can be estimated efficiently using the proposed algorithm.
A new clustering method for vector time series using autoregressive dynamics.
problem Clustering of vector time series based on their dynamics is challenging.
method System identification approach using mixture autoregressive models.
result Developed a computationally manageable algorithm k-LMVAR for clustering vector time series.
The vector autoregressive (VAR) model is a powerful tool in modeling complex time series and has been exploited in many fields. However, fitting high dimensional VAR model poses some unique challenges: On one hand, the dimensionality, caused by modeling a large number of time series and higher order autoregressive proc…
Standard autoregressive seq2seq models are easily trained by max-likelihood, but tend to show poor results under small-data conditions. We introduce a class of seq2seq models, GAMs (Global Autoregressive Models), which combine an autoregressive component with a log-linear component, allowing the use of global \textit{a…
The recently proposed mask-predict decoding algorithm has narrowed the performance gap between semi-autoregressive machine translation models and the traditional left-to-right approach. We introduce a new training method for conditional masked language models, SMART, which mimics the semi-autoregressive behavior of mas…
Linear autoregressive models serve as basic representations of discrete time stochastic processes. Different attempts have been made to provide non-linear versions of the basic autoregressive process, including different versions based on kernel methods. Motivated by the powerful framework of Hilbert space embeddings o…
Latent Block-Diffusion Temporal Point Processes (LBDTPP) is a semi-autoregressive framework for generating asynchronous event sequences.
problem Generating asynchronous event sequences
method Latent Block-Diffusion Temporal Point Processes
result Outperforms state-of-the-art TPP baselines in both unconditional and conditional generation tasks
Normalizing flows and autoregressive models have been successfully combined to produce state-of-the-art results in density estimation, via Masked Autoregressive Flows (MAF), and to accelerate state-of-the-art WaveNet-based speech synthesis to 20x faster than real-time, via Inverse Autoregressive Flows (IAF). We unify a…
A new online learning setting for autoregressive processes with sublinear regret.
problem Sequential decision-making with temporal dependence in autoregressive processes.
method Autoregressive Bandits (ARBs) and AutoRegressive Upper Confidence Bound (AR-UCB) algorithm.
result Sublinear regret of order $\widetilde{\mathcal{O}} \left( \frac{(k+1)^{3/2}\sqrt{nT}}{(1-Γ)^2}
ight)$ for optimal policy.
Bayesian method estimates Kronecker graphical models from autoregressive processes.
problem Estimating Kronecker graphical models from autoregressive Gaussian processes.
method Bayesian approach to estimate Kronecker graphical models.
result Effectiveness demonstrated through numerical experiments and real-world data application.
Autoregressive models struggle with hard-to-compute distributions, alternatives like energy-based and latent-variable models solve this.
problem Autoregressive models struggle with distributions whose next-symbol probability is hard to compute.
method Alternatives include energy-based models and latent-variable autoregressive models.
result Alternatives to autoregressive models can escape limitations of hard-to-compute distributions.
We describe an optimal adversarial attack formulation against autoregressive time series forecast using Linear Quadratic Regulator (LQR). In this threat model, the environment evolves according to a dynamical system; an autoregressive model observes the current environment state and predicts its future values; an attac…
Missing value imputation is a fundamental problem in spatiotemporal modeling, from motion tracking to the dynamics of physical systems. Deep autoregressive models suffer from error propagation which becomes catastrophic for imputing long-range sequences. In this paper, we take a non-autoregressive approach and propose …
EventFlow forecasts event sequences without autoregression, improving accuracy.
problem Forecasting errors in autoregressive models for event sequences.
method EventFlow uses flow matching to learn joint distributions over event times directly.
result EventFlow reduces forecast error by 20%-53% compared to baselines.
We propose a new class of models specifically tailored for spatio-temporal data analysis. To this end, we generalize the spatial autoregressive model with autoregressive and heteroskedastic disturbances, i.e. SARAR(1,1), by exploiting the recent advancements in Score Driven (SD) models typically used in time series eco…
Non-autoregressive method speeds up protein folding prediction 23 times.
problem Generating protein sequences with higher order interactions.
method Discrete diffusion conditioned on 3D structure using ProteinMPNN.
result 23 times speed up in inference without performance loss.
Study improves dividend discount model using VAR process.
problem Improving dividend discount models for better predictions.
method Introduced a Gordon growth model based on Vector Autoregressive Process (VAR).
result Two Propositions related to the new model.
The paper proposes autoregressive models for better offline RL.
problem Offline RL policy evaluation and optimization challenges.
method Autoregressive dynamics models for sequential state and reward prediction.
result Autoregressive models outperform standard methods in log-likelihood and RL tasks.
Proposes a non-autoregressive Transformer for time series forecasting.
problem Autoregressive errors and spatial-temporal dependencies in time series forecasting.
method Introduces a Non-Autoregressive Transformer with a learned temporal influence map.
result Demonstrates state-of-the-art performance on time series forecasting datasets.
While normalizing flows have led to significant advances in modeling high-dimensional continuous distributions, their applicability to discrete distributions remains unknown. In this paper, we show that flows can in fact be extended to discrete events---and under a simple change-of-variables formula not requiring log-d…
New method forecasts multilinear data using tensor autoregression.
problem Forecasting 2D data in big data.
method L-Transform Tensor autoregressive (L-TAR) method.
result Statistical independence achieved through invertible discrete linear transforms.
Efficient autoregressive entity linking with correction for faster, more accurate results.
problem High computational cost and non-parallelizable decoding in autoregressive entity linking.
method Parallelizes autoregressive linking across all mentions, uses a shallow decoder, and adds a discriminative correction term.
result 70 times faster and more accurate than previous methods, outperforming state-of-the-art approaches.
Optimizes Gaussian process hyperparameters using Bayesian autoregression.
problem Optimizing hyperparameters for Matérn kernel temporal Gaussian processes.
method Recursive Bayesian estimation for autoregressive parameters.
result Outperforms traditional optimization methods in runtime and accuracy.
Improved lattice field theory simulations with local-Autoregressive Conditional Normalizing Flow.
problem Efficiently sampling lattice field theories with computational challenges.
method Integrates locality into autoregressive conditional normalizing flows.
result Autocorrelation times improved by orders of magnitude for theory on a 2D lattice.
Autoregressive generative models of images tend to be biased towards capturing local structure, and as a result they often produce samples which are lacking in terms of large-scale coherence. To address this, we propose two methods to learn discrete representations of images which abstract away local detail. We show th…