New GLS estimator handles high-dimensional data with autocorrelated errors.
problem High-dimensional regressions with autocorrelated errors.
method LASSO regression, autoregressive model fitting, and whitening.
result The method outperforms unadjusted LASSO in estimating errors driven by autoregressive processes.
We derive generalization error bounds for stationary univariate autoregressive (AR) models. We show that imposing stationarity is enough to control the Gaussian complexity without further regularization. This lets us use structural risk minimization for model selection. We demonstrate our methods by predicting interest…
Proposes a non-autoregressive Transformer for time series forecasting.
problem Autoregressive errors and spatial-temporal dependencies in time series forecasting.
method Introduces a Non-Autoregressive Transformer with a learned temporal influence map.
result Demonstrates state-of-the-art performance on time series forecasting datasets.
Missing value imputation is a fundamental problem in spatiotemporal modeling, from motion tracking to the dynamics of physical systems. Deep autoregressive models suffer from error propagation which becomes catastrophic for imputing long-range sequences. In this paper, we take a non-autoregressive approach and propose …
Latent Block-Diffusion Temporal Point Processes (LBDTPP) is a semi-autoregressive framework for generating asynchronous event sequences.
problem Generating asynchronous event sequences
method Latent Block-Diffusion Temporal Point Processes
result Outperforms state-of-the-art TPP baselines in both unconditional and conditional generation tasks
EventFlow forecasts event sequences without autoregression, improving accuracy.
problem Forecasting errors in autoregressive models for event sequences.
method EventFlow uses flow matching to learn joint distributions over event times directly.
result EventFlow reduces forecast error by 20%-53% compared to baselines.
We develop methods to estimate lag and parameters for multiple stable autoregressive processes.
problem Estimating lag and parameters for multiple stable autoregressive processes with unknown lag.
method Use convex programming to simultaneously select lag and estimate parameters across multiple processes.
result The estimated process is stable, and forecasting errors can outperform known rates.
Optimizes Gaussian process hyperparameters using Bayesian autoregression.
problem Optimizing hyperparameters for Matérn kernel temporal Gaussian processes.
method Recursive Bayesian estimation for autoregressive parameters.
result Outperforms traditional optimization methods in runtime and accuracy.
This paper provides theoretical foundations for using quantized actions in behavior cloning.
problem Applying autoregressive models to continuous control requires discretizing actions through quantization, which is poorly understood.
method The paper analyzes quantization error propagation and statistical sample complexity, and proposes model-based augmentation.
result Behavior cloning with quantized actions achieves optimal sample complexity, matching existing lower bounds.
Volatility is a key measure of risk in financial analysis. The high volatility of one financial asset today could affect the volatility of another asset tomorrow. These lagged effects among volatilities - which we call volatility spillovers - are studied using the Vector AutoRegressive (VAR) model. We account for the p…
Autoregressive networks can achieve promising performance in many sequence modeling tasks with short-range dependence. However, when handling high-dimensional inputs and outputs, the huge amount of parameters in the network lead to expensive computational cost and low learning efficiency. The problem can be alleviated …
The purpose of this paper is to propose a time-varying vector autoregressive model (TV-VAR) for forecasting multivariate time series. The model is casted into a state-space form that allows flexible description and analysis. The volatility covariance matrix of the time series is modelled via inverted Wishart and singul…
Optimizes prediction error method for time-varying models.
problem Achieving optimal prediction error rates for time-varying models.
method Nonlinear least squares method for time-varying parametric models.
result First rate-optimal non-asymptotic analysis for time-varying models.
The paper introduces a method to model error correlations in multivariate time series forecasting.
problem Accurate modeling of error correlations for reliable uncertainty quantification.
method Plug-and-play method that learns error covariance over multiple steps using low-rank-plus-diagonal and independent latent temporal processes.
result Improves predictive accuracy and uncertainty quantification without significantly increasing parameter size.
Paper explores variable skipping to speed up range density estimation.
problem Efficiently estimating range densities over high-dimensional data.
method Variable skipping technique to accelerate range density estimation.
result 10-100x efficiency improvements in challenging high-quantile error metrics.
New bidirectional model predicts magnetohydrodynamics fields and estimates uncertainty.
problem Predicting multiple fields in magnetohydrodynamics with uncertainty.
method Bidirectional autoregressive latent diffusion approach.
result Model can estimate uncertainty without ground truth using self-supervised consistency.
Single autoregressive model outperforms ensemble methods in offline reinforcement learning.
problem Offline reinforcement learning with limited data and model errors.
method Infer system dynamics from data and optimize policies on model rollouts, using a single autoregressive model.
result Single autoregressive model achieves better performance than ensembles on the D4RL benchmark.
Paper unifies subspace identification and DMD for dynamical systems.
problem Estimating dynamical models from data.
method Unified optimization and regression problems for SID and DMD.
result Proves equivalence of SID and DMD for optimal model construction.
Proposes a Structural Matrix Autoregressive model for joint analysis of asset returns, realized volatility, and trading volume.
problem Joint analysis of asset returns, realized volatility, and trading volume
method Structural Matrix Autoregressive model
result Volatility is primary driver of trading activity, with informational shocks incorporated through price variability.
In this work, we consider the class of multi-state autoregressive processes that can be used to model non-stationary time-series of interest. In order to capture different autoregressive (AR) states underlying an observed time series, it is crucial to select the appropriate number of states. We propose a new model sele…
The paper tackles deep learning from dependent data, achieving optimal performance.
problem Deep learning from strongly mixing observations, especially with regularization and optimality.
method Sparse-penalized regularization for deep neural networks, oracle inequality for expected excess risk.
result Deep neural network estimator achieves minimax optimal rate for nonparametric autoregression.
SpinSVAR estimates SVAR models with sparse input, improving accuracy and scalability.
problem Estimating SVAR models with sparse input assumptions.
method SpinSVAR models input as independent Laplacian variables, enforcing sparsity and using least absolute error regression.
result SpinSVAR outperforms state-of-the-art methods in accuracy and runtime, identifying significant structural shocks.
New framework IIA identifies innovations in general nonlinear vector autoregressive processes.
problem Limited generality of NVAR models due to additive innovation assumption.
method Independent Innovation Analysis (IIA) framework, assuming mutual independence and modulation by an auxiliary variable.
result Guarantees identifiability of innovations with arbitrary nonlinearities, up to permutation and component-wise invertible nonlinearities.
Thermalizer stabilizes autoregressive models for long-term predictions in chaotic systems.
problem Long-term predictions in chaotic spatiotemporal systems are unreliable due to trajectory divergence.
method Diffusion models are used to implicitly estimate the score of an invariant measure, which stabilizes autoregressive emulators by applying denoising during inference.
result Thermalization extends the time horizon of stable predictions by an order of magnitude in chaotic systems.
Bayesian framework selects features and lags for time series forecasting.
problem Variable selection and lagged error term identification in time series models.
method Hierarchical Bayesian models with spike-and-slab priors, two-stage MCMC algorithm.
result Posterior selection consistency under mild conditions, improved predictive performance.
Behavior cloning training instabilities amplified by SGD noise over long horizons.
problem Training instabilities in behavior cloning with deep neural networks.
method Empirical dissection of minibatch SGD updates and their effects on long-horizon rewards.
result Exponential moving average (EMA) of iterates effectively mitigates gradient variance amplification (GVA).
Paper proposes a neural network method for fast, interpretable AR model estimation.
problem Computational inefficiency and convergence issues in conventional AR model estimation.
method Embeds autoregressive structure into a feedforward neural network for coefficient estimation via backpropagation.
result Neural network method consistently recovers AR model coefficients, converging in all cases and providing reliable estimates.
BAVART model combines VAR and BART for non-linear forecasting.
problem Overly restrictive linearity assumption in VAR models.
method Combining VAR with Bayesian additive regression trees (BART).
result BAVART model yields highly competitive forecasts.
Proposes PredVAR model for reduced-dimensional dynamics from noisy data.
problem Extracting low-dimensional dynamics from high-dimensional noisy data.
method Probabilistic reduced-dimensional vector autoregressive model with oblique projection.
result Iterative algorithm yields dynamic latent variables with rank-ordered predictability.
Paper presents a copula-based method to efficiently generate correlated sample paths from multi-step time series models.
problem Generating realistic correlation structures in multi-step forecast sample paths is expensive and time-consuming.
method Copula-based approach to generate correlated sample paths in one forward pass.
result Improved sample path quality and significant speedup over autoregressive sampling.
The paper proposes a new model for predicting and analyzing economic variables.
problem Predicting and analyzing economic variables in developed regions.
method Time-varying parameter global vector autoregressive (TVP-GVAR) framework combined with machine learning models.
result The proposed model provides high precision out-of-sample predictions and novel insights into economic variable connectedness.
CARRNN tackles deep learning for sporadic data, improving prediction errors in healthcare.
problem Challenges in learning temporal patterns from sporadic multivariate longitudinal data.
method Developed a novel deep learning architecture combining RNN and CAR models, using a generalized discrete-time autoregressive model.
result CARRNN achieves the lowest prediction errors in multivariate time-series regression tasks.
A new autoregressive SPO method improves decision-making for dependent data.
problem Improving decision-making for dependent data in stochastic optimization.
method An autoregressive Smart Predict-then-Optimize (SPO) method for time series data.
result Generalization bounds and uniform calibration results for the SPO loss in autoregressive models.
The paper compares multi-fidelity methods for Gaussian process surrogates in physics.
problem Limited availability of data due to expensive simulations.
method Extending non-linear autoregressive methods to multi-fidelity models and incorporating delay terms.
result Multi-fidelity methods generally have smaller prediction error for the same computational cost.
MEM models improve volatility forecasting in financial markets.
problem Improving volatility forecasting in financial markets.
method Multiplicative Error Model (MEM) framework for positive-valued time series.
result MEMs parsimoniously produce good forecasts of asset returns.
In this paper we consider portmanteau tests for testing the adequacy of multiplicative seasonal autoregressive moving-average (SARMA) models under the assumption that the errors are uncorrelated but not necessarily independent.We relax the standard independence assumption on the error term in order to extend the range …
This paper shows ARMs and EBMs are equivalent, revealing ARM lookahead capabilities.
problem Understanding the lookahead capabilities of next-token prediction models.
method Unified view of ARMs and EBMs, establishing a bijection and deriving equivalence.
result ARMs and EBMs are equivalent, revealing ARM lookahead capabilities.
Sparse Tucker decomposition with graph regularization improves time series forecasting accuracy.
problem High-dimensional time series forecasting with over-parameterization issue.
method Sparse Tucker decomposition and graph regularization for tensor-based model.
result Non-asymptotic error bound and superior performance in numerical experiments.
Uncertainty estimation is important for ensuring safety and robustness of AI systems. While most research in the area has focused on un-structured prediction tasks, limited work has investigated general uncertainty estimation approaches for structured prediction. Thus, this work aims to investigate uncertainty estimati…
Variational Bayes (VB) is a recent approximate method for Bayesian inference. It has the merit of being a fast and scalable alternative to Markov Chain Monte Carlo (MCMC) but its approximation error is often unknown. In this paper, we derive the approximation error of VB in terms of mean, mode, variance, predictive den…
Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling th…
The 2006 sudden and immense downturn in U.S. House Prices sparked the 2007 global financial crisis and revived the interest about forecasting such imminent threats for economic stability. In this paper we propose a novel hybrid forecasting methodology that combines the Ensemble Empirical Mode Decomposition (EEMD) from …
This work proposes an efficient autoregressive model for text generation.
problem The challenge of generating high-quality text with autoregressive models.
method Introduces a cascaded decoding approach using Markov transformers to achieve sub-linear parallel time generation.
result Shows competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
New method for estimating and testing impulse responses in high-dimensional VAR systems.
problem Statistical inference for impulse responses in sparse, high-dimensional vector autoregressions.
method Local projection equations and de-sparsified estimators combined with a non-regularized contemporaneous impact matrix.
result Valid inference procedures for structural impulse responses in high-dimensional systems.
This paper establishes non-asymptotic oracle inequalities for the prediction error and estimation accuracy of the LASSO in stationary vector autoregressive models. These inequalities are used to establish consistency of the LASSO even when the number of parameters is of a much larger order of magnitude than the sample …
Autoregressive sequence models achieve state-of-the-art performance in domains like machine translation. However, due to the autoregressive factorization nature, these models suffer from heavy latency during inference. Recently, non-autoregressive sequence models were proposed to reduce the inference time. However, the…
Bayesian method for multivariate autoregressive models with exogenous inputs.
problem Estimating uncertainties in autoregressive models with exogenous inputs.
method Recursive Bayesian estimation via message passing in a factor graph.
result Produces full posterior distributions for autoregressive coefficients and noise precision.
Autoregressive state transitions, where predictions are conditioned on past predictions, are the predominant choice for both deterministic and stochastic sequential models. However, autoregressive feedback exposes the evolution of the hidden state trajectory to potential biases from well-known train-test discrepancies.…