Alternative sampling method for autoregressive models using Langevin dynamics.
problem Efficiently sampling from autoregressive models.
method Initialize sequences with white noise and follow Langevin dynamics on global log-likelihood.
result Parallelizes and generalizes sampling process for autoregressive models.
The paper proposes autoregressive models for better offline RL.
problem Offline RL policy evaluation and optimization challenges.
method Autoregressive dynamics models for sequential state and reward prediction.
result Autoregressive models outperform standard methods in log-likelihood and RL tasks.
A new clustering method for vector time series using autoregressive dynamics.
problem Clustering of vector time series based on their dynamics is challenging.
method System identification approach using mixture autoregressive models.
result Developed a computationally manageable algorithm k-LMVAR for clustering vector time series.
Linear attention in Transformers can be interpreted as dynamic VAR models.
problem Misalignment between Transformers and autoregressive forecasting objectives.
method Interpreting linear attention as VAR, rearranging MLP, attention, and flow.
result SAMoVAR improves performance, interpretability, and efficiency.
Missing value imputation is a fundamental problem in spatiotemporal modeling, from motion tracking to the dynamics of physical systems. Deep autoregressive models suffer from error propagation which becomes catastrophic for imputing long-range sequences. In this paper, we take a non-autoregressive approach and propose …
We propose a new class of models specifically tailored for spatio-temporal data analysis. To this end, we generalize the spatial autoregressive model with autoregressive and heteroskedastic disturbances, i.e. SARAR(1,1), by exploiting the recent advancements in Score Driven (SD) models typically used in time series eco…
The optimal predictor for a linear dynamical system (with hidden state and Gaussian noise) takes the form of an autoregressive linear filter, namely the Kalman filter. However, a fundamental problem in reinforcement learning and control theory is to make optimal predictions in an unknown dynamical system. To this end, …
Modified asymmetric hidden Markov models for time series with autoregressive components.
problem Dynamic relationships between variables in time series data.
method Introducing an asymmetric autoregressive component to recent asymmetric hidden Markov models.
result The model can choose the optimal autoregressive order for better likelihood.
Flow-based generative models are a family of exact log-likelihood models with tractable sampling and latent-variable inference, hence conceptually attractive for modeling complex distributions. However, flow-based models are limited by density estimation performance issues as compared to state-of-the-art autoregressive…
Proposes PredVAR model for reduced-dimensional dynamics from noisy data.
problem Extracting low-dimensional dynamics from high-dimensional noisy data.
method Probabilistic reduced-dimensional vector autoregressive model with oblique projection.
result Iterative algorithm yields dynamic latent variables with rank-ordered predictability.
VB approach for dynamic network models improves efficiency and accuracy.
problem Estimating dynamic network models in large-scale systems.
method Variational Bayesian inference for network autoregression.
result VB approach detects proper active structures and achieves similar or better accuracy.
Proposes a Structural Matrix Autoregressive model for joint analysis of asset returns, realized volatility, and trading volume.
problem Joint analysis of asset returns, realized volatility, and trading volume
method Structural Matrix Autoregressive model
result Volatility is primary driver of trading activity, with informational shocks incorporated through price variability.
AR model forecasts partially observed dynamical time series by estimating evolution function and imputing missing variables.
problem Forecasting dynamical time series with missing variables.
method Autoregressive with slack time series (ARS) model.
result ARS model forecasts future time series with time-invariant and linear assumptions.
Paper proposes AXE loss for non-autoregressive machine translation, improving performance.
problem Challenges in training non-autoregressive models due to lack of autoregressive factors and cross entropy loss penalties.
method Proposes aligned cross entropy (AXE) loss function using a differentiable dynamic program for better word order alignment.
result AXE-based training improves performance on major WMT benchmarks and sets a new state of the art for non-autoregressive models.
The paper introduces reservoir computing models for complex systems.
problem Modeling complex engineering systems using nonlinear autoregression.
method Introduces reservoir computing with output feedback as stationary and ergodic infinite-order nonlinear autoregressive models.
result Demonstrates versatility of classical and quantum reservoir computers in modeling synthetic and real data.
SALT models combine ARHMM and SLDS for efficient, interpretable time-series analysis.
problem Efficient modeling of systems with time-varying dynamics and long-range dependencies.
method Switching autoregressive low-rank tensor models parameterized with a low-rank factorization.
result SALT models provide a balance of interpretability and efficiency, outperforming ARHMMs and SLDSs.
Autoregressive sequence models achieve state-of-the-art performance in domains like machine translation. However, due to the autoregressive factorization nature, these models suffer from heavy latency during inference. Recently, non-autoregressive sequence models were proposed to reduce the inference time. However, the…
We describe an optimal adversarial attack formulation against autoregressive time series forecast using Linear Quadratic Regulator (LQR). In this threat model, the environment evolves according to a dynamical system; an autoregressive model observes the current environment state and predicts its future values; an attac…
We demonstrate the use of conditional autoregressive generative models (van den Oord et al., 2016a) over a discrete latent space (van den Oord et al., 2017b) for forward planning with MCTS. In order to test this method, we introduce a new environment featuring varying difficulty levels, along with moving goals and obst…
WassersteinGrad improves weather forecasting explanations by addressing geometric misalignment issues.
problem Improving explainability of autoregressive neural predictions on dynamic physical fields.
method WassersteinGrad, a geometric consensus method for averaged perturbed attribution maps.
result WassersteinGrad provides more accurate explanations for weather forecasting models.
Many complex dynamical phenomena can be effectively modeled by a system that switches among a set of conditionally linear dynamical modes. We consider two such models: the switching linear dynamical system (SLDS) and the switching vector autoregressive (VAR) process. Our Bayesian nonparametric approach utilizes a hiera…
A new perspective on self-attention models using MLPs.
problem Improving sequence modeling with self-attention mechanisms.
method Introducing HyperMLP and HyperGLU, which use dynamic two-layer MLPs with reverse-offset layout.
result HyperMLP/HyperGLU consistently outperform softmax-attention baselines.
Proposes DCNAR for dynamic causal inference from neural time series.
problem Uncertainty and evolution of causal structure in real-world domains.
method Two-stage neural causal modeling integrating discovery and inference.
result Dynamic causal inferences are more stable and meaningful than alternatives.
Novel F2NARX model improves surrogate modeling for stochastic dynamical systems.
problem Challenges in constructing accurate and efficient surrogate models for stochastic dynamical systems.
method Function-on-Function Nonlinear AutoRegressive model with eXogenous inputs (F2NARX) combining PCA and Gaussian process regression.
result F2NARX outperforms state-of-the-art NARX models in efficiency and accuracy.
FLUID-LLM uses LLMs to predict fluid dynamics with improved accuracy.
problem Leveraging LLMs for CFD due to their pattern recognition abilities but struggles with fluid dynamics complexities.
method Combines pre-trained LLMs with spatiotemporal-aware encoding to predict unsteady fluid dynamics.
result Significant performance improvements in CFD predictions across various datasets.
Paper uses surprisal to dynamically allocate computation between fast and slow models.
problem Dynamic allocation of computation in neural networks.
method Surprisal-based dynamic model selection.
result Model can match baseline performance with 15% fewer FLOPs.
Paper unifies subspace identification and DMD for dynamical systems.
problem Estimating dynamical models from data.
method Unified optimization and regression problems for SID and DMD.
result Proves equivalence of SID and DMD for optimal model construction.
New framework IIA identifies innovations in general nonlinear vector autoregressive processes.
problem Limited generality of NVAR models due to additive innovation assumption.
method Independent Innovation Analysis (IIA) framework, assuming mutual independence and modulation by an auxiliary variable.
result Guarantees identifiability of innovations with arbitrary nonlinearities, up to permutation and component-wise invertible nonlinearities.
Study examines dynamic relationship between BRICS stocks and cryptocurrencies.
problem Understanding the impact of BRICS stock markets on cryptocurrency markets.
method Time-varying parameter vector autoregression model (TVP-VAR).
result Three out of five BRICS stock markets are primary sources of shocks affecting the financial network.
Transformers learn a mesa-optimizer to implement in-context learning.
problem Understanding the convergence of autoregressive training to a mesa-optimizer.
method Investigated a one-layer linear causal self-attention model autoregressively trained by gradient flow.
result Proved that autoregressive training converges to a gradient descent step for an OLS problem, validating the mesa-optimizer hypothesis.
Agent uses message passing to optimize robot navigation, balancing exploration and exploitation.
problem Optimizing robot navigation in continuous-valued spaces with uncertainty.
method Autoregressive active inference agent using message passing on a factor graph.
result Agent modulates action based on predictive uncertainty, leading to better model of dynamics.
A new autoregressive model learns the order of graph generation tasks.
problem Generating graphs in a meaningful order when the canonical order is not obvious.
method Introduces a variant of autoregressive models that dynamically decides the autoregressive order based on data.
result Achieves state-of-the-art results on molecular graph generation benchmarks.
Agents learn and control complex mechanical systems through shared memories.
problem Controlling multi-joint dynamical systems.
method Coupled autoregressive active inference agents using Bayesian filtering and minimizing expected free energy.
result Demonstrated learning and control of a double mass-spring-damper system.
CAESar improves risk forecasting by combining VaR and ES estimates.
problem Lack of tail risk measures in financial risk management.
method Conditional Autoregressive Expected Shortfall model, combining VaR and ES estimates.
result CAESar outperforms existing methods in risk forecasting.
Paper proposes ARPHMM for fault detection and prognosis in aero-engines.
problem Fault detection and prognosis in aero-engines using sensor data.
method Autoregressive Partially-hidden Markov Model (ARPHMM) with prior knowledge integration.
result Model estimates remaining useful life and degradation level.
Bayesian method tests Granger causality in functional time series.
problem Testing Granger causality between functional time series.
method Bayesian dynamic linear models (DLM) and Bayes Factor.
result Captures Granger causality between yield curves and weather conditions.
We introduce a novel multivariate random process producing Bernoulli outputs per dimension, that can possibly formalize binary interactions in various graphical structures and can be used to model opinion dynamics, epidemics, financial and biological time series data, etc. We call this a Bernoulli Autoregressive Proces…
A new framework predicts links in time-dependent networks using Bernoulli autoregression.
problem Predicting links in time-dependent networks with additional auxiliary information.
method A Bernoulli autoregressive model with regularization for link discovery.
result The model can discover new links not present in the data.
Single autoregressive model outperforms ensemble methods in offline reinforcement learning.
problem Offline reinforcement learning with limited data and model errors.
method Infer system dynamics from data and optimize policies on model rollouts, using a single autoregressive model.
result Single autoregressive model achieves better performance than ensembles on the D4RL benchmark.
LGB+ improves macroeconomic forecasting by combining linear and tree models.
problem Efficiency in small samples for forecasting with mixed linear and nonlinear dynamics.
method LGB+ is a boosting procedure that evaluates both tree and linear candidates at each step, advancing only the winner. It decomposes forecasts into linear and nonlinear contributions.
result LGB+ delivers strong gains for targets with pronounced autoregressive dynamics or mixed signals.
Bayesian method detects change points in time series data.
problem Detecting significant regime shifts in time series data.
method Bayesian autoregressive model with time-varying parameters.
result Enhanced estimate accuracy and forecasting power.
FOCUS method forecasts counterfactuals in panel data with time series dynamics.
problem Forecasting unobserved potential outcomes in causal inference with missing entries and latent factors.
method FOCUS extends matrix completion methods by leveraging time series dynamics of latent factors.
result FOCUS method outperforms existing benchmarks in predicting future counterfactuals.
Study shows how market efficiency changes during the pandemic.
problem Understanding market efficiency during the pandemic.
method Applied time-varying vector autoregression model.
result Market efficiency changes over time and can be improved by enhanced linkages.
Incorporating nonlinearity is paramount to predicting the future states of a dynamical system, its response to shocks, and its underlying causal network. However, most existing methods for causality detection and impulse response, such as Vector Autoregression (VAR), assume linearity and are thus unable to capture the …
In order to disentangle the internal dynamics from exogenous factors within the Autoregressive Conditional Duration (ACD) model, we present an effective measure of endogeneity. Inspired from the Hawkes model, this measure is defined as the average fraction of events that are triggered due to internal feedback mechanism…
High-dimensional time series data exist in numerous areas such as finance, genomics, healthcare, and neuroscience. An unavoidable aspect of all such datasets is missing data, and dealing with this issue has been an important focus in statistics, control, and machine learning. In this work, we consider a high-dimensiona…
Enhanced EEG classification using augmented covariance matrix.
problem Improving motor imagery classification from EEG signals.
method Proposes a new framework based on the augmented covariance matrix derived from an autoregressive model.
result The augmented covariance matrix outperformed state-of-the-art methods.
New framework for dynamic causal graph modeling and effect estimation.
problem Dynamic changes in causal relationships over time.
method Score-based causal discovery with autoregressive model structure.
result Dynamic causal graph with time-varying causal relations.