DSARF models complex spatio-temporal data with deep switching auto-regressive factors.
problem Forecasting complex spatio-temporal data with recurring patterns.
method Deep switching auto-regressive factorization (DSARF) with stochastic variational inference.
result DSARF outperforms state-of-the-art methods in long- and short-term prediction accuracy.
Hybrid model combines VAR and neural network for OFI prediction.
problem Accurate prediction of Order Flow Imbalance (OFI) in high frequency trading.
method Combines Vector Auto Regression (VAR) and a simple feedforward neural network (FNN).
result Hybrid model achieves superior predictive accuracy compared to standalone models.
Auto-regressive models improve smoothing efficiency with exponentially tapered windows.
problem Improving time-series smoothing efficiency.
method An auto-regressive formulation for time-series smoothing.
result Auto-regressive models result in moving means with exponentially tapered windows.
Proof of Gaussian ML estimator consistency in linear auto-regressive models.
problem Consistency of Gaussian maximum likelihood estimator in linear auto-regressive models.
method Information-theoretic proof without stability assumptions.
result Nearly optimal non-asymptotic rates for parameter recovery.
CAFLOW uses auto-regressive flows to translate images efficiently.
problem Image-to-image translation tasks.
method Transforms conditioning image into latent encodings using normalizing flows, models conditional distribution with auto-regressive distributions.
result Outperforms former conditional flow designs.
Auto-regressive models learn latent states from partially observed linear dynamical systems.
problem Understanding how auto-regressive models learn latent representations from partially observed linear dynamical systems.
method Empirical risk minimization on partially observed linear dynamical systems.
result Two-layer linear auto-regressive models learn to approximate Kalman filtering, coinciding with optimal state estimates.
Rhino learns causal relationships from time series data with history-dependent noise.
problem Discovering causal relationships from time series data with non-linear relations, instantaneous effects, and history-dependent noise.
method Combines vector auto-regression, deep learning, and variational inference.
result Demonstrates better causal relationship discovery performance compared to baselines.
One popular approach for nonstructural economic and financial forecasting is to include a large number of economic and financial variables, which has been shown to lead to significant improvements for forecasting, for example, by the dynamic factor models. A challenging issue is to determine which variables and (their)…
This paper compares traditional econometric and contemporary machine/deep learning techniques for forecasting foreign exchange rates.
problem Accurate prediction of foreign exchange rates for investment purposes.
method Multivariate time series analysis using Vector Auto Regression, Support Vector Machine, and Recurrent Neural Networks.
result Contemporary machine/deep learning techniques outperform traditional econometric methods in forecasting foreign exchange rates.
VEST automates feature engineering for time series forecasting.
problem Challenges in time series forecasting with improved performance.
method VEST combines auto-regression with statistical summarization of recent past dynamics.
result VEST significantly improves forecasting performance.
In this paper, non-linear time series models are used to describe volatility in financial time series data. To describe volatility, two of the non-linear time series are combined into form TAR (Threshold Auto-Regressive Model) with AARCH (Asymmetric Auto-Regressive Conditional Heteroskedasticity) error term and its par…
Cactus improves auto-regressive decoding speed without sacrificing quality.
problem Accelerating auto-regressive decoding while maintaining output quality.
method Formalizes speculative sampling as constrained optimization and proposes Cactus for controlled divergence from the verifier distribution.
result Empirically validated effectiveness across various benchmarks.
Paper reduces vocabulary losslessly for language model cooperation.
problem Language models struggle to cooperate with different tokenizations.
method Established a theoretical framework for lossless vocabulary reduction.
result Efficiently converts models with different tokenizations to cooperate with maximal common vocabulary.
TraDE uses self-attention for better density estimation of tabular and image data.
problem Improving density estimation for tabular and image data.
method Self-attention-based architecture trained with a penalized maximum likelihood objective.
result TraDE produces significantly better density estimates than existing methods.
The paper proposes a new auto-regressive model for multivariate distributional time series.
problem Statistical analysis of multivariate time series of probability measures.
method Wasserstein space, auto-regressive model, iterated random function systems.
result Consistent estimator for auto-regressive coefficients with sparse structure.
A new RG approach connects discrete and continuous time descriptions of Gaussian processes.
problem Discretization of continuous stochastic processes for accurate simulation or model inference.
method Renormalization Group (RG) approach for Gaussian time series generated by auto-regressive models.
result RG fixed points correspond to discretizations of linear SDEs, providing insights into process accuracy.
VAR-GPs solve continual learning by updating posteriors sequentially.
problem Catastrophic forgetting in sequential learning tasks.
method Sparse inducing point approximations and auto-regressive variational distribution.
result VAR-GPs prevent catastrophic forgetting and outperform baselines.
This paper studies the dynamic generator model for spatial-temporal processes such as dynamic textures and action sequences in video data. In this model, each time frame of the video sequence is generated by a generator model, which is a non-linear transformation of a latent state vector, where the non-linear transform…
Consider a multi-variate time series (Xt)t=0T where Xt∈Rd which may represent spike train responses for multiple neurons in a brain, crime event data across multiple regions, and many others. An important challenge associated with these time series models is to estimate an influence network be…
Online graph learning from matrix-valued time series data.
problem Identifying dependency structure among sensors in a network.
method Extends VAR models to matrix-variate models, proposes online procedures for graph learning, and introduces Lasso-type approaches.
result Demonstrates effectiveness of online graph learning methods in both synthetic and real data.
We propose a vector auto-regressive (VAR) model with a low-rank constraint on the transition matrix. This new model is well suited to predict high-dimensional series that are highly correlated, or that are driven by a small number of hidden factors. We study estimation, prediction, and rank selection for this model in …
New methods correct for time dependencies in IV regression for time series data.
problem Inferring causal effects from time series data with unobserved confounders.
method Proposes new methods for consistent estimation of causal effects in time series models using nuisance covariates and graph marginalization.
result Identifies and corrects for dependencies in the past, leading to consistent estimation of causal effects.
BMBO-DARN optimizes expensive functions with varying fidelities.
problem Optimizing expensive, multi-fidelity functions efficiently.
method Batch Multi-fidelity Bayesian Optimization with Deep Auto-Regressive Networks.
result BMBO-DARN improves surrogate learning and optimization performance.
We present a simple algorithm to forecast vector time series, that is robust against missing data, in both training and inference. It models seasonal annual, weekly, and daily baselines, and a Gaussian process for the seasonally-adjusted residuals. We develop a custom truncated eigendecomposition to fit a low-rank plus…
Improved text generation with constraints using discrete auto-regressive biasing.
problem Balancing fluency and constraint satisfaction in LLM outputs.
method Discrete Auto-regressive Biasing, leveraging gradients in discrete text space.
result Significantly improved constraint satisfaction with comparable fluency.
A new method learns discrete representations for images and videos, improving upon previous models.
problem Learning discrete representations for images and videos to improve performance.
method Depthwise application of Vector Quantized Variational Autoencoders (VQVAE) to feature axis.
result 33% improvement in performance compared to previous discrete models.
In this work we investigate approaches to reconstruct generator models from measurements available at the generator terminal bus using machine learning (ML) techniques. The goal is to develop an emulator which is trained online and is capable of fast predictive computations. The training is illustrated on synthetic dat…
A new method estimates expectations from subtractive mixture models without sampling.
problem Estimating expectations from multimodal distributions using SMMs.
method Difference representation of SMMs to create unbiased IS estimator (ΔextEx). result Demonstrates that ΔextEx can achieve comparable estimation quality to auto-regressive sampling but is faster. While considerable advances have been made in estimating high-dimensional structured models from independent data using Lasso-type models, limited progress has been made for settings when the samples are dependent. We consider estimating structured VAR (vector auto-regressive models), where the structure can be capture…
MDMs train to decode tokens in a random order, which affects performance; we show they can be optimized for a favorable order.
problem Performance of MDMs is affected by the random order in which tokens are decoded.
method We show that MDMs can be optimized for a favorable decoding order by equipping their continuous-time variational objective with multivariate noise schedules.
result MDMs can be decomposed into a weighted auto-regressive losses over orders, making them auto-regressive models with learnable orders.
Auto-regressive diffusion models improve capturing conditional dependence in data.
problem Vanilla diffusion models struggle to capture important, high-level relationships in real-world data.
method Developed auto-regressive diffusion models to better capture conditional dependence structures.
result AR diffusion models produce samples with a reduced gap in approximating the data conditional distribution.
A new method improves text generation quality and diversity.
problem Exposure bias in Maximum Likelihood Estimation for text generation.
method ψ-MLE, a new training scheme based on density ratio estimation.
result ψ-MLE outperforms Maximum Likelihood Estimation and other models in text generation quality and diversity.
Low-rank forecasting improves consistency in time series predictions.
problem Forecasting multiple values of a time series using past values.
method Breaks forecasting into estimating a latent state and future values, using convex optimization.
result Forecast consistency is achieved, meaning estimates of the same value at different times are consistent.
We introduce an auto-regressive model which captures the growing nature of realistic markets. In our model agents do not trade with other agents, they interact indirectly only through a market. Change of their wealth depends, linearly on how much they invest, and stochastically on how much they gain from the noisy mark…
Numerous empirical proofs indicate the adequacy of the time discrete auto-regressive stochastic volatility models introduced by Taylor in the description of the log-returns of financial assets. The pricing and hedging of contingent products that use these models for their underlying assets is a non-trivial exercise due…
DBNs predict cryptocurrency price directions by uncovering causal relationships.
problem Predicting cryptocurrency price movements due to volatility and external factors.
method Dynamic Bayesian Networks (DBN) approach to identify causal relationships among features.
result DBN significantly outperforms baseline models in predicting cryptocurrency prices.
Transformer adapts to graphs with adaptive attention and auto-regressive decoding.
problem Transformers struggle with graph data due to non-sequential nature.
method Proposes GRAT, a Transformer variant with adaptive attention and auto-regressive decoding.
result GRAT achieves state-of-the-art performance on molecule property predictions and generation tasks.
Improved language models by incorporating statistical discriminators.
problem Distinguishing model-generated text from real text reliably.
method Energy-Based Model framework to incorporate discriminators.
result Improves language model performance in perplexity and human evaluation.
Simple GBRT model improved by window-based input transformation outperforms state-of-the-art deep learning models.
problem Improving performance of traditional forecasting models for time series data.
method Transformed GBRT model input structure to include target values and external features, forming one input instance per training window.
result Simple GBRT model with window-based input transformation outperformed state-of-the-art deep learning models on nine datasets.
Enhanced multi-fidelity models improve digital twin accuracy and uncertainty quantification.
problem Lack of detailed application-specific data and inaccurate sensor data hinder surrogate model learning for digital twins.
method Proposes a multi-fidelity surrogate model framework integrating PCFE and GP, and deep-HPCFE with auto-regression schemes.
result Demonstrates improved accuracy and uncertainty quantification in digital twin systems.
New approach predicts tokens in context, explaining how ICL emerges.
problem Limited understanding of in-context learning emergence.
method Auto-regressive next-token prediction (AR-NTP) with prompt token-dependency and a two-level expectation.
result ICL emerges from the generalization of sequences and topics.
The paper analyzes prediction error in nonstationary settings using weighted risk minimization.
problem Prediction under distribution drift and nonstationary conditions.
method General decomposition of excess risk into learning and drift terms, proving oracle inequalities under mixing conditions.
result Oracle inequalities for the learning error, providing bounds that hold uniformly over arbitrary weight classes.
We propose a generic confidence-based approximation that can be plugged in and simplify the auto-regressive generation process with a proved convergence. We first assume that the priors of future samples can be generated in an independently and identically distributed (i.i.d.) manner using an efficient predictor. Given…
Generative models speed up complex system simulations.
problem Accurately forecasting the dynamics of complex systems at reduced cost.
method Generative Learning of Effective Dynamics (G-LED) using auto-regressive attention and Bayesian diffusion models.
result Generative models can accurately forecast complex system dynamics at lower computational cost.
Granger causality reviewed and advanced for complex data.
problem Validity of inferring causal relationships from time series data.
method Recent advances in models for high-dimensional time series, accounting for nonlinear and non-Gaussian observations, and sub-sampled data.
result Improved computational tools for Granger causality.
Recent neural text-to-speech (TTS) models with fine-grained latent features enable precise control of the prosody of synthesized speech. Such models typically incorporate a fine-grained variational autoencoder (VAE) structure, extracting latent features at each input token (e.g., phonemes). However, generating samples …
We present a new classification method for quasar identification in the EROS-2 and MACHO datasets based on a boosted version of Random Forest classifier. We use a set of variability features including parameters of a continuous auto regressive model. We prove that continuous auto regressive parameters are very importan…
Study error bounds and optimal schedules for Masked Diffusions with factorized approximations.
problem Analyzing trade-offs between computation and accuracy in Masked Diffusion Models.
method Provided general error bounds and identified optimal schedules based on data distribution information profiles.
result Identified optimal schedule sizes for Masked Diffusion Models.