A method for generating future samples in auto-regressive processes using confidence-based sampling.
problem Generating future samples in auto-regressive processes efficiently and accurately.
method Confidence-based sampling to simplify and parallelize the auto-regressive generation process.
result The method successfully captures complex data structures and generates meaningful future samples with lower computational cost.
CAFLOW uses auto-regressive flows to translate images efficiently.
problem Image-to-image translation tasks.
method Transforms conditioning image into latent encodings using normalizing flows, models conditional distribution with auto-regressive distributions.
result Outperforms former conditional flow designs.
VAR-GPs solve continual learning by updating posteriors sequentially.
problem Catastrophic forgetting in sequential learning tasks.
method Sparse inducing point approximations and auto-regressive variational distribution.
result VAR-GPs prevent catastrophic forgetting and outperform baselines.
VEST automates feature engineering for time series forecasting.
problem Challenges in time series forecasting with improved performance.
method VEST combines auto-regression with statistical summarization of recent past dynamics.
result VEST significantly improves forecasting performance.
Auto-regressive models improve smoothing efficiency with exponentially tapered windows.
problem Improving time-series smoothing efficiency.
method An auto-regressive formulation for time-series smoothing.
result Auto-regressive models result in moving means with exponentially tapered windows.
Paper reduces vocabulary losslessly for language model cooperation.
problem Language models struggle to cooperate with different tokenizations.
method Established a theoretical framework for lossless vocabulary reduction.
result Efficiently converts models with different tokenizations to cooperate with maximal common vocabulary.
TraDE uses self-attention for better density estimation of tabular and image data.
problem Improving density estimation for tabular and image data.
method Self-attention-based architecture trained with a penalized maximum likelihood objective.
result TraDE produces significantly better density estimates than existing methods.
Proof of Gaussian ML estimator consistency in linear auto-regressive models.
problem Consistency of Gaussian maximum likelihood estimator in linear auto-regressive models.
method Information-theoretic proof without stability assumptions.
result Nearly optimal non-asymptotic rates for parameter recovery.
DSARF models complex spatio-temporal data with deep switching auto-regressive factors.
problem Forecasting complex spatio-temporal data with recurring patterns.
method Deep switching auto-regressive factorization (DSARF) with stochastic variational inference.
result DSARF outperforms state-of-the-art methods in long- and short-term prediction accuracy.
A new RG approach connects discrete and continuous time descriptions of Gaussian processes.
problem Discretization of continuous stochastic processes for accurate simulation or model inference.
method Renormalization Group (RG) approach for Gaussian time series generated by auto-regressive models.
result RG fixed points correspond to discretizations of linear SDEs, providing insights into process accuracy.
Auto-regressive models learn latent states from partially observed linear dynamical systems.
problem Understanding how auto-regressive models learn latent representations from partially observed linear dynamical systems.
method Empirical risk minimization on partially observed linear dynamical systems.
result Two-layer linear auto-regressive models learn to approximate Kalman filtering, coinciding with optimal state estimates.
Enhanced multi-fidelity models improve digital twin accuracy and uncertainty quantification.
problem Lack of detailed application-specific data and inaccurate sensor data hinder surrogate model learning for digital twins.
method Proposes a multi-fidelity surrogate model framework integrating PCFE and GP, and deep-HPCFE with auto-regression schemes.
result Demonstrates improved accuracy and uncertainty quantification in digital twin systems.
Improved language models by incorporating statistical discriminators.
problem Distinguishing model-generated text from real text reliably.
method Energy-Based Model framework to incorporate discriminators.
result Improves language model performance in perplexity and human evaluation.
In this paper, non-linear time series models are used to describe volatility in financial time series data. To describe volatility, two of the non-linear time series are combined into form TAR (Threshold Auto-Regressive Model) with AARCH (Asymmetric Auto-Regressive Conditional Heteroskedasticity) error term and its par…
Cactus improves auto-regressive decoding speed without sacrificing quality.
problem Accelerating auto-regressive decoding while maintaining output quality.
method Formalizes speculative sampling as constrained optimization and proposes Cactus for controlled divergence from the verifier distribution.
result Empirically validated effectiveness across various benchmarks.
Study error bounds and optimal schedules for Masked Diffusions with factorized approximations.
problem Analyzing trade-offs between computation and accuracy in Masked Diffusion Models.
method Provided general error bounds and identified optimal schedules based on data distribution information profiles.
result Identified optimal schedule sizes for Masked Diffusion Models.
Transformer adapts to graphs with adaptive attention and auto-regressive decoding.
problem Transformers struggle with graph data due to non-sequential nature.
method Proposes GRAT, a Transformer variant with adaptive attention and auto-regressive decoding.
result GRAT achieves state-of-the-art performance on molecule property predictions and generation tasks.
The paper proposes a new auto-regressive model for multivariate distributional time series.
problem Statistical analysis of multivariate time series of probability measures.
method Wasserstein space, auto-regressive model, iterated random function systems.
result Consistent estimator for auto-regressive coefficients with sparse structure.
BMBO-DARN optimizes expensive functions with varying fidelities.
problem Optimizing expensive, multi-fidelity functions efficiently.
method Batch Multi-fidelity Bayesian Optimization with Deep Auto-Regressive Networks.
result BMBO-DARN improves surrogate learning and optimization performance.
Improved text generation with constraints using discrete auto-regressive biasing.
problem Balancing fluency and constraint satisfaction in LLM outputs.
method Discrete Auto-regressive Biasing, leveraging gradients in discrete text space.
result Significantly improved constraint satisfaction with comparable fluency.
We present a new classification method for quasar identification in the EROS-2 and MACHO datasets based on a boosted version of Random Forest classifier. We use a set of variability features including parameters of a continuous auto regressive model. We prove that continuous auto regressive parameters are very importan…
In this paper, we introduce the use of a personalized Gaussian Process model (pGP) to predict the key metrics of Alzheimer's Disease progression (MMSE, ADAS-Cog13, CDRSB and CS) based on each patient's previous visits. We start by learning a population-level model using multi-modal data from previously seen patients us…
New methods correct for time dependencies in IV regression for time series data.
problem Inferring causal effects from time series data with unobserved confounders.
method Proposes new methods for consistent estimation of causal effects in time series models using nuisance covariates and graph marginalization.
result Identifies and corrects for dependencies in the past, leading to consistent estimation of causal effects.
A new method estimates expectations from subtractive mixture models without sampling.
problem Estimating expectations from multimodal distributions using SMMs.
method Difference representation of SMMs to create unbiased IS estimator (ΔextEx). result Demonstrates that ΔextEx can achieve comparable estimation quality to auto-regressive sampling but is faster. Consider a multi-variate time series (Xt)t=0T where Xt∈Rd which may represent spike train responses for multiple neurons in a brain, crime event data across multiple regions, and many others. An important challenge associated with these time series models is to estimate an influence network be…
AR-Net models time-series with interpretable coefficients and scalability.
problem Modeling time-series with long-range dependencies and interpretability.
method Feed-forward neural network approach to AR-process dynamics.
result AR-Net learns identical AR-coefficients as Classic-AR and scales to long-range dependencies.
MDMs train to decode tokens in a random order, which affects performance; we show they can be optimized for a favorable order.
problem Performance of MDMs is affected by the random order in which tokens are decoded.
method We show that MDMs can be optimized for a favorable decoding order by equipping their continuous-time variational objective with multivariate noise schedules.
result MDMs can be decomposed into a weighted auto-regressive losses over orders, making them auto-regressive models with learnable orders.
Auto-regressive diffusion models improve capturing conditional dependence in data.
problem Vanilla diffusion models struggle to capture important, high-level relationships in real-world data.
method Developed auto-regressive diffusion models to better capture conditional dependence structures.
result AR diffusion models produce samples with a reduced gap in approximating the data conditional distribution.
A new method improves text generation quality and diversity.
problem Exposure bias in Maximum Likelihood Estimation for text generation.
method ψ-MLE, a new training scheme based on density ratio estimation.
result ψ-MLE outperforms Maximum Likelihood Estimation and other models in text generation quality and diversity.
Improves naturalness in TTS samples using quantized VAE and auto-regressive prosody.
problem Discontinuous and unnatural speech from standard VAE priors.
method Discretized latent features using vector quantization (VQ), and separately trained autoregressive (AR) prior model.
result Significantly improves naturalness in random sample generation.
Modeling the evolution of a financial index as a stochastic process is a problem awaiting a full, satisfactory solution since it was first formulated by Bachelier in 1900. Here it is shown that the scaling with time of the return probability density function sampled from the historical series suggests a successful mode…
Bayesian inference for stochastic differential equations using Wishart diffusions.
problem Inferring stochastic differential equations for regression and dynamical modeling.
method Bayesian non-parametric approach with semi-parametric Wishart processes.
result Modeling diffusion in stochastic differential equations improves performance and avoids overfitting.
Hybrid model combines VAR and neural network for OFI prediction.
problem Accurate prediction of Order Flow Imbalance (OFI) in high frequency trading.
method Combines Vector Auto Regression (VAR) and a simple feedforward neural network (FNN).
result Hybrid model achieves superior predictive accuracy compared to standalone models.
This work investigates how multi-round reasoning improves LLM performance.
problem Improving problem-solving abilities in complex tasks with LLMs.
method Investigates approximation, learnability, and generalization properties of multi-round auto-regressive models.
result Transformers with finite context windows are universal approximators for Turing-computable functions and can approximate any Turing-computable sequence-to-sequence function through multi-round reasoning.
We introduce an auto-regressive model which captures the growing nature of realistic markets. In our model agents do not trade with other agents, they interact indirectly only through a market. Change of their wealth depends, linearly on how much they invest, and stochastically on how much they gain from the noisy mark…
Numerous empirical proofs indicate the adequacy of the time discrete auto-regressive stochastic volatility models introduced by Taylor in the description of the log-returns of financial assets. The pricing and hedging of contingent products that use these models for their underlying assets is a non-trivial exercise due…
Auto-regressive conditionally heteroskedastic (ARCH) family models are still used, by practitioners in business and economic policy making, as a conditional volatility forecasting models. Furthermore ARCH models still are attracting an interest of the researchers. In this contribution we consider the well known GARCH(1…
We consider the problem of estimating the parameters of a multivariate Bernoulli process with auto-regressive feedback in the high-dimensional setting where the number of samples available is much less than the number of parameters. This problem arises in learning interconnections of networks of dynamical systems with …
New approach predicts tokens in context, explaining how ICL emerges.
problem Limited understanding of in-context learning emergence.
method Auto-regressive next-token prediction (AR-NTP) with prompt token-dependency and a two-level expectation.
result ICL emerges from the generalization of sequences and topics.
Variational autoencoders (VAEs) with an auto-regressive decoder have been applied for many natural language processing (NLP) tasks. The VAE objective consists of two terms, (i) reconstruction and (ii) KL regularization, balanced by a weighting hyper-parameter β. One notorious training difficulty is that the KL term ten…
Autoregressive feedback is considered a necessity for successful unconditional text generation using stochastic sequence models. However, such feedback is known to introduce systematic biases into the training process and it obscures a principle of generation: committing to global information and forgetting local nuanc…
Propagating input uncertainty through non-linear Gaussian process (GP) mappings is intractable. This hinders the task of training GPs using uncertain and partially observed inputs. In this paper we refer to this task as "semi-described learning". We then introduce a GP framework that solves both, the semi-described and…
Probabilistic forecasting, i.e. estimating the probability distribution of a time series' future given its past, is a key enabler for optimizing business processes. In retail businesses, for example, forecasting demand is crucial for having the right inventory available at the right time at the right place. In this pap…
Generative models speed up complex system simulations.
problem Accurately forecasting the dynamics of complex systems at reduced cost.
method Generative Learning of Effective Dynamics (G-LED) using auto-regressive attention and Bayesian diffusion models.
result Generative models can accurately forecast complex system dynamics at lower computational cost.
A robust algorithm for forecasting vector time series with seasonal components.
problem Forecasting vector time series with seasonal patterns and handling missing data.
method Auto-regression with seasonal annual, weekly, and daily baselines, and a Gaussian process for residuals. Custom truncated eigendecomposition and low-rank plus block-diagonal Gaussian kernel. Schur complement and Tikhonov regularization for efficient inference.
result The model can scale to very large datasets and is efficient in terms of memory and computation.
New Feedback Transformer architecture improves model performance by exposing past representations to future.
problem Limitations of Transformers in fully exploiting sequential input.
method Proposes Feedback Transformer exposing all past representations to future.
result Demonstrates improved performance with smaller, shallower models.
The paper compares advanced deep learning models for Indian stock price forecasting.
problem Complexity of stock price forecasting due to numerous influencing factors.
method Utilizes historical data from national banks in India, combines deep learning models and sentiment analysis.
result Achieved higher accuracy in stock price forecasting compared to traditional methods.
Consider observing a collection of discrete events within a network that reflect how network nodes influence one another. Such data are common in spike trains recorded from biological neural networks, interactions within a social network, and a variety of other settings. Data of this form may be modeled as self-excitin…