The framework of normalizing flows provides a general strategy for flexible variational inference of posteriors over latent variables. We propose a new type of normalizing flow, inverse autoregressive flow (IAF), that, in contrast to earlier published flows, scales well to high-dimensional latent spaces. The proposed f…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “variational autoregressive networks”
Statistical Mechanics Calculations Using Variational Autoregressive Networks and Quantum Annealingcond-mat.dis-nn
New method uses quantum annealing and VAN for better statistical mechanics calculations.
problem Difficulty in computing partition function in statistical mechanics.
method Combines quantum annealing samples with variational autoregressive networks.
result Enhanced accuracy in finite-size Sherrington-Kirkpatrick model.
Solving Statistical Mechanics Using Variational Autoregressive Networkscond-mat.stat-mech
We propose a general framework for solving statistical mechanics of systems with finite size. The approach extends the celebrated variational mean-field approaches using autoregressive neural networks, which support direct sampling and exact calculation of normalized probability of configurations. It computes variation…
VB approach for dynamic network models improves efficiency and accuracy.
problem Estimating dynamic network models in large-scale systems.
method Variational Bayesian inference for network autoregression.
result VB approach detects proper active structures and achieves similar or better accuracy.
Develops a new model for network estimation from multi-variate data.
problem Network estimation from multi-variate point process or time series data.
method Semi-parametric approach based on the monotone single-index multi-variate autoregressive model (SIMAM).
result Achieves optimal rates of convergence and superior performance in prediction and network estimation.
Solving Statistical Mechanics on Sparse Graphs with Feedback Set Variational Autoregressive Networkscond-mat.stat-mech
We propose a method for solving statistical mechanics problems defined on sparse graphs. It extracts a small Feedback Vertex Set (FVS) from the sparse graph, converting the sparse system to a much smaller system with many-body and dense interactions with an effective energy on every configuration of the FVS, then learn…
Parallelizes autoregressive generation using VSSM.
problem Autoregressive models' inability to parallelize generation.
method Variational SSM (VSSM) with parallelizable sampling and decoding.
result Parallel generation possible with VSSM.
Normalizing flows and autoregressive models have been successfully combined to produce state-of-the-art results in density estimation, via Masked Autoregressive Flows (MAF), and to accelerate state-of-the-art WaveNet-based speech synthesis to 20x faster than real-time, via Inverse Autoregressive Flows (IAF). We unify a…
This review compares various deep generative models.
problem Training deep neural networks to model data distributions.
method Comprehensive comparison of VAEs, GANs, flows, energy models, and autoregressives.
result Trade-offs and interrelationships among different models.
We introduce a deep, generative autoencoder capable of learning hierarchies of distributed representations from data. Successive deep stochastic hidden layers are equipped with autoregressive connections, which enable the model to be sampled from quickly and exactly via ancestral sampling. We derive an efficient approx…
Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models such as variational autoencoders (VAE), recent empirical work suggests that inference networks can produce suboptimal variational parameter…
Generating graph structures is a challenging problem due to the diverse representations and complex dependencies among nodes. In this paper, we introduce Graph Variational Recurrent Neural Network (GraphVRNN), a probabilistic autoregressive model for graph generation. Through modeling the latent variables of graph data…
A new model trains prior and encoder/decoder networks simultaneously for efficient generation.
problem Complex autoregressive prior in VQ-VAE models leads to slow generation.
method Builds a diffusion bridge between continuous and non-informative prior distributions.
result Model is competitive and efficient in optimization and sampling.
SHVC improves image compression with fewer parameters.
problem Challenges in VAE compression, especially with bits-back coding.
method Introduces autoregressive sub-pixel convolution and autoregressive initial bits.
result Achieves state-of-the-art compression performance with fewer model parameters.
A new method for separating mixed signals in space and time.
problem Nonlinear and nonstationary spatio-temporal data challenges.
method Identifiable autoregressive variational autoencoder.
result The method outperforms existing techniques in blind source separation and spatio-temporal prediction.
Variational Causal Networks approximate Bayesian inference over causal structures.
problem Quantifying uncertainty in causal structure inference from finite data.
method Parametric variational family over DAGs, using Evidence Lower Bound (ELBO) for tractable learning.
result Approximation of the true posterior over DAGs is demonstrated to be good.
ARTree uses deep learning to infer tree topologies efficiently.
problem Efficient phylogenetic inference from tree topologies.
method Deep autoregressive model based on graph neural networks (GNNs).
result ARTree provides a flexible family of distributions over tree topologies.
Monotonic neural networks have recently been proposed as a way to define invertible transformations. These transformations can be combined into powerful autoregressive flows that have been shown to be universal approximators of continuous probability distributions. Architectures that ensure monotonicity typically enfor…
A new autoregressive model learns the order of graph generation tasks.
problem Generating graphs in a meaningful order when the canonical order is not obvious.
method Introduces a variant of autoregressive models that dynamically decides the autoregressive order based on data.
result Achieves state-of-the-art results on molecular graph generation benchmarks.
VHVM models financial time series with varying volatility.
problem Modeling heteroscedastic behavior in multivariate financial time series.
method Variational autoencoder and recurrent neural network for capturing relationships and temporal dynamics.
result VHVM outperforms GARCH and SV models on FX datasets.
Diffusion models generate music sequences without autoregressive loops.
problem Generating music sequences from symbolic data using diffusion models.
method Parameterize discrete symbolic data in continuous latent space, train diffusion model, generate sequences through reverse process.
result Strong unconditional generation and post-hoc conditional infilling compared to autoregressive models.
Highly expressive directed latent variable models, such as sigmoid belief networks, are difficult to train on large datasets because exact inference in them is intractable and none of the approximate inference methods that have been applied to them scale well. We propose a fast non-iterative approximate inference metho…
LTMs use latent vectors for efficient autoregressive generation.
problem Efficient autoregressive generation in language models.
method Dual-rate optimization in variational Bayes framework.
result LTMs achieve superior sample and parameter efficiency.
Flow-based generative models are powerful exact likelihood models with efficient sampling and inference. Despite their computational efficiency, flow-based models generally have much worse density modeling performance compared to state-of-the-art autoregressive models. In this paper, we investigate and improve upon thr…
Improves sampling speed of ARM models without losing accuracy.
problem Slow sampling from ARMs.
method Predictive sampling algorithm using ARM fixed-point iteration and learned forecasting modules.
result Significant improvements in sampling speed and efficiency.
We introduce the use of autoregressive normalizing flows for rapid likelihood-free inference of binary black hole system parameters from gravitational-wave data with deep neural networks. A normalizing flow is an invertible mapping on a sample space that can be used to induce a transformation from a simple probability …
New model generates graphs with tighter likelihood bounds and better quality.
problem Intractable likelihood of autoregressive graph models.
method Derive exact joint probability, approximate node orderings, variational inference.
result Lower bound on log-likelihood is significantly tighter than previous methods.
Estimates TV distance between autoregressive models under different access models.
problem Estimating the total variation distance between two autoregressive distributions.
method Three access models: sample access, logit access, and noisy logit access; provides query complexity for each.
result Improved query complexity for estimating TV distance in autoregressive models.
AR-Flow VAE improves blind source separation with flexible autoregressive priors.
problem Unsupervised blind source separation of latent signals from mixtures.
method AR-Flow VAE uses autoregressive flows to model latent sources, enhancing flexibility and capturing complex dependencies.
result AR-Flow VAE effectively separates latent sources, demonstrating improved performance over conventional methods.
Self-reflective VAE improves inference and generative modeling without complex components.
problem Limitations of typical VAEs in inference and generative modeling.
method Introduces self-reflective inference, a new hierarchical structure that matches variational posterior to exact posterior.
result Self-reflective inference achieves state-of-the-art performance on binarized MNIST without autoregressive layers.
NVAE improves VAE performance on large image datasets.
problem Improving variational autoencoder performance for large image datasets.
method Deep hierarchical VAE with depth-wise separable convolutions and batch normalization, residual parameterization of Normal distributions, and spectral regularization.
result NVAE achieves state-of-the-art results on MNIST, CIFAR-10, CelebA 64, and CelebA HQ datasets.
Autoregressive networks can achieve promising performance in many sequence modeling tasks with short-range dependence. However, when handling high-dimensional inputs and outputs, the huge amount of parameters in the network lead to expensive computational cost and low learning efficiency. The problem can be alleviated …
We explore the use of Vector Quantized Variational AutoEncoder (VQ-VAE) models for large scale image generation. To this end, we scale and enhance the autoregressive priors used in VQ-VAE to generate synthetic samples of much higher coherence and fidelity than possible before. We use simple feed-forward encoder and dec…
New MIF architecture improves posterior approximations in Bayesian models.
problem Challenges in variational inference for complex hierarchical models.
method Combines VIP and autoregressive flow with prior information and hierarchical ordering.
result Empirically, MIF delivers tighter posterior approximations and state-of-the-art performance.
A new neural network for efficient density estimation.
problem Efficient density estimation for high-dimensional data.
method Triangular neural network implementation of neural autoregressive flow (NAF).
result Achieves state-of-the-art bits-per-dimension indices on MNIST and CIFAR-10.
Neural GARCH models financial time series with time-varying coefficients.
problem Modeling conditional heteroskedasticity in financial time series.
method Neural network adaptation of GARCH and BEKK models with time-varying coefficients parameterized by a recurrent neural network.
result Neural Students t model consistently outperforms other models on financial time series.
Vector autoregressive models characterize a variety of time series in which linear combinations of current and past observations can be used to accurately predict future observations. For instance, each element of an observation vector could correspond to a different node in a network, and the parameters of an autoregr…
Unbiased Monte Carlo Cluster Updates with Autoregressive Neural Networkscond-mat.stat-mech
New method makes machine learning approximations unbiased and efficient.
problem Efficient sampling of complex probability distributions.
method Uses autoregressive neural networks with cluster updates and physical symmetries.
result Shows unbiased and low-variance approximations for phase transitions.
A new method for filling in missing traffic data improves accuracy over existing techniques.
problem Incomplete spatiotemporal traffic data.
method Low-rank autoregressive tensor completion (LATC) framework.
result LATC framework better captures spatiotemporal consistency and local consistency.
Improving Variational Auto-Encoders using convex combination linear Inverse Autoregressive Flowstat.ML
In this paper, we propose a new volume-preserving flow and show that it performs similarly to the linear general normalizing flow. The idea is to enrich a linear Inverse Autoregressive Flow by introducing multiple lower-triangular matrices with ones on the diagonal and combining them using a convex combination. In the …
This paper introduces Associative Compression Networks (ACNs), a new framework for variational autoencoding with neural networks. The system differs from existing variational autoencoders (VAEs) in that the prior distribution used to model each code is conditioned on a similar code from the dataset. In compression term…
Estimating hidden processes from non-linear noisy observations is particularly difficult when the parameters of these processes are not known. This paper adopts a machine learning approach to devise variational Bayesian inference for such scenarios. In particular, a random process generated by the autoregressive moving…
Variational inference relies on flexible approximate posterior distributions. Normalizing flows provide a general recipe to construct flexible variational posteriors. We introduce Sylvester normalizing flows, which can be seen as a generalization of planar flows. Sylvester normalizing flows remove the well-known single…
ARMA cell simplifies neural autoregressive modeling for time series.
problem Complex RNN cells are not always necessary and can be inferior.
method Introduces ARMA cell, a simpler, modular approach for neural time series modeling.
result The ARMA cell is competitive with popular alternatives in performance.
Mutual information of spin systems from autoregressive neural networkscond-mat.stat-mech
New method estimates spin system mutual information using neural networks.
problem Estimating mutual information in spin systems.
method Monte Carlo sampling enhanced by autoregressive neural networks.
result Area law satisfied for temperatures away from critical temperature.
Global information is essential for dense prediction problems, whose goal is to compute a discrete or continuous label for each pixel in the images. Traditional convolutional layers in neural networks, initially designed for image classification, are restrictive in these problems since the filter size limits their rece…
Comment on "Solving Statistical Mechanics Using VANs": Introducing saVANt - VANs Enhanced by Importance and MCMC Samplingcond-mat.stat-mech
In this comment on "Solving Statistical Mechanics Using Variational Autoregressive Networks" by Wu et al., we propose a subtle yet powerful modification of their approach. We show that the inherent sampling error of their method can be corrected by using neural network-based MCMC or importance sampling which leads to a…
Neural Spline Flowsstat.ML
A normalizing flow models a complex probability density as an invertible transformation of a simple base density. Flows based on either coupling or autoregressive transforms both offer exact density evaluation and sampling, but rely on the parameterization of an easily invertible elementwise transformation, whose choic…