A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This paper compares gradient estimators in importance-weighted VI and justifies the superiority of DREP over REP.
Paper improves variance control in importance weighted variational bounds.
The standard interpretation of importance-weighted autoencoders is that they maximize a tighter lower bound on the marginal likelihood than the standard evidence lower bound. We give an alternate interpretation of this procedure: that it optimizes the standard variational lower bound, but using a more complex distribut…
Paper formalizes and analyzes a new bound for variational inference.
Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI) technique is an instance of augmented variational inference, thus identifying th…
New insights into variational inference using Monte Carlo estimates.
New methods improve gradient estimation in autoencoders, enhancing generative network performance.
Combines control variates and adaptive importance sampling for Monte Carlo integration.
We provide theoretical and empirical evidence that using tighter evidence lower bounds (ELBOs) can be detrimental to the process of learning an inference network by reducing the signal-to-noise ratio of the gradient estimator. Our results call into question common implicit assumptions that tighter ELBOs are better vari…
Boltzmann machines are powerful distributions that have been shown to be an effective prior over binary latent variables in variational autoencoders (VAEs). However, previous methods for training discrete VAEs have used the evidence lower bound and not the tighter importance-weighted bound. We propose two approaches fo…
New method improves inference for hierarchical models.
Importance weighted variational inference (Burda et al., 2015) uses multiple i.i.d. samples to have a tighter variational lower bound. We believe a joint proposal has the potential of reducing the number of redundant samples, and introduce a hierarchical structure to induce correlation. The hope is that the proposals w…
Paper improves REINFORCE for VI without restrictive assumptions.
The recognition network in deep latent variable models such as variational autoencoders (VAEs) relies on amortized inference for efficient posterior approximation that can scale up to large datasets. However, this technique has also been demonstrated to select suboptimal variational parameters, often resulting in consi…
Adapts score matching for missing data in flexible settings.
U-statistics improve gradient estimation in importance-weighted variational inference.
Improved variational inference for GPLVMs using AIS.
Paper introduces VDE, a variance-reduced determinant estimator.
Variational Inference is a powerful tool in the Bayesian modeling toolkit, however, its effectiveness is determined by the expressivity of the utilized variational distributions in terms of their ability to match the true posterior distribution. In turn, the expressivity of the variational family is largely limited by …
We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that corresponds to the score function. Removing this term produces an unbiased gradient …
The Importance Weighted Auto Encoder (IWAE) objective has been shown to improve the training of generative models over the standard Variational Auto Encoder (VAE) objective. Here, we derive importance weighted extensions to AVB and AAE. These latent variable models use implicitly defined inference networks whose approx…
New Holder bounds improve variational inference by flattening thermodynamic curves.
Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces sub-optimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a funct…
Deep Gaussian processes (DGPs) can model complex marginal densities as well as complex mappings. Non-Gaussian marginals are essential for modelling real-world data, and can be generated from the DGP by incorporating uncorrelated variables to the model. Previous work on DGP models has introduced noise additively and use…
R2D2-Net improves Bayesian neural networks by preventing over-shrinkage of important weights.
Paper finds efficient OPE estimator for multiple logging policies with minimum variance.
This paper introduces a spline-based method for nonparametric ADVI that handles complex posterior distributions.
The variational autoencoder (VAE; Kingma, Welling (2014)) is a recently proposed generative model pairing a top-down generative network with a bottom-up recognition network which approximates posterior inference. It typically makes strong assumptions about posterior inference, for instance that the posterior distributi…
Multi-sample, importance-weighted variational autoencoders (IWAE) give tighter bounds and more accurate uncertainty estimates than variational autoencoders (VAE) trained with a standard single-sample objective. However, IWAEs scale poorly: as the latent dimensionality grows, they require exponentially many samples to r…
The importance weighted autoencoder (IWAE) (Burda et al., 2016) is a popular variational-inference method which achieves a tighter evidence bound (and hence a lower bias) than standard variational autoencoders by optimising a multi-sample objective, i.e. an objective that is expressible as an integral over Mont…
Develops a method for learning proposals in nested importance samplers.
VISA improves inference efficiency for complex models.
Paper proposes a new method for feature importance in model selection.
New algorithm improves accuracy of importance weights for diverse applications.
Unbiased gradient estimation improves VAE performance.
New model classes for function approximation by neural networks defined on domains.
We introduce overdispersed black-box variational inference, a method to reduce the variance of the Monte Carlo estimator of the gradient in black-box variational inference. Instead of taking samples from the variational distribution, we use importance sampling to take samples from an overdispersed distribution in the s…
Paper introduces f-divergence variational inference for broader application.
WR-CP reduces prediction set size and coverage gap under distribution shift.
RLFA estimates misstated monetary fraction with weighted sampling without replacement.
Unified framework for analyzing pessimism in off-policy learning with regularized importance sampling.
Efficiently estimates online variational learning using importance sampling.
Boosting variational inference (BVI) approximates an intractable probability density by iteratively building up a mixture of simple component distributions one at a time, using techniques from sparse convex optimization to provide both computational scalability and approximation error guarantees. But the guarantees hav…
Corrects distribution shift in target shift scenarios using importance weighting.
Paper improves VAEs using Monte Carlo methods.
New method learns complex, multimodal distributions in ADVI.
Variational inference with α-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using α-divergences (with positive α values) is their mass-covering property. However, estimating and optimizing α-divergences require to use importa…
DGPs with variational inference suffer from SNR issues that degrade gradient estimates, leading to unreliable training.