Paper proposes a robust variational inference method for noisy data.
problem Variational inference struggles with random noise in training data.
method Develops a new robust evidence lower bound for variational autoencoders.
result The new objective outperforms the standard evidence lower bound on noisy datasets.
The paper introduces validated variational inference with practical error bounds.
problem Lack of accurate post-hoc measures for variational inference.
method The paper provides rigorous bounds on variational inference error.
result The bounds are widely applicable and computationally efficient.
New method improves variational inference for hierarchical models.
problem Limited expressivity of variational distributions in Bayesian models.
method Importance weighted hierarchical variational inference.
result Superior performance in experiments compared to existing methods.
Unified variational bounds for mutual information, addressing high-dimensional challenges.
problem Estimating and optimizing Mutual Information (MI) in high dimensions is challenging.
method Unified framework of variational lower bounds parameterized by neural networks, trading off bias and variance.
result Unified bounds flexibly trade off bias and variance, improving estimation and representation learning.
Researchers develop variational bounds for perceptron learning from structured data.
problem Learning from structured data with concave utilities and log-concave priors.
method Variational approach combining interpolation method, log-concavity, and concentration estimates.
result Lower and upper minimax variational bounds match, identifying the solution of the model.
We introduce the Variational Holder (VH) bound as an alternative to Variational Bayes (VB) for approximate Bayesian inference. Unlike VB which typically involves maximization of a non-convex lower bound with respect to the variational parameters, the VH bound involves minimization of a convex upper bound to the intract…
DSIVI improves variational autoencoders by optimizing a proper lower bound on ELBO.
problem Improving variational autoencoders with implicit priors.
method Introducing DSIVI, a method that optimizes a proper lower bound on ELBO for models with semi-implicit priors and posteriors.
result DSIVI improves the performance of VampPrior, a state-of-the-art prior for variational autoencoders.
Paper improves variational inference by tightening bounds using perturbation theory.
problem Improving variational inference's bias and KL divergence approximation.
method Revisits perturbation theory to derive corrections that tighten variational bounds.
result New bounds are tighter and more mass-covering, leading to higher likelihoods.
A new upper bound for variational inference improves the efficiency of Bayesian deep learning.
problem Improving variational inference in Bayesian deep learning.
method Presented a new upper bound (EUBO) for evidence, derived from KL-divergence and log marginal likelihood, and used SGD for optimization.
result The new upper bound (EUBO) is tighter than previous methods and outperforms state-of-the-art results in Bayesian neural networks.
Reinterprets IWAE as optimizing a tighter variational bound.
problem Improving variational lower bounds in autoencoders.
method Derives and visualizes a tighter lower bound using a more complex distribution.
result IWAE optimizes the standard variational lower bound with a complex distribution.
Improved variational inference for logistic regression and classification.
problem Intractability of Evidence Lower Bound in variational logistic regression.
method Introducing a new bound for the expectation of softplus function, applied to variational logistic regression and Gaussian process classification.
result The new bound results in a tighter posterior and faster computation compared to Monte-Carlo methods.
This work improves VAEs using MCMC methods for better variational bounds.
problem Improving the expressiveness of variational distributions in VAEs.
method Entropy-based adaptation for MALA/HMC chains to optimize tighter variational bounds.
result Higher held-out log-likelihoods and improved generative metrics.
This tutorial derives the VAE loss function under Gaussian assumptions.
problem Computational intractability of posterior distributions in Bayesian machine learning.
method Derives the variational lower bound loss function of a standard VAE.
result The Kullback-Leibler divergence has a closed form solution under Gaussian assumptions.
Paper provides variance bounds for variational inference.
problem Understanding the variance of stochastic gradient estimators in variational inference.
method Analyzes reparameterization estimators under smoothness and location-scale assumptions.
result Gives provable bounds on gradient variance, showing they are optimal under stated conditions.
Semi-Implicit Variational Inference broadens variational distributions.
problem Expanding the variational family to include implicit distributions.
method Mixing variational parameters with a flexible distribution.
result SIVI provides an asymptotically exact surrogate ELBO for optimization.
Paper formalizes and analyzes a new bound for variational inference.
problem Lack of theoretical guarantees in variational algorithms.
method Introduces VR-IWAE bound, a generalization of IWAE.
result VR-IWAE bound leads to unbiased gradient estimators.
TVO tightens variational inference bounds for deep models.
problem Improving variational inference bounds for deep models.
method Introduces thermodynamic variational objective (TVO) connecting variational inference and thermodynamic integration.
result TVO provides tighter lower bound to log marginal likelihood than ELBO.
Paper establishes lower bounds for non-stationary kernelized bandits.
problem Optimizing functions with noisy observations in non-stationary scenarios.
method Develops algorithm-independent lower bounds for time-varying functions under total variation constraints.
result First algorithm-independent lower bounds for time-varying kernelized bandits.
New method quantifies and bounds changes in non-stationary optimization problems.
problem Optimizing under changing cost functions with limited variation.
method Proposes Lp,q-variation functional for quantifying changes and derives regret bounds. result Upper and lower bounds for smooth and strongly convex functions, matching optimal rates.
Variational Prediction simplifies Bayesian inference without test time costs.
problem Bayesian inference's computational costs and posterior predictive distribution marginalization.
method Variational Prediction learns a variational approximation to the posterior predictive distribution using a variational bound.
result Directly learns a variational approximation to the posterior predictive distribution without test time marginalization costs.
This work provides guaranteed bounds on the total variation distance for univariate mixtures.
problem Lack of closed-form expressions for total variation distance between mixtures.
method Two methods: information monotonicity for lower bounds and geometric envelopes for upper bounds.
result Demonstrated tightness of bounds on Gaussian, Gamma, and Rayleigh mixtures.
This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is co…
A new method simplifies variational inference for complex models.
problem Challenges in exact Bayesian inference due to intractable integrals.
method Rewriting the lower bound on model log-likelihood using a finite sample of Gaussian latent variables.
result Demonstrated effectiveness on synthetic and real-world examples.
The paper analyzes variational autoencoders for state space models with risk bounds.
problem Analyzing the risk associated with variational autoencoders for state space models.
method Backward factorization of variational distributions to analyze excess risk, providing oracle inequalities and upper bounds.
result Explicit upper bounds on variational estimation error for state space models under strong mixing assumptions.
This research explores using Alpha-Divergences in variational dropout for better inference.
problem Improving variational inference methods using alternative divergences.
method Extending the Stochastic Gradient Variational Bayes (SGVB) framework with Alpha-Divergences.
result The α-divergence with αightarrow1 yields the lowest training error and optimizes the ELBO. New algorithm uses control variates to improve multi-armed bandit performance.
problem Stochastic multi-armed bandits with auxiliary reward information.
method Developed UCB-CV algorithm using control variates for mean estimation.
result UCB-CV algorithm provides tighter confidence bounds and smaller variance.
New variational bounds improve posterior covariances and likelihoods.
problem Improving variational inference with different divergence measures.
method Applying variational perturbation theory to construct new variational bounds.
result New variational bounds lead to more accurate posterior covariances and higher likelihoods.
Paper finds a lower bound for estimating low-rank matrices in logistic regression.
problem Estimating low-rank coefficient matrices in logistic regression.
method Derives a minimax lower bound on the risk.
result The bound depends on matrix dimensions, rank, and sample size.
Uniform linear bounds on volume changes in 3D hyperbolic spaces.
problem Volume variation in hyperbolic 3-manifolds.
method Uniform linear bounds proof for drilling and filling operations.
result Uniform linear bounds on volume variation proved.
Closed-form variational objectives for Bayesian neural networks with ReLU layers.
problem Efficient computation of Bayesian neural networks with closed-form variational objectives.
method Single-layer networks with piecewise polynomial activations (ReLU). Structured Normal variational distributions for Normal likelihoods. Approximate lower bounds for other likelihoods.
result Closed-form computation of variational lower bounds, predictive mean, and variance for Bayesian neural networks.
New algorithm limits regret in changing MDPs.
problem Reinforcement learning in MDPs with time-varying rewards and transitions.
method Proposed an algorithm with performance guarantees for non-stationary policies.
result First variational regret bound for general RL setting.
Lower bound for VAE training objective for binary data.
problem Finding a lower bound for the ELBO of Bernoulli VAE.
method Interpretable lower bound, modified initialization, faster training architecture, PCA for latent space dimension.
result Theoretical result and improved performance of new architecture.
Paper improves variance control in importance weighted variational bounds.
problem Improving the variance of gradient estimators for IWAE.
method Develops a novel control variate that grows SNR as √K for large K.
result Empirically, the method yields superior variance reduction for generative models.
New guarantees for black-box variational inference methods.
problem Insufficient theoretical guarantees for black-box variational inference.
method Novel convergence guarantees for stochastic optimization of variational inference.
result Provable convergence of proximal and projected stochastic gradient descent for variational inference.
New insights into variational inference using Monte Carlo estimates.
problem Improving variational bounds in latent variable models.
method Analyzing properties of Monte Carlo estimates and their impact on variational gaps.
result Negative correlation reduces variational gaps, contrary to intuition.
The paper studies curves in Riemannian manifolds using total variation flow.
problem Analyzing the evolution of curves in Riemannian manifolds using total variation.
method Defining and proving the existence of strong solutions to the flow equations, showing variational equality, and proving convergence.
result Strong solutions converge to a constant map in finite time for non-positive sectional curvature.
We present a general method for deriving collapsed variational inference algo- rithms for probabilistic models in the conjugate exponential family. Our method unifies many existing approaches to collapsed variational inference. Our collapsed variational inference leads to a new lower bound on the marginal likelihood. W…
New method improves variational bounds on partition function.
problem Computing the partition function of discrete graphical models is intractable.
method Combines gauge transformations with weighted mini-bucket elimination (WMBE).
result WMBE-G strictly improves earlier WMBE approximation for symmetric models.
Develops a new unsupervised clustering method using Variational Information Bottleneck and Gaussian Mixture Model.
problem Unsupervised clustering of unlabeled data.
method Combines Variational Information Bottleneck and Gaussian Mixture Model in a deep neural network framework.
result Derives a new bound on the cost function and provides an algorithm for efficient computation.
New method improves GP regression by relaxing variational assumption.
problem Improving variational Gaussian processes for better predictive performance.
method Relaxing the variational assumption to a more general distribution for optimization.
result New tighter evidence lower bound for GP regression.
New bounds enable training of probabilistic models for deep networks.
problem Training scalable latent variable models for deep networks.
method Introducing new variational bounds for specific output layers of neural networks.
result Analytical bounds for certain output layers allow training without re-parameterization or Monte Carlo approximations.
The paper analyzes the error in variational Bayesian NMF compared to Bayesian NMF.
problem Analyzing the variational approximation error in Bayesian NMF.
method Using algebraic geometrical methods, the paper derives an upper bound for the learning coefficient and a lower bound for the approximation error.
result The paper finds a lower bound for the approximation error, showing how well VBNMF approximates Bayesian NMF.
Mean-field variational inference is a method for approximate Bayesian posterior inference. It approximates a full posterior distribution with a factorized set of distributions by maximizing a lower bound on the marginal likelihood. This requires the ability to integrate a sum of terms in the log joint likelihood using …
New method improves inference for hierarchical models.
problem Challenges in inference for large hierarchical models.
method Locally enhanced variational bounds with subsampling.
result Better posterior approximations than baselines.
Deep Gaussian processes provide a flexible approach to probabilistic modelling of data using either supervised or unsupervised learning. For tractable inference approximations to the marginal likelihood of the model must be made. The original approach to approximate inference in these models used variational compressio…
Study introduces indecomposability for varifolds, leading to geometric consequences.
problem Understanding the structure of varifolds and their connectedness properties.
method Introducing indecomposability and related concepts for varifolds.
result Substantial geometric consequences derived from the connectedness properties of varifolds.
EVI improves variational inference for non-Gaussian models.
problem Infeasibility of finding analytically tractable solutions for non-Gaussian statistical models.
method Extended Variational Inference (EVI) using lower-bound approximation to the variational objective function.
result Convergence of EVI depends on lower-bound approximation strategy.
Recent research has made significant progress on the problem of bounding log partition functions for exponential family graphical models. Such bounds have associated dual parameters that are often used as heuristic estimates of the marginal probabilities required in inference and learning. However these variational est…