We introduce overdispersed black-box variational inference, a method to reduce the variance of the Monte Carlo estimator of the gradient in black-box variational inference. Instead of taking samples from the variational distribution, we use importance sampling to take samples from an overdispersed distribution in the s…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,291 papers · 148 categories
Trend · papers per month
48 results for “Black-Box Variational inference”
New method reduces inference variance for faster optimization.
problem High variance in black-box variational inference.
method Joint control variate addressing both data subsampling and Monte Carlo noise.
result Significantly reduced gradient variance, leading to faster optimization.
New guarantees for black-box variational inference methods.
problem Insufficient theoretical guarantees for black-box variational inference.
method Novel convergence guarantees for stochastic optimization of variational inference.
result Provable convergence of proximal and projected stochastic gradient descent for variational inference.
TrustVI is a fast second-order algorithm for black-box variational inference.
problem Efficiently optimizing variational distributions in complex models.
method Trust-region optimization with minibatch reparameterization.
result TrustVI converges faster and finds better variational distributions than alternatives.
Guarantees convergence for black-box variational inference without modifications.
problem Convergence guarantees for black-box variational inference.
method Analysis of log-smooth posterior densities, location-scale variational family, and convergence rates of algorithm design choices.
result Proximal stochastic gradient descent fixes suboptimal convergence rates and achieves strongest known guarantees.
Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriorscs.LG
DDVI uses diffusion models for variational inference, improving latent variable model performance.
problem Improving variational inference in latent variable models.
method Introduces diffusion-based variational posteriors trained with a regularized ELBO.
result Outperforms alternative variational posteriors on various benchmarks and a biology task.
The paper introduces structured variational families to improve scalability in black-box variational inference.
problem Scalability issues in black-box variational inference, especially for large datasets and hierarchical models.
method Developed structured variational families that achieve better iteration complexity of O(N) compared to full-rank families.
result Structured variational families can achieve better scaling with respect to dataset size N, improving iteration complexity from O(N^2) to O(N).
Variational inference has become a widely used method to approximate posteriors in complex latent variables models. However, deriving a variational inference algorithm generally requires significant model-specific analysis, and these efforts can hinder and deter us from quickly developing and exploring a variety of mod…
Proposes a method to stabilize Black Box Variational Inference using the James-Stein estimator.
problem Stability issues and fine-tuning required in basic Black Box Variational Inference.
method Reframe stochastic gradient ascent as multivariate estimation problem using James-Stein estimator.
result Provides a simpler method with consistent performance in terms of model fit and convergence time.
Improves scalability and efficiency of mixture models in black-box variational inference.
problem Scaling mixture models in black-box variational inference leads to high parameter and time costs.
method Introduces MISVAE for amortized mixture parameter space and new ELBO estimators.
result Achieves superior estimation performance with fewer parameters and shorter inference time.
New framework for variational coresets simplifies Bayesian inference for complex models.
problem Efficient Bayesian inference for complex models like neural networks.
method Black-box variational inference for coresets that handle intractable posterior distributions.
result Principled application of variational coresets to Bayesian neural networks.
This paper shows equivalence between SVGD and BBVI using kernel gradient flows.
problem Bayesian inference methods and their equivalence.
method Formalizes equivalence between SVGD and BBVI using kernel gradient flows.
result BBVI corresponds precisely to SVGD when using the neural tangent kernel.
Inference for SDEs using variational methods and neural networks.
problem Parameter inference for stochastic differential equations is challenging due to latent diffusion processes.
method Variational inference with a mean-field approximation for parameters and a recurrent neural network for diffusion paths.
result Accurate parameter estimates for SDE systems, demonstrated on Lotka-Volterra and epidemic models.
QBVI uses natural gradients for efficient Bayesian learning.
problem Efficient Bayesian learning in complex models.
method Natural gradient updates in a black-box framework for exponential-family distributions.
result QBVI framework is effective for a wide range of Bayesian inference problems.
Paper advances black-box VI using flows and Monte-Carlo methods.
problem Improving automatic posterior inference in black-box VI.
method Combines normalizing flows, Monte-Carlo methods, and optimization considerations.
result Significant improvement in state-of-the-art variational inference.
Variational boosting refines posterior approximations through iterative optimization.
problem Approximating intractable distributions with rich approximations.
method Iteratively solves optimization problems to refine variational approximations.
result Posterior inferences using variational boosting are more accurate and efficient.
VERA uses variational inference to jailbreak LLMs without manual optimization.
problem Lack of principled objective for gradient-based optimization in jailbreaking LLMs.
method VERA casts black-box jailbreak prompting as a variational inference problem, training a small attacker LLM to approximate the target LLM's posterior over adversarial prompts.
result VERA achieves strong performance across various target LLMs, demonstrating the value of probabilistic inference for adversarial prompt generation.
VBMC combines variational inference and Bayesian quadrature for efficient posterior and model evidence estimation.
problem Efficient inference for models with expensive, black-box likelihoods.
method Combines variational inference with Gaussian-process based active-sampling Bayesian quadrature.
result Produces both a nonparametric approximation of the posterior and an approximate lower bound of the model evidence efficiently.
BBVI with STL converges geometrically under perfect specification, with quadratic variance bound.
problem Convergence rate of BBVI with STL estimator.
method Proved geometric convergence rate with quadratic variance bound for BBVI with STL estimator.
result BBVI with STL converges geometrically under perfect variational family specification.
Improved inference for models with continuous latent variables.
problem Inference accuracy with traditional variational methods is limited.
method Reparameterized Variational Rejection Sampling (RVRS) using a proposal distribution with a reparameterized gradient estimator.
result RVRS offers a better trade-off between computational cost and inference fidelity.
Paper uses black-box inference to estimate non-linear latent force models.
problem Estimating posterior state and forcing term in non-linear systems with unknown forcing terms.
method Black-box variational inference with local inverse autoregressive flows.
result Demonstrates effectiveness of approximation on known posterior systems and non-linear dynamics.
Paper proves smoothness for variational inference, giving convergence guarantees.
problem Proving convergence guarantees for black-box variational inference.
method Describes gradients in an inner-product space, using Bessel's inequality.
result Objective is M-Lipschitz smooth if target is, excluding entropy.
Improves reliability of BBVI optimization methods.
problem Reliability issues and expertise required for BBVI optimization.
method RABVI framework with automated learning rate adjustment and KL divergence estimation.
result RABVI detects inaccurate variational approximations and optimizes reliability.
BaM improves BBVI by optimizing a score-based divergence, leading to faster convergence.
problem Slow convergence of black-box variational inference methods.
method Batch and match (BaM) approach based on a score-based divergence.
result BaM converges exponentially quickly to the target mean and covariance.
This work proposes a new method for variational inference using Wasserstein gradient descent.
problem Optimizing variational parameters to match a true posterior distribution.
method Reinterpreting VI as an optimization problem over a variational parameter space, using Wasserstein gradient descent.
result The proposed Wasserstein gradient descent can be seen as a generalization of existing optimization techniques in VI.
FlowVAT improves variational inference for multi-modal distributions.
problem Mode-seeking behavior and collapse in variational inference for complex posteriors.
method Conditional tempering approach for normalizing flow variational inference.
result FlowVAT outperforms traditional and adaptive annealing methods in multi-modal distributions, finding more modes and achieving better ELBO values.
New bounds show BBVI's gradient variance matches SGD conditions, improving parameterization efficiency.
problem Understanding and improving the convergence of black-box variational inference (BBVI).
method Showed BBVI satisfies matching gradient variance bounds corresponding to the ABC condition for smooth and quadratically-growing log-likelihoods.
result Proven BBVI's gradient variance matches SGD conditions, with superior dimensional dependence for mean-field parameterization.
Efficiently trains dynamic word embedding models with structured variational inference.
problem Training continuous latent time series models with structured variational approximations.
method Analogous to the forward-backward algorithm, a BBVI algorithm that scales linearly in time.
result Efficiently samples from variational distribution and estimates ELBO gradients.
AdVIL improves inference and learning for MRFs with minimal assumptions.
problem Improving inference and learning for Markov random fields (MRFs) with minimal assumptions.
method AdVIL uses adversarial variational inference and learning to approximate latent variables and estimate partition functions.
result AdVIL provides a tighter estimate of the log partition function and better empirical results.
Paper provides variance bounds for variational inference.
problem Understanding the variance of stochastic gradient estimators in variational inference.
method Analyzes reparameterization estimators under smoothness and location-scale assumptions.
result Gives provable bounds on gradient variance, showing they are optimal under stated conditions.
Derives VMP for LDA, simplifying inference for topic modeling.
problem Manual derivation of VMP equations for LDA is challenging and time-consuming.
method Detailed derivation of VMP update equations for LDA.
result Enables easier implementation of VMP for LDA models.
Post-process Bayesian inference speeds up posterior approximation.
problem Leveraging pre-existing model evaluations for quick posterior approximation.
method Variational Sparse Bayesian Quadrature (VSBQ) using sparse Gaussian process (GP) surrogate model.
result VSBQ builds high-quality posterior approximations from existing optimization traces.
BBVI converges nearly dimensionally independent for log-concave targets.
problem Efficiently optimizing variational parameters in high-dimensional spaces.
method Proved convergence rate of BBVI with reparametrization gradient for log-concave targets.
result BBVI converges with nearly independent dimension dependence for log-concave targets.
Paper offers fast deep learning approach for SSMs.
problem Approximate Bayesian inference in SSMs.
method Deep learning and variational inference.
result Fast approach for SSMs.
A new variational inference method using Gaussian score matching.
problem Approximating posterior distributions in Bayesian statistics.
method Score matching principle applied to variational inference.
result Gaussian score matching VI (GSM-VI) is faster and requires fewer gradient evaluations.
A new method trains neural networks to sample from complex distributions.
problem Training neural networks to efficiently sample from complex target distributions.
method Amortized Stein Variational Gradient Descent (SVGD) to iteratively adjust network parameters.
result Our method effectively trains neural networks to sample from any target distribution.
New variational bounds improve posterior covariances and likelihoods.
problem Improving variational inference with different divergence measures.
method Applying variational perturbation theory to construct new variational bounds.
result New variational bounds lead to more accurate posterior covariances and higher likelihoods.
VBMC+VIQR outperforms noisy models in Bayesian inference.
problem Bayesian inference with noisy likelihoods in complex models.
method Gaussian process surrogates, expected information gain, variational interquantile range.
result VBMC+VIQR achieves state-of-the-art performance in noisy inference benchmarks.
Black box variational inference allows researchers to easily prototype and evaluate an array of models. Recent advances allow such algorithms to scale to high dimensions. However, a central question remains: How to specify an expressive variational distribution that maintains efficient computation? To address this, we …
OPVI uses operators to optimize variational objectives, improving scalability and approximation quality.
problem Statistical properties of classical variational inference can be undesirable.
method OPVI redefines variational inference using operators to optimize variational objectives.
result OPVI enables data subsampling and variational programs, improving scalability and approximation quality.
VES-Gamma adapts EI using information-theoretic principles.
problem Optimizing black-box functions using Bayesian optimization.
method Variational Entropy Search (VES) and VES-Gamma algorithm.
result VES-Gamma improves EI by incorporating information-theoretic concepts.
Vprop uses RMSprop for efficient Bayesian deep learning.
problem Efficient implementation of Bayesian deep learning methods.
method Gaussian variational inference using RMSprop optimizer.
result Significant reduction in memory requirements for Black-Box Variational Inference.
DADVI improves ADVI by using deterministic approximation for faster, more accurate posterior estimation.
problem Intractable posterior uncertainty estimates and lack of clear convergence criteria in ADVI.
method Replaces stochastic MFVB objective with deterministic Monte Carlo approximation (SAA) and uses second-order optimization.
result DADVI provides faster and more accurate posterior estimates with default settings.
Latent variable time-series models are among the most heavily used tools from machine learning and applied statistics. These models have the advantage of learning latent structure both from noisy observations and from the temporal ordering in the data, where it is assumed that meaningful correlation structure exists ac…
MCMC-driven learningstat.ML
Unified framework for MCMC and machine learning problems.
problem Intersection of MCMC and machine learning problems.
method Unified framework integrating various MCMC and machine learning techniques.
result Translation and generalization of theory and methods.
Automated variational method for GP models with scalable inference.
problem Efficient inference in models with Gaussian process priors and general likelihoods.
method Automated variational method using mixture of Gaussians, scalable via inducing variables and parallel computation.
result Competitive performance on large datasets, matching state-of-the-art approaches.
Variational inference is a powerful tool for approximate inference, and it has been recently applied for representation learning with deep generative models. We develop the variational Gaussian process (VGP), a Bayesian nonparametric variational family, which adapts its shape to match complex posterior distributions. T…
A new optimization algorithm for Gaussian Variational Inference on precision matrices.
problem Complex models with positive definite constraints on covariance matrices.
method Manifold Gaussian Variational Bayes (MGVBP) with natural gradient updates.
result Empirically validated as a feasible and efficient solution for VI in complex models.