BF-VI improves posterior approximation in complex models.
problem Inefficient posterior approximations in complex models.
method Combines normalizing flows and Bernstein polynomial transformations.
result BF-VI outperforms other VI methods in approximating complex multivariate posteriors.
Analyzes factors affecting flow VI performance.
problem Consistent performance of flow VI across studies.
method Step-by-step analysis of capacity, objectives, batchsize, estimators, and step-sizes.
result Specific recommendations and a flow VI recipe.
AMF-VI uses adaptive mixtures of flows for robust VI across diverse distributions.
problem Inconsistent behavior of single-flow models across different distributions.
method Sequential expert training of individual flows and adaptive global weight estimation via likelihood-driven updates.
result AMF-VI achieves lower negative log-likelihood and stable gains in transport metrics across various posterior families.
Paper advances black-box VI using flows and Monte-Carlo methods.
problem Improving automatic posterior inference in black-box VI.
method Combines normalizing flows, Monte-Carlo methods, and optimization considerations.
result Significant improvement in state-of-the-art variational inference.
New particle-based VI algorithm expands function class and improves scalability.
problem Limited function class in particle-based VI algorithms restricts flexibility and scalability.
method Introduces a functional regularization term to expand the function class and proposes PFG algorithm.
result Proposed PFG algorithm has larger function class, improved scalability, better adaptation to ill-conditioned distributions, and provable convergence.
CIFs improve VI by providing flexible posteriors for complex topologies.
problem Complex posterior distributions in VI problems.
method Using CIFs as part of an auxiliary VI scheme, exploiting conditional independence.
result CIFs yield low-variance estimators of model evidence and improve VI results.
This work proposes new methods for variational inference using gradient flows on Gaussian measures.
problem Developing algorithmic guarantees for variational inference.
method Proposes principled methods for variational inference using gradient flows on the Bures--Wasserstein space of Gaussian measures.
result Strong theoretical guarantees for log-concave posteriors.
This work proposes a new method for variational inference using Wasserstein gradient descent.
problem Optimizing variational parameters to match a true posterior distribution.
method Reinterpreting VI as an optimization problem over a variational parameter space, using Wasserstein gradient descent.
result The proposed Wasserstein gradient descent can be seen as a generalization of existing optimization techniques in VI.
Paper bridges VI and GFlowNets, showing their equivalence in certain cases.
problem Modeling distributions over continuous and discrete structures.
method Demonstrates equivalence between VI and GFlowNets in specific scenarios.
result GFlowNets are more suitable for off-policy training without high gradient variance.
MetFlow combines MCMC and VI efficiently for better inference.
problem Combining MCMC and VI for efficient inference.
method Introduces MetFlow, a novel MCMC algorithm with Normalizing Flows, and a new method to combine it with VI.
result MetFlow produces expressive variational families with improved computational efficiency.
Stochastic normalizing flows use SDEs for efficient training and sampling.
problem Efficient maximum likelihood estimation and variational inference.
method Continuous normalizing flows extended with stochastic differential equations (SDEs) and rough path theory.
result Stochastic normalizing flows enable efficient training and sampling from complex distributions.
Unified framework for efficient trans-dimensional Bayesian inference using VI and NFs.
problem Efficient trans-dimensional Bayesian inference with reduced computational cost.
method Variational inference with normalizing flows to train transport proposals.
result Our approach minimizes reverse KL divergence and reduces computational cost.
Privacy-preserving synthetic data from EHRs for learning and inference.
problem Sharing sensitive EHR data while maintaining patient privacy.
method Differentially private normalizing flows for density estimation and variational inference.
result Privacy-preserving synthetic data can yield good utility at a reasonable privacy cost.
A new method normalizes flow mixtures for better inference across different data types.
problem Inference failure across diverse posterior geometries in normalizing flows.
method Introduces a two-stage framework with a stable global weighting mechanism based on sEMA.
result Achieves consistent NLL improvements and stable weight trajectories over baselines.
Theoretical work on mode collapse in variational inference models.
problem Mode collapse in variational inference models, where models focus on a few modes instead of all possible ones.
method Theoretical investigation of mode collapse in Gaussian mixture models, identifying key low-dimensional statistics and equations governing their evolution.
result Mode collapse is present even in favorable scenarios, driven by mean alignment and vanishing weight mechanisms.
We consider a finite horizon optimal stopping problem related to trade-off strategies between expected profit and cost cash-flows of an investment under uncertainty. The optimal problem is first formulated in terms of a system of Snell envelopes for the profit and cost yields which act as obstacles to each other. We th…
A new method for solving complex inverse problems using deep learning.
problem Estimating complex spatially-varying parameters in high-dimensional Bayesian inverse problems.
method A variational inference method with a deep generative prior to approximate the posterior distribution.
result The method improves estimation accuracy and efficiency for solving high-dimensional inverse problems.
Hybrid approach combines VI and HMC for efficient Bayesian inference in neural networks.
problem Computational demands and inaccuracies in Bayesian inference for neural networks.
method Combines VI and HMC, reducing parameter space and accelerating inference.
result Significantly reduces inference time for large neural networks, improving uncertainty quantification.
Paper introduces f-divergence variational inference for broader application.
problem Variational inference limited to specific divergences.
method Generalizes variational inference to all f-divergences using f-divergence minimization.
result Unified framework for variational inference with arbitrary f-divergences.
RVI accelerates encoderless VI for faster convergence.
problem Slow convergence in encoderless VI methods.
method Introduces Relay Variational Inference (RVI) for faster learning.
result RVI outperforms existing methods in convergence speed and performance.
A-VI can approximate F-VI under certain conditions, improving inference in some models.
problem Approximate Bayesian inference using factorized distributions.
method Amortized variational inference (A-VI) learns a common inference function for latent variables.
result A-VI can close the amortization gap in simple hierarchical models.
This paper reviews recent advancements in amortized Variational Inference.
problem Scalability and efficiency issues in traditional Variational Inference.
method Systematic review of various Variational Inference techniques, focusing on amortized approaches.
result Amortized Variational Inference improves scalability and efficiency for generative modeling tasks.
New framework for Bayesian inference using neural Schrödinger-Föllmer flows.
problem Approximate Bayesian inference in large datasets.
method Stochastic control, Schrödinger bridges, SDE-based models.
result Advocates stochastic control as a finite time and low variance alternative to SGLD.
Automates VI divergence selection for efficient few-shot learning.
problem Efficiently selecting divergence measures for VI to improve performance.
method Meta-learning algorithm to learn optimal divergence metric and variational parameter initialization.
result Meta-learning approach outperforms standard VI methods across various tasks.
We develop a parallel variational inference (VI) procedure for use in data-distributed settings, where each machine only has access to a subset of data and runs VI independently, without communicating with other machines. This type of "embarrassingly parallel" procedure has recently been developed for MCMC inference al…
A new variational inference method using Gaussian score matching.
problem Approximating posterior distributions in Bayesian statistics.
method Score matching principle applied to variational inference.
result Gaussian score matching VI (GSM-VI) is faster and requires fewer gradient evaluations.
A new method prices time-to-event cash flows using survival analysis.
problem Pricing insurance investment portfolios with time-to-event cash flows.
method Discrete-time survival analysis framework, hazard rate estimators, asymptotic multivariate normality.
result Pricing model yields estimates closer to actual cash flows than non-random models.
New findings on Malgrange-Galois groupoid for Painlevé VI equation parameters.
problem Understanding transformations preserving specific forms for Painlevé VI equation.
method Computed Malgrange-Galois groupoid for Painlevé VI family with all parameters.
result Solutions of Painlevé VI do not satisfy new partial differential equations.
A new method interpolates between sampling and variational inference using stochastic mixtures.
problem Combining the strengths of sampling and variational inference methods.
method Develops a framework using stochastic mixtures of simple component distributions to interpolate between sampling and variational inference.
result Improves on both sampling and variational inference methods by reducing bias and variance.
Novel approach simplifies VI problems with faster performance.
problem Black-box VI optimization problems.
method Sample Average Approximation (SAA) combined with quasi-Newton methods and line search.
result Achieves faster performance than existing methods.
A new VIS approach improves log-likelihood estimation in latent variable models.
problem Challenges in achieving high log-likelihood with VI for complex posterior distributions.
method Uses forward χ2 divergence to optimize proposal distribution for better log-likelihood estimation. result Consistently outperforms state-of-the-art baselines in log-likelihood and parameter estimation.
Improved VI with Price's gradient estimator for target log-density.
problem Approximating target distributions from unnormalized log-densities.
method Stochastic gradient-based variational inference with Price's gradient estimator.
result Identifies Price's gradient as the key to WVI's superior performance.
Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a high-dimensional Bayesian posterior with a simpler variational distribution by solv…
New method improves approximate inference for Bayesian models.
problem Approximate inference for high-dimensional Bayesian models.
method Entropic regularization of mean-field variational inference.
result Improved recovery of true posterior dependency.
New Holder bounds improve variational inference by flattening thermodynamic curves.
problem Improving variational inference by addressing performance gaps between theory and practice.
method Generalizing thermodynamic integration to weighted Holder mean, introducing Holder bounds.
result Holder bounds promise a one-step approximation of exact marginal log-likelihood.
VI approximates complex densities faster than classical methods.
problem Approximating complex probability densities.
method Optimization of a family of probability density functions using KL divergence.
result VI converges faster than Markov Chain Monte Carlo.
New algorithms solve stochastic variational inequalities without bounded variance assumption.
problem Solving stochastic variational inequalities without bounded variance assumption.
method Developed algorithms for two classes of problems: monotone and structured nonmonotone VIs.
result Oracle complexity of O(ε^-4) for solving VIs with unbounded domains and possibly unbounded variance.
Simulation-free VI closes the approximation gap in latent SDEs
problem Recovering dynamical systems from noisy observations
method Helmholtz-SDE
result Recovers dynamics more faithfully than prior methods
This paper bridges statistical and machine learning approaches to variational inference.
problem Statisticians struggle to understand variational inference from a Frequentist perspective.
method Explains VI, VAEs, and DDMs from a Frequentist viewpoint, starting with EM.
result VI emerges as a scalable solution for intractable E-steps in VAEs and DDMs.
Develops possibilistic VI using maxitive Donsker-Varadhan formulation.
problem Adapting VI to possibilistic framework to handle uncertainty.
method Maxitive Donsker-Varadhan formulation for possibilistic VI.
result CBOpt optimizers achieve competitive performance in image classification.
One of the core problems of modern statistics is to approximate difficult-to-compute probability densities. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation involving the posterior density. In this paper, we review variational inference (V…
Solve Painleve VI to relate instanton bundles.
problem Relate instanton bundles to Painleve VI solutions.
method Generalize Hitchin's logarithmic connection to vector bundles with SL2 action.
result Identify Okamoto transformations as creation operators.
This paper introduces VI for physics-informed deep learning, enhancing uncertainty quantification.
problem Uncertainty quantification in physics-informed deep learning.
method Variational inference for generative and inverse problems.
result VI provides a flexible and scalable approach for physics-based inference.
New method for constrained sampling using gradient flows.
problem Sampling from constrained domains.
method Introducing a boundary condition for gradient flow to confine particles within the domain.
result Provable continuous-time convergence in total variation for constrained sampling.
To obtain uncertainty estimates with real-world Bayesian deep learning models, practical inference approximations are needed. Dropout variational inference (VI) for example has been used for machine vision and medical applications, but VI can severely underestimates model uncertainty. Alpha-divergences are alternative …
New geometric insights reveal the persistence distribution in spin systems.
problem Determining the full persistence probability distribution in non-Markovian stochastic processes.
method Exact Fredholm Pfaffian structure and Painlevé VI system analysis.
result Recovery of the universal persistence exponent and its geometric interpretation.
Variational inference (VI) provides fast approximations of a Bayesian posterior in part because it formulates posterior approximation as an optimization problem: to find the closest distribution to the exact posterior over some family of distributions. For practical reasons, the family of distributions in VI is usually…
Various standard texts on differential topology maintain that the level-preserving map defined by the track of an isotopy of embeddings is itself an embedding. This note describes a simple counterexample to this assertion.