Study convergence rates of variational posterior distributions for inference.
problem Characterize convergence rates of variational posterior distributions for nonparametric and high-dimensional inference.
method Formulate general conditions on prior, likelihood, and variational class to characterize convergence rates. Propose novel prior mass conditions for specific prior distributions.
result The convergence rate of variational posterior distributions is the sum of the convergence rate of the true posterior and the variational approximation error.
A new meta-learning method using shared variational inference.
problem Meta-learning with uncertainty over model parameters.
method Shared amortized variational inference network for conditional prior and posterior.
result Prevents collapse of conditional prior to Dirac delta function.
CATVI improves variational inference for Bayesian nonparametric models by reducing divergence and improving prediction accuracy.
problem Limitations of current variational inference methods in characterizing latent correlations and inferring true posterior dimensions.
method CATVI integrates conditional and adaptive truncation into variational inference, maximizing nonparametric evidence lower bound and using Monte Carlo integration.
result CATVI outperforms traditional methods in Bayesian nonparametric topic models, reducing perplexity and improving topic-word clustering.
EVI improves variational inference for non-Gaussian models.
problem Infeasibility of finding analytically tractable solutions for non-Gaussian statistical models.
method Extended Variational Inference (EVI) using lower-bound approximation to the variational objective function.
result Convergence of EVI depends on lower-bound approximation strategy.
A new particle algorithm improves mean-field variational inference.
problem Efficiently approximating nonparametric posterior distributions in machine learning.
method Introduces PArticle VI (PAVI), a novel particle-based algorithm for nonparametric mean-field approximation.
result Obtains non-asymptotic error bounds for PArticle VI, providing the first end-to-end guarantee for particle-based MFVI.
Study high-dimensional Bayesian linear regression using variational inference.
problem High-dimensional Bayesian linear regression with product priors.
method Non-linear large deviations theory and variational inference.
result Unique optimizer in variational problem governs posterior distribution under separation condition.
A new framework speeds up Bayesian inference in probabilistic programs.
problem Efficient Bayesian inference in complex probabilistic programs.
method Embeds a sampler within a refined variational posterior approximation, using automatic differentiation for parameter tuning.
result Significantly speeds up mixing time and improves efficiency in various probabilistic program tasks.
Inference for SDEs using variational methods and neural networks.
problem Parameter inference for stochastic differential equations is challenging due to latent diffusion processes.
method Variational inference with a mean-field approximation for parameters and a recurrent neural network for diffusion paths.
result Accurate parameter estimates for SDE systems, demonstrated on Lotka-Volterra and epidemic models.
FlowVAT improves variational inference for multi-modal distributions.
problem Mode-seeking behavior and collapse in variational inference for complex posteriors.
method Conditional tempering approach for normalizing flow variational inference.
result FlowVAT outperforms traditional and adaptive annealing methods in multi-modal distributions, finding more modes and achieving better ELBO values.
CoSMIC extends flow-based SVI to transdimensional problems.
problem Bayesian structure learning and model selection with multi-model parameter spaces.
method Normalizing flows with a combined stochastic variational transdimensional inference approach.
result Improved performance on high-cardinality model spaces.
Proposes a new algorithm for efficient probabilistic inference.
problem Efficient probabilistic inference in deep models with graphical structures.
method Structured inference networks and variational message-passing algorithm.
result Enables fast and efficient natural-gradient inference for deep structured models.
A-VI can approximate F-VI under certain conditions, improving inference in some models.
problem Approximate Bayesian inference using factorized distributions.
method Amortized variational inference (A-VI) learns a common inference function for latent variables.
result A-VI can close the amortization gap in simple hierarchical models.
Meta-learn Bayesian inference for task-specific BNNs using amortised inference.
problem Efficiently learning Bayesian inference for small-scale probabilistic meta-learning.
method Replace global inducing points with actual data to create a set of approximate likelihoods, train a meta-model to learn these parameters across related datasets.
result Meta-learned inference can be applied to task-specific BNNs, improving efficiency and scalability.
Paper proposes cross-coding to improve conditional inference in VAEs.
problem Challenges in conditional inference with VAEs, especially for arbitrary queries.
method Cross-coding to approximate latent distributions after conditioning.
result Cross-coding variations outperform Hamiltonian Monte Carlo.
Recent advances in neural variational inference have spawned a renaissance in deep latent variable models. In this paper we introduce a generic variational inference framework for generative and conditional models of text. While traditional variational methods derive an analytic approximation for the intractable distri…
The paper provides theoretical guarantees for transformation-based models in variational inference.
problem Theoretical justification for transformation-based models in variational inference.
method Theoretical analysis of non-linear latent variable models and Gaussian process priors.
result Theoretical guarantees for implicit variational inference, achieving optimal risk bounds and approximating the true posterior.
New method estimates uncertainty in knowledge graph embeddings using neural variational inference.
problem Estimating uncertainty in knowledge graph embeddings.
method Constructs an inference network conditioned on symbolic representations of entities and relation types in a Knowledge Graph.
result Improved predictive uncertainty estimates during link prediction.
Proposes a method to learn conditional VAEs from datasets with missing covariates.
problem Learning conditional VAEs from datasets with missing covariates.
method Augments conditional VAEs with a prior distribution for missing covariates and estimates their posterior using amortised variational inference.
result The proposed method outperforms previous methods in learning conditional VAEs from non-temporal, temporal, and longitudinal datasets.
New model improves multimodal autoencoders by learning joint and conditional distributions.
problem Limitations in recent multimodal autoencoders restrict their quality on complex datasets.
method Proposes a multistage training process with variational inference and Normalizing Flows, leveraging shared modality information.
result Achieves state-of-the-art results on benchmark datasets.
Boosting variational inference uses optimization to improve posterior approximations.
problem Improving the tractability of Bayesian posterior approximations.
method An optimization-based approach to greedily add components to a mixture of densities.
result Established convergence properties and rates for the boosting variational inference algorithm.
A method for estimating signal distributions from inverse problems using normalizing flows.
problem Estimating the distribution of the underlying signal from observations in inverse problems.
method A framework for approximate inference on a pre-trained unconditional flow model, using a composition of two flow models for stable variational inference.
result Our method produces high-quality samples with uncertainty quantification and can be amortized for zero-shot inference.
New method improves generative model performance by fully conditioning variational posteriors.
problem Inaccurate inference due to partial conditioning of variational posteriors in sequential LVMs.
method Introduces fully-conditioned approximate posteriors to improve generative model performance.
result Improves generative modelling and multi-step prediction performance.
The paper connects neural network ensembles to Bayesian inference using variational methods.
problem Explaining the behavior of ensemble methods in neural networks.
method Deriving conditions for ensemble optimization to reduce divergence to the posterior distribution.
result Ensemble methods can be a valid alternative to approximate Bayesian inference.
Algorithm improves variational inference in Wasserstein distance.
problem Improving variational inference methods for complex models.
method Wasserstein contraction analysis of coordinate ascent.
result General and sharp convergence guarantees for various models.
A Closer Look at Disentangling in β-VAE shows non-monotonic inference performance.
problem Learning disentangled representations from data.
method Generalization of VAE using variational inference with hyperparameter β.
result Non-monotonic inference performance in β-VAE with a finite optimal β.
APG samplers use neural suff stats to improve deep model inference.
problem Efficient inference in deep generative models.
method Amortized population Gibbs, neural suff stats, KL divergence minimization.
result Significant improvement in inference accuracy.
The paper proves consistency of GVI posteriors under minimal conditions.
problem Consistency of generalized variational inference posteriors.
method Proves consistency using Γ-convergence theory. result GVI posteriors are consistent and collapse to the population-optimal parameter value.
A new method for estimating causal parameters from observables reduces the need for finite moment conditions.
problem Estimating causal parameters from observational data with unknown or infinite moment conditions.
method Variational Method of Moments (VMM) for a general class of estimators, including kernel and neural net-based methods.
result VMM estimators are consistent, asymptotically normal, and semiparametrically efficient.
New bounds show BBVI's gradient variance matches SGD conditions, improving parameterization efficiency.
problem Understanding and improving the convergence of black-box variational inference (BBVI).
method Showed BBVI satisfies matching gradient variance bounds corresponding to the ABC condition for smooth and quadratically-growing log-likelihoods.
result Proven BBVI's gradient variance matches SGD conditions, with superior dimensional dependence for mean-field parameterization.
A new method for efficient inference in sequential latent-variable models.
problem Computational challenges in integrating subject-specific random effects.
method Anchored variational inference framework to approximate posterior distributions.
result The method achieves accurate estimation with significant computational gains.
A new method improves SVI for high-dimensional, poorly-conditioned distributions.
problem Challenges in existing SVI methods for high-dimensional, poorly-conditioned distributions.
method Trust-region optimization approach leveraging conditional independences and second-order information.
result Superior numerical performance and better scalability in high-dimensional distributions.
PE-SVI reduces SVI inference complexity by finding a suitable start point.
problem Complex posterior inference in graphical models leads to suboptimal learning.
method PE-SVI uses a pseudo-encoded start point to reduce gradient steps and step sizes.
result PE-SVI achieves the same ELBo objective as SVI with less than 1% of the required steps.
Cascading flows improve variational inference in structured programs.
problem Challenges in variational inference for complex probabilistic programs.
method Integrates normalizing flows and ASVI to create cascading flows, which embed the forward-pass of probabilistic programs.
result Cascading flows outperform normalizing flows and ASVI in structured inference problems.
Two approximate lifted variational methods for hybrid domains improve inference scalability and accuracy.
problem Efficient inference in hybrid probabilistic relational models with multi-modality and continuous evidence.
method Two approximate lifted variational approaches applicable to hybrid domains, exploiting model symmetries.
result The proposed variational methods are scalable and can leverage approximate model symmetries, outperforming existing message-passing approaches.
New method models longitudinal data using variational inference and normalizing flows.
problem Handling high-dimensional longitudinal data with time dependency.
method Variational inference with normalizing flows for latent variables.
result The method achieves better likelihood estimates and more reliable missing data imputation.
Sequential coordinate ascent is more robust in high-dimensional linear regression.
problem Behavior difference between sequential and parallel coordinate ascent in variational inference.
method Comparison of sequential and parallel coordinate ascent algorithms in high-dimensional linear regression.
result Sequential algorithm converges under more relaxed conditions than parallel algorithm.
NeVI-Cut uses neural networks to efficiently propagate uncertainty without feedback.
problem Efficiently propagating uncertainty in downstream Bayesian analysis without feedback.
method NeVI-Cut combines neural networks and normalizing flows for variational inference.
result NeVI-Cut achieves significant computational gains and higher accuracy than traditional methods.
Amortized VI for DGPs learns efficient inference.
problem Expressive limitations in GP approximations.
method Amortized variational inference for DGPs.
result Improved expressive prior and posterior for DGPs.
Tutorial on combining latent variable models with deep learning.
problem Combining latent variable models with deep learning to model natural language.
method Exploring variational inference to address intractable posterior inference and non-differentiability issues.
result Exploration of variational inference techniques to handle deep latent variable models.
Adaptive variational Bayes framework improves inference adaptively.
problem Lack of general and computationally tractable variational Bayes method for adaptive inference.
method Proposes a novel adaptive variational Bayes framework combining variational posteriors over individual models.
result Adaptive variational Bayes achieves optimal contraction rates adaptively under general conditions.
New algorithm speeds up large-scale statistical inference.
problem Efficiently solving large-scale mean-field variational inference problems.
method Developed a novel primal-dual algorithm (PD-VI) and a block-preconditioned extension (P2D-VI) for mean-field variational inference. result PD-VI and P2D-VI achieve faster convergence and better solution quality compared to existing methods. This paper bridges variational inference and Wasserstein gradient flows.
problem Combining variational inference and Wasserstein gradient flows for more efficient approximations.
method Recasting Bures-Wasserstein gradient flow as a Euclidean gradient flow and using path-derivative gradient estimator.
result A new gradient estimator for f-divergences that can be implemented using machine learning libraries. VarGrad reduces variance in ELBO gradient estimation for variational inference.
problem Improving the variance of gradient estimators in variational inference.
method VarGrad uses a new log-variance loss to estimate the ELBO gradient, achieving lower variance than the score function method.
result VarGrad offers a lower variance gradient estimator compared to other methods.
AdVIL improves inference and learning for MRFs with minimal assumptions.
problem Improving inference and learning for Markov random fields (MRFs) with minimal assumptions.
method AdVIL uses adversarial variational inference and learning to approximate latent variables and estimate partition functions.
result AdVIL provides a tighter estimate of the log partition function and better empirical results.
The paper improves guarantees for VI in symmetric cases.
problem Approximating intractable densities via VI with misspecified families.
method Extends previous robust VI results to wider divergences and non-log-concave targets.
result Guarantees for exact recovery of target mean and correlation matrix under various conditions.
Geometric framework analyzes bias in variational inference for posterior functionals.
problem Analyzing the bias of posterior functionals under variational approximations.
method Developed a geometric framework to evaluate the bias of posterior functionals using the variational tangent space.
result The leading-order bias of a posterior functional is determined by its component orthogonal to the variational tangent space.
Unified theory for semi-implicit variational inference, bridging approximation and optimization.
problem Developing a statistical theory for semi-implicit variational inference.
method Unified theory combining approximation and optimization analyses.
result Unified theory characterizes SIVI's ability to recover target distributions and governs asymptotic behavior.
Study improves posterior inference in neural processes with limited data.
problem Improving posterior predictive inference in probabilistic models with scarce conditioning data.
method Examined effects of pooling operators and variational families on posterior quality in neural processes.
result Novel neural process architectures lead to superior posterior predictive samples in image completion/in-painting tasks.