Proposes sampling from reverse diffusion posteriors for contextual bandits.
problem Complex distributions in contextual bandits.
method Approximate posterior sampling with a diffusion model prior using Laplace approximation.
result Empirically consistent and efficient approximations for contextual bandits.
Posterior refinement improves sample efficiency in Bayesian neural networks.
problem Bayesian neural networks suffer from poor predictive performance due to inaccurate posterior approximations.
method Propose refining Gaussian approximate posteriors with normalizing flows to improve predictive distributions.
result Posterior refinement yields competitive predictive performance with minimal computational overhead.
Simulation-based inference methods can produce unreliable posterior approximations.
problem Reliability of simulation-based inference methods for scientific use cases.
method Benchmarked algorithms including Neural Posterior Estimation, Neural Ratio Estimation, Sequential Neural Likelihood, and Approximate Bayesian Computation.
result Ensembling posterior surrogates provides more reliable approximations.
Bayesian neural networks approximate Gaussian, this method adapts to non-Gaussian posteriors.
problem Bayesian neural networks struggle with non-Gaussian posteriors, leading to poor performance.
method Proposes a Riemannian Laplace approximation to adapt to the shape of the true posterior.
result Consistently improves over conventional Laplace approximation across tasks.
The representation of the approximate posterior is a critical aspect of effective variational autoencoders (VAEs). Poor choices for the approximate posterior have a detrimental impact on the generative performance of VAEs due to the mismatch with the true posterior. We extend the class of posterior models that may be l…
BF-VI improves posterior approximation in complex models.
problem Inefficient posterior approximations in complex models.
method Combines normalizing flows and Bernstein polynomial transformations.
result BF-VI outperforms other VI methods in approximating complex multivariate posteriors.
Optimized α-posteriors reduce KL divergence from true posterior in parametric misspecification.
problem Reduction of KL divergence from true posterior in parametric model misspecification.
method Derivation of Bernstein-von Mises theorem and optimization of α-posteriors. result Optimized α-posteriors minimize KL divergence from true posterior, especially in severe misspecification. Markov chain Monte Carlo (MCMC) algorithms have become powerful tools for Bayesian inference. However, they do not scale well to large-data problems. Divide-and-conquer strategies, which split the data into batches and, for each batch, run independent MCMC algorithms targeting the corresponding subposterior, can spread…
A new Fusion method combines multiple distributions efficiently.
problem Combining multiple distributions for big data or privacy.
method Recursive divide-and-conquer Monte Carlo approach.
result A robust Fusion method for many sub-posteriors.
Bayesian method improves approximate model posteriors.
problem Poor uncertainty quantification in approximate Bayesian inference.
method Optimizing a transformation of the approximate posterior to maximize a scoring rule.
result Significant reduction in bias and improvement in posterior coverage properties.
We propose a general algorithm for approximating nonstandard Bayesian posterior distributions. The algorithm minimizes the Kullback-Leibler divergence of an approximating distribution to the intractable posterior distribution. Our method can be used to approximate any posterior distribution, provided that it is given i…
Variational inference (VI) provides fast approximations of a Bayesian posterior in part because it formulates posterior approximation as an optimization problem: to find the closest distribution to the exact posterior over some family of distributions. For practical reasons, the family of distributions in VI is usually…
This paper introduces Bayes Hilbert spaces for efficient posterior approximation.
problem Efficient posterior approximation in Bayesian models for large datasets.
method Develops Bayes Hilbert spaces for posterior approximation and connects them to Bayesian coresets and kernel-based distances.
result Bayes Hilbert spaces provide a novel framework for posterior approximation that is computationally efficient.
The paper introduces a new Wasserstein distance for approximating posteriors in inverse problems.
problem Approximating posterior measures in inverse problems using conditional Wasserstein distances.
method Introduces a conditional Wasserstein distance with restricted couplings and derives its dual.
result Shows that conditional Wasserstein GANs can yield favorable properties for posterior sampling.
The paper simplifies Bayesian posterior using clustering to make inference more manageable.
problem Handling large-scale, redundant datasets in Bayesian learning.
method Construct an approximate posterior by replacing data points in the same cluster with the centroid.
result The approximate posterior is close to the exact posterior and easier to sample from.
This paper introduces a spline-based method for nonparametric ADVI that handles complex posterior distributions.
problem Learning complex posterior distributions with skewness, multimodality, and bounded support.
method Develops a spline-based nonparametric approximation approach for ADVI.
result Establishes the asymptotic consistency of the derived lower bound for importance weighted autoencoder.
Paper uses averaging from many particle filters to approximate posterior predictive distributions.
problem Approximating posterior predictive distributions efficiently and accurately.
method Particle swarm filter algorithm that averages many particle filter approximations.
result Law of large numbers and central limit theorem support the method's effectiveness.
Thompson sampling (TS) is a class of algorithms for sequential decision-making, which requires maintaining a posterior distribution over a model. However, calculating exact posterior distributions is intractable for all but the simplest models. Consequently, efficient computation of an approximate posterior distributio…
This work extends balancing to various simulation-based inference algorithms for more conservative posterior approximations.
problem Overconfident posterior approximations in simulation-based inference.
method Introduces a balanced version of neural posterior estimation and contrastive neural ratio estimation.
result Balanced versions tend to produce conservative posterior approximations on various benchmarks.
A new method approximates posterior for VAEs without iterative training.
problem Inference models in VAEs are poor early on, leading to suboptimal models.
method Train generative and inference models independently, using a model-agnostic posterior approximation (MAPA).
result MAPA approximates the true posterior deterministically and can improve density estimation.
The Laplace approximation has been one of the workhorses of Bayesian inference. It often delivers good approximations in practice despite the fact that it does not strictly take into account where the volume of posterior density lies. Variational approaches avoid this issue by explicitly minimising the Kullback-Leibler…
QEM uses parallel importance weighting for fast approximate Bayesian inference.
problem Bayesian inference challenges in large models with many observations and latent variables.
method Expectation Maximization (EM) with massively parallel importance weighting.
result QEM is faster and more scalable than RWS and VI.
New method improves generative model performance by fully conditioning variational posteriors.
problem Inaccurate inference due to partial conditioning of variational posteriors in sequential LVMs.
method Introduces fully-conditioned approximate posteriors to improve generative model performance.
result Improves generative modelling and multi-step prediction performance.
The choice of approximate posterior distribution is one of the core problems in variational inference. Most applications of variational inference employ simple families of posterior approximations in order to allow for efficient inference, focusing on mean-field or other simple structured approximations. This restricti…
Develops a fast variational approximation for high-dimensional empirical Bayes posteriors.
problem Optimal posterior computation in high-dimensional settings with prior tails effect.
method Variational approximation of empirical Bayes posterior with data-driven centers and thin-tailed conjugate priors.
result Retains optimal concentration rate properties and superior performance compared to existing methods.
TM-VI uses flexible transformation models to approximate complex posteriors in Bayesian models.
problem Approximating complex posteriors in Bayesian models with limited flexibility.
method Transformation models for variational inference (TM-VI).
result TM-VI allows accurate approximation of complex posteriors in models with one parameter and works in a mean-field fashion for multi-parameter models.
Bayesian nonparametric models get better posterior estimates via SPDE methods.
problem Estimating posterior distributions in nonparametric Bayesian models.
method Extending diffusion methods to SPDEs on Hilbert spaces for posterior contraction and Laplace approximation.
result Derivation of posterior contraction rates and finite-sample Bernstein von Mises results.
Improved Bayesian inference via variational approximations of generalized rho-posteriors.
problem Robust Bayesian inference under model misspecification and data contamination.
method Introducing a modified ρ-posterior and using PAC-Bayesian analysis with variational approximations. result Theoretical guarantees for tractable inference with competitive robustness and computational efficiency.
PVI seeks a posterior that makes predictions closer to true data, not approximating the Bayesian posterior.
problem Finding meaningful posterior distributions under model misspecification.
method Predictive variational inference (PVI) seeks an optimal posterior density for close predictive matching to true data.
result PVI learns a posterior that is not the same as the Bayesian posterior, but is closer to the true data generating process.
Variational inference methods often focus on the problem of efficient model optimization, with little emphasis on the choice of the approximating posterior. In this paper, we review and implement the various methods that enable us to develop a rich family of approximating posteriors. We show that one particular method …
Post-process Bayesian inference speeds up posterior approximation.
problem Leveraging pre-existing model evaluations for quick posterior approximation.
method Variational Sparse Bayesian Quadrature (VSBQ) using sparse Gaussian process (GP) surrogate model.
result VSBQ builds high-quality posterior approximations from existing optimization traces.
VPR improves posterior uncertainty quantification by combining VI and predictive resampling.
problem Inaccurate posterior sampling with MCMC due to computational constraints.
method Variational predictive resampling (VPR) that uses VI's predictive strength and imputes future observations.
result VPR converges to the exact Bayesian posterior in a Gaussian location model and improves uncertainty quantification.
The paper addresses the invariance issue in Bayesian neural networks using linearized Laplace approximation.
problem Bayesian neural networks fail to maintain invariance under reparameterization, leading to different posterior densities for identical functions.
method Developed a geometric view of reparameterizations and a Riemannian diffusion process to extend reparameterization invariance to neural network predictive.
result Empirically improved posterior fit through approximate posterior sampling.
Loss-calibrated EP improves Bayesian decision-making by focusing on utility-sensitive posterior approximations.
problem Bayesian decision-making under asymmetric utility functions.
method Loss-calibrated expectation propagation (Loss-EP) that tilts the posterior towards higher utility decisions.
result Loss-EP can capture useful information for decision-making under asymmetric penalties.
We propose a black-box variational inference method to approximate intractable distributions with an increasingly rich approximating class. Our method, termed variational boosting, iteratively refines an existing variational approximation by solving a sequence of optimization problems, allowing the practitioner to trad…
The paper corrects Bayesian neural network approximations to improve decision quality.
problem Inaccurate posterior approximations in Bayesian neural networks lead to suboptimal decisions.
method Develops methods to calibrate approximate posterior predictive distributions for better decision making.
result Empirically produces higher quality decisions compared to previous methods.
ConDiSim uses diffusion models to approximate complex system posteriors efficiently.
problem Simulation-based inference of systems with intractable likelihoods.
method Conditional diffusion model with forward and reverse processes.
result Effective posterior approximation across various benchmark and real-world problems.
A new method combines scores of individual observations to efficiently approximate posterior distributions.
problem Handling posterior distributions conditioned on multiple observations with neural methods.
method Conditional score modeling to combine learned scores from individual observations.
result Sample-efficient method that can aggregate multiple observations at inference time.
Enhances predictive performance in Bayesian deep learning via generalized Laplace approximation.
problem Inconsistency in Bayesian deep learning.
method Interprets posterior tempering as a correction for model misspecification and recalibration of priors. Introduces generalized Laplace approximation.
result Generalized Laplace approximation enhances predictive performance.
New methods for scalable inference in modular models with misspecified sub-models.
problem Model misspecification in multi-modular models complicates evidence combination.
method Variational methods for approximating Cut and SMI posteriors, and Variational Meta-Posterior.
result Feasibility of analysis with multiple cuts using a single set of variational parameters.
Method trains emulators to estimate posterior probabilities safely.
problem Uncertainty in slow forward model calculations.
method Trains emulators while estimating posterior probabilities with MCMC, propagating error.
result Demonstrates robust posterior inference for ΛCDM cosmology model. NPE trains neural networks to approximate posterior distributions in SIR models from final outcome data.
problem Computational challenges in Bayesian inference for SIR models with final outcome data.
method Neural posterior estimation (NPE) using a logNormal posterior approximated by a neural network.
result NPE accurately recovers reference posteriors across various population sizes and transmission regimes.
L-C2ST improves local diagnostics for SBI approximations.
problem Evaluating trustworthiness of posterior approximations in SBI.
method Local evaluation of posterior estimators at any observation.
result Offers better statistical power and interpretability.
Estimating the predictive uncertainty of a Bayesian learning model is critical in various decision-making problems, e.g., reinforcement learning, detecting adversarial attack, self-driving car. As the model posterior is almost always intractable, most efforts were made on finding an accurate approximation the true post…
S-VBMC improves VBMC's exploration of complex posterior distributions.
problem Efficient inference for computationally expensive models with complex posterior distributions.
method Stacking multiple independent VBMC runs to create a robust global posterior approximation.
result Significant improvements in posterior approximation quality across various applications.
A new method improves posterior approximation for complex distributions.
problem Difficulty in capturing multimodal and heavy-tailed posteriors with standard normalizing flows.
method StiCTAF: stick-breaking mixture base with component-wise tail adaptation.
result Improved tail recovery and better mode coverage compared to benchmarks.
Enhanced Gaussian process models accelerate optimization and posterior approximation.
problem Improving the accuracy and speed of Gaussian process models for optimization and inference.
method Introduces a random exploration step to classical GP-UCB algorithms, facilitating faster convergence.
result New algorithms achieve nearly optimal convergence rates and provide bounds for Hellinger distance.
Sparse variational approximations allow for principled and scalable inference in Gaussian Process (GP) models. In settings where several GPs are part of the generative model, theses GPs are a posteriori coupled. For many applications such as regression where predictive accuracy is the quantity of interest, this couplin…