Post-process Bayesian inference speeds up posterior approximation.
problem Leveraging pre-existing model evaluations for quick posterior approximation.
method Variational Sparse Bayesian Quadrature (VSBQ) using sparse Gaussian process (GP) surrogate model.
result VSBQ builds high-quality posterior approximations from existing optimization traces.
The paper explores how to evaluate Bayesian approximations in neural networks.
problem The difficulty in evaluating Bayesian approximations in neural networks.
method Exploring the interactions between probabilistic models, approximating distributions, optimization algorithms, and datasets.
result The expected utility of the approximate posterior can measure inference quality.
Bayesian bandit algorithms with approximate inference improve regret bounds in stochastic linear bandits.
problem Theoretical justification for Bayesian bandit algorithms with approximate inference in stochastic linear bandits.
method Proposed a theoretical framework to analyze approximate inference impact and conducted frequentist regret analysis on LinTS and LinBUCB.
result LinTS and LinBUCB preserve their original regret upper bounds with larger constant terms in approximate inference settings.
Bayesian methods enhance deep learning models by improving reliability and uncertainty.
problem Improving reliability and uncertainty awareness in deep learning models.
method Approximate Bayesian inference techniques, including SG-MCMC and VI, applied to deep learning models.
result Enhanced posterior inference for deep learning models, particularly in neural networks and generative models.
New diagnostic tool for assessing approximate Bayesian inference.
problem Assessing the trustworthiness of approximate Bayesian inference.
method Reframe the problem in terms of incompatible conditional distributions and use Gibbs priors.
result The diagnostic tool can discover the inductive bias in various Bayesian models and approximations.
Improved Bayesian neural network inference by selectively removing redundant modes.
problem Redundant modes in Bayesian neural network posteriors complicate approximate inference.
method Structured partial stochasticity and deterministic subset selection of weights.
result Improved performance of approximate inference schemes with simplified posterior distribution.
Paper proposes Walsh-Hadamard Variational Inference for efficient approximate inference in large models.
problem Over-regularization in variational inference for large models.
method Walsh-Hadamard factorization strategies to reduce parameterization, accelerate computations, and increase posterior expressiveness.
result Efficient approximate inference achieved in over-parameterized models.
This paper improves Bayesian inference for predictive models with limited data.
problem Effective uncertainty quantification for training predictive models with limited data.
method Entropy-regularized gradient estimators to approximate the Bayesian posterior.
result The method generates diverse samples from the posterior distribution efficiently.
The paper proposes using path signatures for better inference in time series data.
problem Simulation models with time series data often lack tractable likelihood functions.
method Approximate Bayesian Computation with path signatures to handle sequential data.
result Theoretical guarantees on the resultant posteriors for Bayesian parameter inference.
Proposes TAGI for efficient Gaussian inference in Bayesian neural networks.
problem Efficient inference in Bayesian neural networks with complex architectures.
method Analytical method for tractable approximate Gaussian inference (TAGI).
result Matches performance of gradient-based methods with O(n) computational complexity. Bayesian inference using stochastic neural networks ensembles.
problem Approximating Bayesian posterior distributions.
method Formulate stochastic ensembles of neural networks, train with variational inference, and evaluate using Monte Carlo dropout.
result Stochastic ensembles provide more accurate posterior estimates than other methods.
This paper explores approximations for fully Bayesian Gaussian Process Regression.
problem Learning in Gaussian Process models through hyperparameter adaptation.
method Two approximation schemes: Hamiltonian Monte Carlo and Variational Inference.
result Predictive performance analysis on various benchmark datasets.
Bayesian deep learning method using subnetwork inference.
problem Improving deep neural networks' calibration and efficiency.
method Perform inference over a subset of model weights, keeping others as point estimates.
result Subnetwork inference enables accurate predictive posteriors without full network approximations.
Meta-learn Bayesian inference for task-specific BNNs using amortised inference.
problem Efficiently learning Bayesian inference for small-scale probabilistic meta-learning.
method Replace global inducing points with actual data to create a set of approximate likelihoods, train a meta-model to learn these parameters across related datasets.
result Meta-learned inference can be applied to task-specific BNNs, improving efficiency and scalability.
Fast Bayesian inference with adaptable priors for real-time applications.
problem Intractable exact posterior computation limits Bayesian inference's adoption.
method Distribution Transformer architecture that learns mappings between priors and posteriors.
result Significant reduction in computation time from minutes to milliseconds.
Improved Bayesian inference via variational approximations of generalized rho-posteriors.
problem Robust Bayesian inference under model misspecification and data contamination.
method Introducing a modified ρ-posterior and using PAC-Bayesian analysis with variational approximations. result Theoretical guarantees for tractable inference with competitive robustness and computational efficiency.
This thesis disentangles Gauss-Newton and variational approximations in Bayesian deep learning.
problem Understanding the interplay between the Gauss-Newton method and variational approximations in Bayesian deep learning.
method Analysis of the Gauss-Newton method and Laplace/Gaussian variational approximations for neural networks.
result The combination of the Gauss-Newton method with approximate inference can be cast as inference in a linear or Gaussian process model.
Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
Simulation-based inference methods can produce unreliable posterior approximations.
problem Reliability of simulation-based inference methods for scientific use cases.
method Benchmarked algorithms including Neural Posterior Estimation, Neural Ratio Estimation, Sequential Neural Likelihood, and Approximate Bayesian Computation.
result Ensembling posterior surrogates provides more reliable approximations.
URSABench benchmarks Bayesian methods for deep learning models.
problem Scalability issues in Bayesian inference for deep learning.
method Open-source benchmark suite for assessing approximate Bayesian inference methods.
result Initial results show promise for addressing uncertainty and robustness in deep learning.
This work explores function-space inference using KL divergence and proposes Bayesian linear regression as a benchmark.
problem Approximating the predictive posterior distribution of Bayesian models without parameter posterior approximation.
method Employing Kullback-Leibler divergence and proposing featurized Bayesian linear regression as a benchmark.
result Minimizing KL divergence leads to an ill-defined objective function, highlighting limitations of this approach.
Normalizing flow regression approximates posterior distributions without additional sampling.
problem Bayesian inference with computationally expensive likelihood evaluations.
method Normalizing flow regression (NFR) for offline inference.
result NFR yields a tractable posterior approximation through regression on existing log-density evaluations.
Framework for Bayesian inference using GP emulated MH sampler for noisy likelihoods.
problem Approximate Bayesian inference with limited noisy log-likelihood evaluations.
method Gaussian process emulates MH sampler for log-likelihood evaluations; sequential experimental design selects evaluation points.
result Approximate sampler is sample-efficient and robust to GP assumptions.
Bayesian model averaging fails under covariate shift, affecting neural networks' performance.
problem Bayesian model averaging's failure in neural networks under covariate shift.
method Explained the issue and proposed novel priors to improve robustness.
result Bayesian model averaging is problematic under covariate shift, especially with linear feature dependencies.
EBUCB framework achieves optimal regret with bounded approximate inference error.
problem Theoretical gap between practical performance and theoretical justification of Bayesian bandit algorithms with approximate inference.
method Enhanced Bayesian Upper Confidence Bound (EBUCB) framework that accommodates bandit problems with approximate inference.
result EBUCB achieves optimal regret order O(logT) under certain conditions on inference error. Researchers develop methods to calibrate ABMs using Bayesian techniques.
problem Challenges in calibrating ABMs due to likelihood intractability and non-differentiability.
method Generalised variational inference for differentiable ABMs.
result Accurate Bayesian parameter inferences for differentiable ABMs demonstrated.
UA-SABI uses surrogates to speed up Bayesian inference for expensive models.
problem Inference for computationally expensive models is slow and uncertain.
method Combines surrogate modeling with Amortized Bayesian Inference (ABI) to propagate uncertainties.
result Reliable, fast, and repeated Bayesian inference for expensive models is achieved.
Bayesian inference typically requires the computation of an approximation to the posterior distribution. An important requirement for an approximate Bayesian inference algorithm is to output high-accuracy posterior mean and uncertainty estimates. Classical Monte Carlo methods, particularly Markov Chain Monte Carlo, rem…
We develop a scalable method for Bayesian neural networks with stochastic differential equations.
problem Uncertainty quantification in deep neural networks.
method Gradient-based stochastic variational inference in continuous-depth Bayesian neural networks.
result Gradient estimator with zero variance as the approximation improves.
The paper connects ABC to GBI, suggesting ABC as a robustification strategy.
problem Approximate Bayesian Computation struggles with tractability in complex simulators.
method Reinterpreting ABC as an implicitly defined error model and suggesting GBI.
result ABC can be seen as a robustification strategy for approximating Bayesian posteriors.
Bayesian priors and penalties are equivalent in variational inference.
problem Understanding the relationship between Bayesian priors and penalties in variational inference.
method Characterizing the regularizers that can arise in variational inference and providing a systematic way to compute the prior corresponding to a given penalty.
result Equivalence between Bayesian priors and penalties in variational inference.
JANA trains networks to approximate Bayesian models efficiently.
problem Intractable likelihood functions and posterior densities in Bayesian models.
method End-to-end training of three networks: summary, posterior, and likelihood networks.
result JANA provides accurate amortized marginal likelihood and posterior predictive estimation.
A new method combines ANN and Laplace for fast Bayesian inference in ODE models.
problem Bayesian inference for ODE systems with non-analytical solutions is computationally expensive.
method Hybrid approach using ANN for tractable likelihood and Laplace approximation.
result Effective posterior inference with improved computational cost compared to traditional methods.
PyVBMC speeds up Bayesian inference for expensive models in Python.
problem Efficient Bayesian inference for computationally expensive models.
method Variational Bayesian Monte Carlo (VBMC) algorithm.
result PyVBMC provides a flexible and efficient method for parameter estimation and model assessment.
BayesDLL offers a PyTorch library for Bayesian deep learning with large models.
problem Bayesian inference for large-scale deep networks.
method Variational inference, MC-dropout, stochastic-gradient MCMC, Laplace approximation.
result BayesDLL can handle Vision Transformers and pre-trained model weights as priors.
A new method improves likelihood-free Bayesian inference by transforming summary statistics and using efficient Variational Bayes.
problem Incorrectly assuming normally distributed summary statistics in likelihood-free Bayesian inference.
method Wasserstein Gaussianization transformation combined with robust BSL and efficient Variational Bayes.
result Highly efficient and reliable approximate Bayesian inference for likelihood-free problems.
We present a scalable approach to performing approximate fully Bayesian inference in generic state space models. The proposed method is an alternative to particle MCMC that provides fully Bayesian inference of both the dynamic latent states and the static parameters of the model. We build up on recent advances in compu…
Improves Bayesian neural networks inference efficiency and accuracy.
problem Inflexibility of factorized structure in Dropout posterior.
method Introduces Variational Structured Dropout (VSD) with orthogonal transformation.
result VSD induces adaptive regularization and better generalization.
Pseudo-Likelihood Inference improves ABC for high-dimensional Bayesian inference.
problem Intractable likelihood in Bayesian system identification.
method PLI combines neural approximation with integral probability metrics and adaptive bandwidth.
result PLI outperforms SNPE on challenging tasks, especially with more data.
This paper introduces Bayes Hilbert spaces for efficient posterior approximation.
problem Efficient posterior approximation in Bayesian models for large datasets.
method Develops Bayes Hilbert spaces for posterior approximation and connects them to Bayesian coresets and kernel-based distances.
result Bayes Hilbert spaces provide a novel framework for posterior approximation that is computationally efficient.
Corrects errors in ILA for Bayesian inference in LGMs.
problem Error in ILA for non-Gaussian likelihoods in LGMs.
method Importance sampling scheme to correct ILA errors.
result Corrected posterior converges to the true posterior with increased samples.
A new method combines VI and IS to improve Bayesian inference accuracy.
problem Bayesian inference often underestimates posterior tails, leading to miscalibration and degeneracy.
method Proposes a novel combination of optimization and sampling techniques using the forward KL divergence.
result The method guarantees asymptotic consistency and fast convergence to optimal IS and variational approximations.
TM-VI uses flexible transformation models to approximate complex posteriors in Bayesian models.
problem Approximating complex posteriors in Bayesian models with limited flexibility.
method Transformation models for variational inference (TM-VI).
result TM-VI allows accurate approximation of complex posteriors in models with one parameter and works in a mean-field fashion for multi-parameter models.
We consider Bayesian inference problems with computationally intensive likelihood functions. We propose a Gaussian process (GP) based method to approximate the joint distribution of the unknown parameters and the data. In particular, we write the joint density approximately as a product of an approximate posterior dens…
Bayesian models quantify uncertainty and facilitate optimal decision-making in downstream applications. For most models, however, practitioners are forced to use approximate inference techniques that lead to sub-optimal decisions due to incorrect posterior predictive distributions. We present a novel approach that corr…
New variational inference approach using Hilbert space for robotic state estimation.
problem Robotic state estimation with high-dimensional data.
method Variational inference reformulated in a Bayesian Hilbert space, using iterative projection.
result Variational inference can be seen as iterative projection in Euclidean space.
This paper examines how neural architectures support amortized Bayesian inference and its performance under varying conditions.
problem Understanding and evaluating amortized inference under signal-to-noise variation and distribution shift.
method Statistical analysis of neural architectures including feedforward networks, Deep Sets, and Transformers.
result Neural architectures support amortized Bayesian inference, offering controlled generalization error and robustness under varying conditions.
We consider the problem of approximate Bayesian parameter inference in non-linear state-space models with intractable likelihoods. Sequential Monte Carlo with approximate Bayesian computations (SMC-ABC) is one approach to approximate the likelihood in this type of models. However, such approximations can be noisy and c…