VISA improves inference efficiency for complex models.
problem Efficient approximate inference in computationally intensive models.
method Sequential sample-average approximations within a trust region.
result VISA achieves comparable accuracy with computational savings.
Paper proposes Walsh-Hadamard Variational Inference for efficient approximate inference in large models.
problem Over-regularization in variational inference for large models.
method Walsh-Hadamard factorization strategies to reduce parameterization, accelerate computations, and increase posterior expressiveness.
result Efficient approximate inference achieved in over-parameterized models.
Paper introduces probabilistic module interface for complex models and inference.
problem Handling complex probabilistic models with latent variables and custom inference methods.
method Develops a platform-agnostic interface for encapsulating models and inference programs, allowing sound approximate inference algorithms for networks of modules.
result Sound approximate inference algorithms can be constructed for networks of probabilistic modules.
Paper introduces a diagnostic for approximate inference methods.
problem Estimating errors in probabilistic inference algorithms, especially for approximate methods.
method Repeatedly simulate datasets from the prior and perform inference on each, estimating a symmetric KL-divergence.
result A diagnostic for approximate inference methods can be estimated using symmetric KL-divergence.
Thompson Sampling's performance degrades with approximate inference, especially under α-divergence.
problem Thompson Sampling's performance degradation due to approximate inference.
method Study of approximate inference effects on Thompson Sampling in k-armed bandit problems. result Small inference error can lead to poor performance (linear regret) in Thompson Sampling.
Meta-learn Bayesian inference for task-specific BNNs using amortised inference.
problem Efficiently learning Bayesian inference for small-scale probabilistic meta-learning.
method Replace global inducing points with actual data to create a set of approximate likelihoods, train a meta-model to learn these parameters across related datasets.
result Meta-learned inference can be applied to task-specific BNNs, improving efficiency and scalability.
Simulation-based inference methods can produce unreliable posterior approximations.
problem Reliability of simulation-based inference methods for scientific use cases.
method Benchmarked algorithms including Neural Posterior Estimation, Neural Ratio Estimation, Sequential Neural Likelihood, and Approximate Bayesian Computation.
result Ensembling posterior surrogates provides more reliable approximations.
Variational autoencoders struggle with inference quality due to recognition network limitations.
problem Inference suboptimality in variational autoencoders.
method Examined approximate inference in terms of variational distribution capacity and recognition network quality.
result Inference quality is more influenced by recognition network limitations than variational distribution complexity.
Fast approximate inference for non-Gaussian data.
problem Efficient inference for non-Gaussian data.
method Laplace Matching for fast approximate inference in latent Gaussian models.
result Achieves high approximation quality with low computational cost.
Proposes efficient Gaussian approximations for non-Gaussian likelihoods.
problem Computational challenges in learning and inference with non-Gaussian likelihoods.
method Variational inference and moment matching in transformed bases.
result Good approximation quality for binary and multiclass classification.
Paper generalizes EC inference for loopy models.
problem Probabilistic inference in loopy models.
method Generalized Expectation Consistency (GEC) method.
result GEC can be applied to MAP and MMSE estimation.
Improves hyperparameter learning in GP models with non-conjugate likelihoods.
problem Hyperparameter learning entangled with approximate inference in GP models.
method Hybrid training procedure combining VI for inference and EP-like marginal likelihood approximation for hyperparameter learning.
result Empirically demonstrates the effectiveness of the proposed training procedure across various data sets.
Develops bounds on Bayesian posterior approximations using Fisher distance.
problem Lack of finite-sample theory for scalable inference methods.
method Bounding Wasserstein distance via generalized Fisher distance.
result Derives bounds on Wasserstein error for various approximations.
Post-process Bayesian inference speeds up posterior approximation.
problem Leveraging pre-existing model evaluations for quick posterior approximation.
method Variational Sparse Bayesian Quadrature (VSBQ) using sparse Gaussian process (GP) surrogate model.
result VSBQ builds high-quality posterior approximations from existing optimization traces.
Combines Laplace approximation and variational inference for better posterior correlations.
problem Lack of posterior correlations in variational inference.
method Combines Laplace approximation and variational inference, explicitly minimising KL divergence.
result Improves over Laplace approximation and variational inference with factorised Gaussian posteriors.
Corrects approximate Bayesian inference for better decision-making.
problem Sub-optimal decisions due to inaccurate posterior predictive distributions.
method Trains a separate model to correct decision-making under approximate posterior, combining Bayesian modeling with optimization.
result Empirically demonstrates improved predictive accuracy in various problems.
Variational inference simplifies Bayesian model approximations.
problem Approximating complex Bayesian posterior distributions.
method Solving optimization problems to approximate posterior distributions with simpler variational distributions.
result Variational inference has been successfully applied in various models and large-scale applications.
New diagnostic tool for assessing approximate Bayesian inference.
problem Assessing the trustworthiness of approximate Bayesian inference.
method Reframe the problem in terms of incompatible conditional distributions and use Gibbs priors.
result The diagnostic tool can discover the inductive bias in various Bayesian models and approximations.
Fast Bayesian inference with adaptable priors for real-time applications.
problem Intractable exact posterior computation limits Bayesian inference's adoption.
method Distribution Transformer architecture that learns mappings between priors and posteriors.
result Significant reduction in computation time from minutes to milliseconds.
The paper explores how to evaluate Bayesian approximations in neural networks.
problem The difficulty in evaluating Bayesian approximations in neural networks.
method Exploring the interactions between probabilistic models, approximating distributions, optimization algorithms, and datasets.
result The expected utility of the approximate posterior can measure inference quality.
A-NeSI scales approximate inference for probabilistic neurosymbolic learning.
problem Combining neural networks with symbolic reasoning for scalable inference.
method A-NeSI: a new framework for PNL using neural networks for approximate inference.
result A-NeSI achieves scalable approximate inference without semantic changes.
Improves learning of spectral mixture kernels with approximate Bayesian inference.
problem Difficult optimization of large number of SM kernel parameters.
method Approximate Bayesian inference using variational distribution of spectral points and random Fourier features.
result Accelerates convergence and leads to better optimal parameters.
Paper quantifies error in probabilistic inference programs.
problem Estimating approximation error in probabilistic inference.
method Derives a subjective bound on KL divergence using reference and meta-inference programs.
result Validates technique on various inference problems, detects implementation bugs.
AIDE measures the accuracy of probabilistic inference algorithms.
problem Measuring the accuracy of approximate inference algorithms on specific data sets.
method AIDE is an algorithm based on viewing inference algorithms as probabilistic models and auxiliary variables.
result AIDE captures the qualitative behavior of inference algorithms and detects failure modes.
Bayesian bandit algorithms with approximate inference improve regret bounds in stochastic linear bandits.
problem Theoretical justification for Bayesian bandit algorithms with approximate inference in stochastic linear bandits.
method Proposed a theoretical framework to analyze approximate inference impact and conducted frequentist regret analysis on LinTS and LinBUCB.
result LinTS and LinBUCB preserve their original regret upper bounds with larger constant terms in approximate inference settings.
We extend natural-gradient methods to mixtures of exponential-family distributions, improving inference speed.
problem Complex, multimodal posterior distributions are difficult to approximate with simple exponential-family distributions.
method We use minimal conditional-EF representations and derive simple natural-gradient updates.
result Our natural-gradient method converges faster than black-box methods with reparameterization gradients.
A new VSMC family improves variational inference efficiency and accuracy.
problem Efficient and accurate Bayesian inference for complex models.
method Integrates variational inference and sequential Monte Carlo for flexible posterior approximation.
result VSMC family can approximate posterior arbitrarily well and optimize parameters efficiently.
Optimizes decision-making with variational Bayesian methods for continuous utilities.
problem Inference approximations for continuous utilities without full posterior knowledge.
method Automatic pipeline that co-opts continuous utilities into variational inference algorithms.
result Consistent improvement in decision-making when calibrating approximations for specific utilities.
Adaptive Gaussian process approximates Bayesian inference for costly likelihoods.
problem Bayesian inference with computationally expensive likelihood functions.
method Gaussian process approximation with active learning design points.
result Competitive performance compared to existing methods for Bayesian computation.
The paper explores efficient posterior approximation methods in variational inference.
problem Efficient model optimization with little emphasis on the choice of approximating posterior.
method Develops a rich family of approximating posteriors using transformations on distributions.
result A particular method employing transformations on distributions yields better posterior distributions.
A new variational inference method using optimal transport.
problem Approximating complex posterior distributions with flexible particle-based methods.
method Introducing a new particle-based variational inference method based on semi-discrete optimal transport.
result The method provides a particle approximation and optimal transportation densities.
This work shows how approximate reward models can significantly improve inference-time scaling.
problem Improving the efficiency of inference for large language models.
method Identifying the Bellman error of approximate reward models and using Sequential Monte Carlo (SMC) for inference.
result Approximate reward models can reduce computational complexity from exponential to polynomial in T. Boosting variational inference improves posterior approximations using mixtures of base distributions.
problem VI approximations do not converge to exact posterior due to constrained distribution family.
method Develops BVI algorithm that iteratively mixes current approximation with new components from base distribution family.
result BVI yields progressively more accurate posterior approximations, capturing multimodality and nonstandard shapes.
Adaptive approximations improve variational inference for complex models.
problem Efficiently approximate marginal distributions and partition functions in complex probabilistic models.
method Two classes of adaptive approximations that include Bethe, tree-reweighted, and convex free energies.
result Proposed approximations automatically adapt to a given model and outperform existing methods.
VIND reduces gradient variance for non-Gaussian approximations.
problem Improving Variational Inference for non-Gaussian distributions.
method Extends reparameterization trick to exponential families using numerical derivatives and tight coupling.
result Reduces gradient variance, leading to better posterior approximations.
The choice of approximate posterior distribution is one of the core problems in variational inference. Most applications of variational inference employ simple families of posterior approximations in order to allow for efficient inference, focusing on mean-field or other simple structured approximations. This restricti…
Improved Bayesian neural network inference by selectively removing redundant modes.
problem Redundant modes in Bayesian neural network posteriors complicate approximate inference.
method Structured partial stochasticity and deterministic subset selection of weights.
result Improved performance of approximate inference schemes with simplified posterior distribution.
Study compares uncertainty estimation methods for Bayesian Neural Networks.
problem Quality of uncertainty quantification in Bayesian Neural Networks.
method Empirical comparison of 10 inference methods on regression and classification tasks.
result Common inference metrics can be misleading, and methods designed to capture posterior structure do not always produce high-quality approximations.
This paper introduces a spline-based method for nonparametric ADVI that handles complex posterior distributions.
problem Learning complex posterior distributions with skewness, multimodality, and bounded support.
method Develops a spline-based nonparametric approximation approach for ADVI.
result Establishes the asymptotic consistency of the derived lower bound for importance weighted autoencoder.
Study on variational methods in Bayesian neural networks, revealing limitations and universality.
problem Understanding the quality of variational approximations in Bayesian neural networks.
method Analysis of mean-field Gaussian and Monte Carlo dropout methods in single-hidden layer ReLU BNNs and deep networks.
result Variational methods can have pathologies in estimating uncertainty, especially in deep networks.
Max-Entropy approach improves variational inference for complex posterior distributions.
problem Efficient inference with simple families vs. accuracy in variational inference.
method Greedy approximation of the posterior distribution with Max-Entropy approach.
result Demonstrated ability to capture complex multimodal posterior distributions.
ConDiSim uses diffusion models to approximate complex system posteriors efficiently.
problem Simulation-based inference of systems with intractable likelihoods.
method Conditional diffusion model with forward and reverse processes.
result Effective posterior approximation across various benchmark and real-world problems.
Improved inference via combining Rényi divergence and rejection sampling.
problem Efficiently approximate complex distributions in inference problems.
method Synergistic combination of Rényi α-divergence variational inference and rejection sampling.
result Two-stage hybrid inference algorithm yields more accurate approximations.
A new method approximates posterior for VAEs without iterative training.
problem Inference models in VAEs are poor early on, leading to suboptimal models.
method Train generative and inference models independently, using a model-agnostic posterior approximation (MAPA).
result MAPA approximates the true posterior deterministically and can improve density estimation.
Agents learn state ambiguity from non-linear sensor data using Gaussian approximations.
problem Learning state representation from non-linear sensor data.
method Second-order Taylor approximation of Gaussian distribution for non-linear measurement functions.
result Induces a preference for states based on inferability from observations.
Normalizing flow regression approximates posterior distributions without additional sampling.
problem Bayesian inference with computationally expensive likelihood evaluations.
method Normalizing flow regression (NFR) for offline inference.
result NFR yields a tractable posterior approximation through regression on existing log-density evaluations.
The paper proposes a method to approximate posterior distributions of diffusion and jump processes from discrete observations.
problem Reconstructing posterior measures over trajectories of diffusion and jump processes from discrete observations.
method Variational approximate inference applied to Bayesian framework for diffusion processes, extended to Markov jump processes.
result The method provides computationally efficient approximations for inverse problems in diffusion and jump processes.
Boosting Variational Inference improves posterior approximations with adaptive step-sizes.
problem Limited resources hinder the widespread adoption of Boosting Variational Inference.
method Characterized global curvature impact, introduced local curvature, and developed an approximate backtracking algorithm.
result New theoretical convergence rates and experimental validation demonstrate improved performance.