Posterior refinement improves sample efficiency in Bayesian neural networks.
problem Bayesian neural networks suffer from poor predictive performance due to inaccurate posterior approximations.
method Propose refining Gaussian approximate posteriors with normalizing flows to improve predictive distributions.
result Posterior refinement yields competitive predictive performance with minimal computational overhead.
New algorithms for fast online decision making using neural networks and martingale posteriors.
problem Online sequential decision making under uncertainty.
method Martingale posterior neural networks for fast online learning and decision making.
result Achieves competitive performance-speed trade-offs in non-stationary contextual bandits and Bayesian optimization.
Improved predictive posterior density estimation through optimized importance sampling.
problem Low signal-to-noise ratio in posterior predictive densities.
method Optimized importance sampling using a test-time variational proxy.
result Significantly improved estimates of predictive posterior densities.
VPR improves posterior uncertainty quantification by combining VI and predictive resampling.
problem Inaccurate posterior sampling with MCMC due to computational constraints.
method Variational predictive resampling (VPR) that uses VI's predictive strength and imputes future observations.
result VPR converges to the exact Bayesian posterior in a Gaussian location model and improves uncertainty quantification.
Bayes posterior yields worse predictions than simpler methods in deep neural networks.
problem Understanding and improving predictive performance in Bayesian deep learning.
method Careful MCMC sampling and evaluation of cold posteriors.
result Cold posteriors yield significantly better predictive performance than the true Bayes posterior.
Advocates for a new posterior that predicts better than classical and generalised Bayes.
problem Combining parameter inference and density estimation for better predictive models.
method Predictively Oriented (PrO) posterior using mean field Langevin dynamics.
result PrO posteriors converge to the predictively optimal model average, adapting to model misspecification.
PFN-TS uses Thompson sampling with PFNs to improve contextual bandit performance.
problem Improving contextual bandit performance using Thompson sampling with prior-data fitted networks.
method PFN-TS converts PFN posterior predictives into mean-reward samples using a subsampled predictive central limit theorem.
result PFN-TS achieves the best average rank across nonlinear synthetic and OpenML classification-to-bandit benchmarks.
PPT optimizes transformer behavior by steering its latent posterior using prior samples.
problem Eliciting desired behavior from transformers without backpropagation.
method Posterior Prefix Tuning (PPT) uses predictive Monte Carlo (PMC) samples and importance sampling to optimize the latent posterior.
result PPT optimizes transformer behavior without backpropagation, achieving high utility across different utility functions.
Differential privacy of Gaussian process posterior sampling
problem Privacy of posterior sample paths from Gaussian process
method Intrinsic randomness yields DP guarantees
result Intrinsic randomness yields DP guarantees
Markov chain Monte Carlo (MCMC) methods have not been broadly adopted in Bayesian neural networks (BNNs). This paper initially reviews the main challenges in sampling from the parameter posterior of a neural network via MCMC. Such challenges culminate to lack of convergence to the parameter posterior. Nevertheless, thi…
Brain uses synaptic failure to sample from posterior distributions.
problem Bayesian inference in the brain's probabilistic computations.
method Adapting synaptic failure to sample posterior predictive distributions.
result Synaptic failure enables sampling of complete posterior predictive distributions.
Recent decades have seen an interest in prediction problems for which Bayesian methodology has been used ubiquitously. Sampling from or approximating the posterior predictive distribution in a Bayesian model allows one to make inferential statements about potentially observable random quantities given observed data. Th…
PVI seeks a posterior that makes predictions closer to true data, not approximating the Bayesian posterior.
problem Finding meaningful posterior distributions under model misspecification.
method Predictive variational inference (PVI) seeks an optimal posterior density for close predictive matching to true data.
result PVI learns a posterior that is not the same as the Bayesian posterior, but is closer to the true data generating process.
FMP sampling improves model calibration without sharing data.
problem Difficulties in specifying priors for modern neural networks.
method FMP sampling replaces prior and likelihood with a predictive distribution, running the sampler centrally.
result FMP sampling significantly improves calibration over baseline methods.
We use diffusion models to sample from complex GP priors in climate data.
problem Sampling from non-stationary Gaussian process priors is computationally hard.
method Replace GP prior with a diffusion model surrogate and use training-free guidance algorithms.
result Generated distributions are close to GP priors and can be fine-tuned.
PostNet predicts uncertainty without OOD data, improving OOD detection and calibration.
problem Accurate uncertainty estimation for safe systems.
method PostNet uses Normalizing Flows to learn individual posterior distributions over predicted probabilities.
result PostNet achieves state-of-the-art results in OOD detection and uncertainty calibration.
Study improves posterior inference in neural processes with limited data.
problem Improving posterior predictive inference in probabilistic models with scarce conditioning data.
method Examined effects of pooling operators and variational families on posterior quality in neural processes.
result Novel neural process architectures lead to superior posterior predictive samples in image completion/in-painting tasks.
A new method learns posterior and predictive distributions together, reducing computational cost.
problem Sequential two-stage Bayesian inference is computationally expensive.
method Amortized variational inference targeting posterior-predictive distribution.
result Efficient online inference with more accurate predictive distributions.
New method improves posterior sampling for complex data models.
problem Sampling from posterior distributions in high-dimensional data.
method Tilted transport technique combining denoising oracle and log-likelihood.
result Boosted posterior is strongly log-concave, facilitating easier sampling.
Neighborhood sampling affects graph neural network training outcomes.
problem Understanding the impact of neighborhood sampling on graph neural network training.
method Theoretical analysis using neural tangent kernels and Gaussian processes.
result Posterior covariance differs for different neighborhood sampling approaches, indicating no dominant approach.
Bayesian neural networks reveal multimodal predictive distributions.
problem Uncertainty quantification and interpretability in neural networks.
method Discretized prior for inner layer weights, Gaussian mixture approximation of posterior predictive distribution.
result Distinct parameter realizations can produce the same training error but different posterior predictive distributions.
One of the most compelling features of Gaussian process (GP) regression is its ability to provide well-calibrated posterior distributions. Recent advances in inducing point methods have sped up GP marginal likelihood and posterior mean computations, leaving posterior covariance estimation and sampling as the remaining …
Two approaches improve conformal Bayes for label shift, one post-hoc and one in-training.
problem Improving prediction sets for target domain under label shift.
method Two complementary approaches: post-hoc calibration and in-training adaptation.
result In-training adaptation achieves up to 43% width reduction at unchanged coverage.
New method efficiently samples Gaussian process posteriors without cubic scaling.
problem Intractable posterior distributions in Gaussian processes.
method Decomposes Gaussian processes into prior and data components for scalable sampling.
result Fast posterior sampling at a fraction of the usual cost.
The paper solves the problem of optimal portfolio choice when the parameters of the asset returns distribution, like the mean vector and the covariance matrix are unknown and have to be estimated by using historical data of the asset returns. The new approach employs the Bayesian posterior predictive distribution which…
Develops variational inference for Neyman-Scott processes for faster sampling.
problem Slow mixing time in MCMC for posterior sampling in Neyman-Scott processes.
method Variational inference algorithm for Neyman-Scott processes, minimizing KL divergence.
result Achieves better prediction performance than MCMC with limited computational time.
Transformers can approximate posterior predictive distributions through in-context learning.
problem Bayesian prediction tasks, especially beyond point predictions.
method Gradient descent algorithm targeting posterior predictive mean and variance, followed by nonlinear mappings.
result Transformers can implement algorithms to approximate posterior predictive distributions.
Generative AI can solve in-context learning problems using a martingale perspective.
problem Estimating when a conditional generative model can solve an in-context learning problem.
method Bayesian interpretation, ancestral sampling, generative predictive p-value.
result Developed a method to assess the suitability of CGMs for ICL problems using generative predictive p-values.
FP-BMA improves generalization by encouraging flat posteriors in Bayesian Model Averaging.
problem Lack of flat posterior in approximate Bayesian inference methods hinders effective Bayesian Model Averaging.
method Proposes Flat Posterior-aware Bayesian Model Averaging (FP-BMA) and Flat Posterior-aware Bayesian Transfer Learning schemes.
result FP-BMA successfully captures flat posteriors, improving generalization performance.
WBCP improves conformal prediction for distribution shifts using weighted Dirichlet posteriors.
problem Handling distribution shifts in conformal prediction.
method Generalizes Bayesian Quadrature Conformal Prediction (BQ-CP) to arbitrary importance-weighted settings.
result WBCP maintains coverage guarantees while providing richer uncertainty information.
Estimating the predictive uncertainty of a Bayesian learning model is critical in various decision-making problems, e.g., reinforcement learning, detecting adversarial attack, self-driving car. As the model posterior is almost always intractable, most efforts were made on finding an accurate approximation the true post…
Adversarial robustness of amortized Bayesian inference is studied, showing it can be improved.
problem Adversarial robustness of amortized Bayesian inference.
method Simulation-based estimation, regularization scheme based on Fisher information.
result Adversarial robustness can be improved with a regularization scheme.
The paper addresses the invariance issue in Bayesian neural networks using linearized Laplace approximation.
problem Bayesian neural networks fail to maintain invariance under reparameterization, leading to different posterior densities for identical functions.
method Developed a geometric view of reparameterizations and a Riemannian diffusion process to extend reparameterization invariance to neural network predictive.
result Empirically improved posterior fit through approximate posterior sampling.
One-round FL method improves robustness and reduces communication rounds.
problem Making predictions robust and reducing FL communication rounds in heterogeneous data.
method Bayesian predictive space aggregation of client posteriors in one round.
result One-round FL method outperforms other techniques on heterogeneous settings.
Improved Bayesian FL method calibrates predictions for federated learning.
problem Overconfident predictions in Bayesian FL methods for federated learning.
method β-Predictive Bayes algorithm interpolates between mixture and product of local predictive posteriors, tuning parameter β for better calibration.
result Demonstrated superior calibration compared to other baselines, even with increased data heterogeneity.
Proposes a method to quantify uncertainty in PFNs.
problem Lack of uncertainty quantification in PFNs.
method Martingale posteriors for efficient, tuning-free sampling.
result Proves convergence of proposed sampling procedure.
Bayesian framework for semiparametric regression of discrete data.
problem Complex distributional features of discrete data.
method Semiparametric modeling with nonparametric marginal and latent linear regression.
result Posterior consistency and analytical/posterior predictive distributions.
An infinite parallel tempering bouncy particle sampler improves sampling efficiency for multimodal distributions.
problem Sampling from complex posterior distributions with high accuracy and efficiency.
method Introduced an infinite parallel tempering bouncy particle sampler (BPS-PT) to accelerate convergence.
result Demonstrated improved sampling efficiency for multimodal distributions through numerical simulations.
This paper distills Bayesian posterior expectations for deep neural networks.
problem Improving deep neural network performance and uncertainty quantification.
method Develops a framework for distilling expectations from Bayesian posterior distributions using Monte Carlo samples.
result The framework successfully distills posterior predictive distribution and expected entropy.
Conformal Bayes under label shift: post-hoc calibration vs. in-training adaptation
problem Bayesian prediction sets under label shift
method Post-hoc calibration vs. In-training adaptation
result Both strategies achieve valid coverage equally in an unbiased training regime
This work introduces a method for visualizing high-dimensional posteriors using hierarchical tree-valued predictions.
problem Visualizing high-dimensional posterior distributions for complex problems like image restoration.
method A neural network predicts a tree-valued hierarchical summarization of the posterior distribution in a single forward pass.
result The method efficiently summarizes and visualizes posteriors, achieving comparable results to hierarchical clustering but at a much faster speed.
Improves Bayesian predictive performance in misspecified models.
problem Misspecification gap between inferential and predictive risks.
method Develops a multi-sample loss (PACm) to bridge the gap. result Empirical study shows improved predictive distribution.
Many statistical models can be simulated forwards but have intractable likelihoods. Approximate Bayesian Computation (ABC) methods are used to infer properties of these models from data. Traditionally these methods approximate the posterior over parameters by conditioning on data being inside an ε-ball around the obs…
A method predicts posterior PCs for faster uncertainty quantification in imaging.
problem Uncertainty visualization in image restoration models is limited by per-pixel variances.
method Neural Posterior Principal Components (NPPC) method for predicting PCs in a single forward pass.
result Orders of magnitude faster uncertainty quantification compared to posterior samplers.
Itô maps provide a method for any-step SDE integration.
problem Stochastic dynamics
method Itô map formulation
result Empirical results on synthetic and image-generation benchmarks
Bayesian ODEs with Gaussian processes infer unknown dynamics from data.
problem Estimating unknown continuous-time system dynamics from data.
method Bayesian nonparametric model using Gaussian processes, sparse variational inference, probabilistic shooting.
result Posterior predictive uncertainty scores outperform alternative methods on multiple ODE learning tasks.
New method samples from LLM posterior for coherent, useful responses.
problem Hallucinations in large language models.
method Posterior sampling for conditional generation, with calibration.
result Achieves statistical guarantees with higher downstream utility.
Bayesian neural learning feature a rigorous approach to estimation and uncertainty quantification via the posterior distribution of weights that represent knowledge of the neural network. This not only provides point estimates of optimal set of weights but also the ability to quantify uncertainty in decision making usi…