Flow Annealing Posterior Sampling unifies stochastic-process regression and PDE inverse problems.
problem Function-space posterior sampling for stochastic processes and inverse problems.
method Flow Annealing Posterior Sampling (FAPS) using pretrained function-space flow-matching priors.
result Coherent posterior samples with accurate uncertainty quantification.
FTIP uses normalizing flows to improve posterior inference in function space.
problem Challenges in posterior inference with implicit-process priors.
method FTIP uses normalizing flows to define a richer variational distribution over combination weights.
result FTIP captures asymmetric and multimodal posterior structure better than Gaussian coefficient approximations.
This paper introduces Bayes Hilbert spaces for efficient posterior approximation.
problem Efficient posterior approximation in Bayesian models for large datasets.
method Develops Bayes Hilbert spaces for posterior approximation and connects them to Bayesian coresets and kernel-based distances.
result Bayes Hilbert spaces provide a novel framework for posterior approximation that is computationally efficient.
Geometric framework analyzes bias in variational inference for posterior functionals.
problem Analyzing the bias of posterior functionals under variational approximations.
method Developed a geometric framework to evaluate the bias of posterior functionals using the variational tangent space.
result The leading-order bias of a posterior functional is determined by its component orthogonal to the variational tangent space.
Implied posterior probability of a given model (say, Support Vector Machines (SVM)) at a point x is an estimate of the class posterior probability pertaining to the class of functions of the model applied to a given dataset. It can be regarded as a score (or estimate) for the true posterior probability, which ca…
Theoretical framework for M-posteriors connects Bayesian and frequentist statistics.
problem Connecting Bayesian and frequentist approaches in statistical inference.
method Developed a theoretical framework for M-posteriors, showing asymptotic normality and frequentist consistency.
result M-posteriors are robust and contract around M-estimators under mild conditions.
Sparse matrices simplify computation of GP variances and likelihoods.
problem Efficient computation of posterior variance and log-likelihood for additive Matérn GPs.
method Represented posterior mean, variance, log-likelihood, and gradient using sparse matrices.
result Efficient computation of posterior mean, variance, log-likelihood, and gradient in O(nlogn) time. New method controls posterior collapse in VAEs without network architecture constraints.
problem Posterior collapse in VAEs reduces diversity of generated samples.
method Introduces Latent Reconstruction (LR) loss to control posterior collapse.
result Controls posterior collapse on various datasets without architectural constraints.
Posterior sampling-based EI achieves sublinear regret bounds for expensive function optimization.
problem Theoretical analysis of expected improvement (EI) in Bayesian optimization.
method Randomized posterior sampling of EI.
result Achieves sublinear Bayesian cumulative regret bounds.
We extend diffusion models to function spaces and introduce a new method for sampling from posterior distributions.
problem Sampling from posterior distributions in infinite-dimensional function spaces using diffusion models.
method Infinite-dimensional extension of Doob's h-transform, Supervised Guidance Training for efficient sampling. result We prove that diffusion models can be conditioned to sample from posterior distributions and introduce a simulation-free score matching objective.
Study compares chi-squared divergence and KL-divergence posteriors for PAC-Bayesian bounds.
problem Investigates optimal posteriors for PAC-Bayesian bounds using chi-squared divergence.
method Analyzes bounds for three distance functions, derives FP equations for computation.
result Chi-squared divergence based posteriors have weaker bounds and worse test errors.
Bayesian method with Gaussian process priors achieves optimal convergence rates for regression function and its derivatives.
problem Estimating the regression function and its derivatives in nonparametric regression.
method Bayesian approach with Gaussian process priors, focusing on convergence rates and plug-in property.
result Equivalence of convergence rates of posterior distributions and Bayes estimators for regression function and its derivatives.
We prove exact BNN posterior convergence to GP limit and provide sampling methods.
problem Theoretical and empirical challenges in obtaining exact posterior distributions of wide BNNs.
method Theoretical proof and rejection sampling for generating exact samples.
result Exact BNN posterior converges to GP limit as width increases.
New method prevents posterior collapse in iVAE models.
problem Posterior collapse in iVAE models where observations and ICs are independent given covariates.
method Developed CI-iVAE by considering a mixture of encoder and posterior distributions in the objective function.
result Prevents posterior collapse, resulting in latent representations with more information of the observations.
Posterior sampling from diffusion models is computationally hard.
problem Sampling from posterior distributions in diffusion models is intractable.
method Analyzes the computational complexity of posterior sampling in diffusion models.
result Posterior sampling is computationally intractable under cryptographic assumptions.
New GP-based method improves uncertainty quantification for causal functions.
problem Challenges in quantifying uncertainty for causal effects, especially for entire functions.
method GP-based approach using inner-product of observational functions in RKHS, with tractable posterior moments and calibration.
result Improves uncertainty quantification while maintaining causal effect estimation performance.
Meta-learning improves Bayesian causal discovery by sampling from the posterior.
problem Difficulty in estimating the full posterior over causal structures due to large number of possible graphs and functional relationships.
method Proposes a Bayesian meta-learning model that encodes key properties of the posterior and allows for sampling causal structures.
result Meta-Bayesian causal discovery allows for reliable sampling from the posterior over causal structures.
New research shows CPE only occurs when Bayesian posterior underfits.
problem Model misspecification leading to CPE under perfect model specification.
method Theoretical analysis of Bayesian posterior and underfitting.
result No CPE if there is no underfitting of the Bayesian posterior.
Bayesian neural networks achieve optimal posterior contraction rates in Besov spaces with intrinsic dimensionality.
problem High-dimensional structured estimation problems with unknown smoothness levels.
method Sparse Bayesian neural networks with either sparse or continuous shrinkage priors.
result Optimal posterior contraction rates are achieved, adapting to the unknown smoothness level of the true function.
Study adaptive sensing of Cox processes using posterior sampling and positive bases.
problem Adaptive sensing of Cox point processes with intensity function modeling.
method Model intensity function as truncated Gaussian process in positive basis, use Langevin dynamics and posterior sampling.
result Demonstrated improved sensing compared to classical Bayesian experimental design.
Bayesian method improves approximate model posteriors.
problem Poor uncertainty quantification in approximate Bayesian inference.
method Optimizing a transformation of the approximate posterior to maximize a scoring rule.
result Significant reduction in bias and improvement in posterior coverage properties.
Bayesian neural networks with data augmentation show a persistent cold posterior effect.
problem Understanding the cold posterior effect in Bayesian neural networks with data augmentation.
method Developed principled Bayesian neural networks using data augmentation, providing exact likelihoods and tight bounds.
result The cold posterior effect persists even in models incorporating data augmentation, suggesting it's not an artifact.
Bayesian neural networks explore rare fluctuations for better feature learning.
problem Understanding rare but dominant fluctuations in Bayesian neural networks.
method Large-deviation theory and joint optimization over predictors and internal kernels.
result Posterior rate function optimization reveals data-dependent kernel selection.
Function-space MAP estimation leads to better generalization and robustness.
problem The mismatch between parameter posterior and function posterior in model training.
method Directly estimating the most likely function implied by the model and data.
result Function-space MAP estimation can lead to flatter minima, better generalization, and improved robustness.
Increasingly complex datasets pose a number of challenges for Bayesian inference. Conventional posterior sampling based on Markov chain Monte Carlo can be too computationally intensive, is serial in nature and mixes poorly between posterior modes. Further, all models are misspecified, which brings into question the val…
New method optimizes Bayesian optimization for high-dimensional posterior samples.
problem Difficult inner-loop optimization of posterior sample paths in Bayesian optimization.
method Global rootfinding approach with carefully selected starting points.
result The method discovers the global optimum most of the time with just one starting point per set.
This work explores function-space inference using KL divergence and proposes Bayesian linear regression as a benchmark.
problem Approximating the predictive posterior distribution of Bayesian models without parameter posterior approximation.
method Employing Kullback-Leibler divergence and proposing featurized Bayesian linear regression as a benchmark.
result Minimizing KL divergence leads to an ill-defined objective function, highlighting limitations of this approach.
SNPLA uses normalizing flows for efficient inference in implicit models.
problem Efficient inference in implicit models with complex likelihood and posterior learning.
method Sequential Neural Posterior and Likelihood Approximation (SNPLA) algorithm using normalizing flows.
result SNPLA achieves competitive performance with faster posterior draws compared to MCMC methods.
A new algorithm optimizes Gaussian process posterior mean functions efficiently.
problem Optimizing Gaussian process posterior mean functions over hyperrectangles is challenging due to nonlinearity and nonconvexity.
method PALM-Mean, a piecewise-analytic lower-bounding framework embedded in reduced-space spatial branch-and-bound.
result PALM-Mean improves scalability for large datasets compared to general-purpose solvers.
The paper develops algorithms for competitive RL in partially observable MGs.
problem Challenges in reinforcement learning with function approximation and partial observability.
method Proposes posterior sampling methods for self-play and adversarial learning in zero-sum MGs.
result Developed algorithms achieve low regret bounds scaling sublinearly with GEC and episode number.
New method uses fractional posteriors for semiparametric inference with improved uncertainty quantification.
problem Semiparametric inference with nonparametric priors and fractional posteriors.
method Established a general Bernstein--von Mises theorem for fractional posterior distributions, proposed shifted-and-rescaled credible sets.
result Fractional posterior credible sets provide reliable uncertainty quantification but have inflated size; shifted-and-rescaled set is an efficient confidence set.
In the popular approach of "Bayesian variable selection" (BVS), one uses prior and posterior distributions to select a subset of candidate variables to enter the model. A completely new direction will be considered here to study BVS with a Gibbs posterior originating in statistical mechanics. The Gibbs posterior is con…
A new method approximates the exact posterior score for diffusion models.
problem Training-free guidance of diffusion models for image restoration and inverse problems.
method Presented a novel expression for the exact posterior score, leveraging it to compute step sizes on the fly.
result Demonstrated competitive performance with fewer time steps compared to state-of-the-art techniques.
New decision-theoretic characterization separates belief and decision posteriors.
problem Understanding the conditions under which loss-based updating coincides with Bayesian updating.
method Decision-theoretic approach to distinguish belief and decision posteriors.
result Generalized Bayes coincides with ordinary Bayesian updating only if the loss is proportional to negative log-likelihood.
A new machine learning method for Bayesian inverse problems in function spaces.
problem Bayesian inverse problems in function spaces with incompatibility of white noise sources.
method One-step generative transport with amortized neural operator and prior-aligned Gaussian random field.
result Generative operator trained on prior samples and noisy observations generates posterior samples efficiently.
We use neural networks to estimate complex model posteriors efficiently.
problem Intractable likelihood functions in complex models.
method Train a neural network to map data to posterior distributions of model parameters.
result Our method converges to true posteriors in Kullback-Leibler divergence.
Bayesian inference for Levy density with Gibbs posterior in discrete sampling.
problem Inference on Levy density for financial models with jumps.
method Gibbs posterior framework using a loss function for intractable likelihood.
result Gibbs posterior achieves nearly optimal rate of convergence under certain conditions.
SDG uses optimal control to improve classifier guidance in low-density regions.
problem Inefficient guidance in low-density regions of posterior distributions.
method Integrates stochastic optimal control with Stein variational inference to compute the steepest descent direction.
result SDG improves guidance in low-density regions, outperforming standard methods.
New RL algorithm handles delayed feedback with posterior sampling.
problem Challenges of delayed feedback in reinforcement learning with linear function approximation.
method Posterior sampling with delayed feedback for value-based RL.
result Achieves optimal regret guarantee with improved computational efficiency.
Bayesian nonparametric LABS model adapts to function smoothness in Besov spaces.
problem Estimating functions with unknown smoothness in Besov spaces.
method Lévy Adaptive B-spline (LABS) regression model with automatic smoothness adaptation.
result LABS posterior contracts around true function in Besov classes at nearly minimax-optimal rates.
Method reformulates constrained optimization as latent space inference.
problem Optimizing black-box functions with hard constraints.
method Posterior inference in latent space using flow-based models and diffusion models.
result Method achieves superior performance across various tasks.
The paper addresses the invariance issue in Bayesian neural networks using linearized Laplace approximation.
problem Bayesian neural networks fail to maintain invariance under reparameterization, leading to different posterior densities for identical functions.
method Developed a geometric view of reparameterizations and a Riemannian diffusion process to extend reparameterization invariance to neural network predictive.
result Empirically improved posterior fit through approximate posterior sampling.
A new method improves uncertainty quantification in Bayesian inference.
problem Poor uncertainty quantification in traditional Gibbs posteriors.
method Sequential Gibbs posteriors with a Bernstein-von Mises theorem.
result Sequential Gibbs posteriors provide better frequentist coverage.
Gaussian process (GP) models form a core part of probabilistic machine learning. Considerable research effort has been made into attacking three issues with GP models: how to compute efficiently when the number of data is large; how to approximate the posterior when the likelihood is not Gaussian and how to estimate co…
Loss-calibrated EP improves Bayesian decision-making by focusing on utility-sensitive posterior approximations.
problem Bayesian decision-making under asymmetric utility functions.
method Loss-calibrated expectation propagation (Loss-EP) that tilts the posterior towards higher utility decisions.
result Loss-EP can capture useful information for decision-making under asymmetric penalties.
Efficient MCMC sampling in Bayesian neural networks by exploiting symmetries.
problem Challenges in Bayesian inference due to high-dimensional, multi-modal posterior density landscapes.
method Exploiting symmetries in the posterior landscape to restrict the parameter space and derive an upper bound on Monte Carlo chains.
result Efficient sampling is possible, offering a promising path for accurate uncertainty quantification in deep learning.
Bayesian approach reduces FL communication cost by one-shot.
problem High communication cost in optimization-based FL for high-dimensional models.
method Bayesian pseudocoresets and function-space inference for one-shot FL.
result Achieves prediction performance competitive to state-of-the-art with up to 2 orders of magnitude reduction in communication cost.
New perspective on federated learning as posterior inference, improving optimization.
problem Optimizing global models in distributed learning settings.
method Formulated as posterior inference problem, using MCMC for approximate inference and federated averaging for refinement.
result Federated posterior averaging (FedPA) outperforms existing methods on benchmarks.