Bayesian methods often misinterpret data and asymptotic concepts.
problem Misunderstandings in Bayesian predictive inference.
method Discussion of two specific misunderstandings.
result Consequences of misinterpretations illustrated through examples.
Paper proves LCVB method's consistency in Bayesian posteriors and decision rules.
problem Approximating Bayesian posteriors and decision rules.
method Loss-calibrated variational Bayes (LCVB) method.
result LCVB method's consistency in both approximate posterior and decision rules.
Adjustment reduces bias in widely applicable Bayesian information criterion.
problem Overestimation of widely applicable Bayesian information criterion.
method Identified and adjusted an overestimating term in the criterion.
result Asymptotically unbiased estimator of log marginal likelihood.
Bayesian ReLU nets fix asymptotic overconfidence with infinite features.
problem Bayesian ReLU nets can be asymptotically overconfident far from training data.
method Extend finite ReLU BNNs with infinite ReLU features via a Gaussian process.
result The resulting model is asymptotically maximally uncertain far from the data.
We present and implement two algorithms for analytic asymptotic evaluation of the marginal likelihood of data given a Bayesian network with hidden nodes. As shown by previous work, this evaluation is particularly hard for latent Bayesian network models, namely networks that include hidden variables, where asymptotic ap…
Paper proves VB methods are consistent and asymptotically normal.
problem Scalable inference of posterior distributions in Bayesian statistics.
method Establishes consistency and asymptotic normality of VB methods.
result VB posterior converges to KL minimizer and variational expectation is consistent and asymptotically normal.
Bayesian UQ matches frequentist UQ for adaptively collected data.
problem Uncertainty quantification for adaptive data collection.
method Extends Bernstein-von Mises theorem to adaptively collected data.
result Bayesian UQ asymptotically matches Wald-type frequentist UQ.
The paper compares Bayesian uncertainty to MAP estimator in random features regression.
problem Comparing Bayesian uncertainty to MAP estimator in random features regression.
method Analyzing the variance of the posterior predictive distribution and comparing it to the risk of the MAP estimator.
result Asymptotic agreement between Bayesian uncertainty and MAP estimator under specific signal-to-noise ratios and sample sizes.
We study how finite Bayesian neural networks adapt their hidden representations.
problem Understanding how finite Bayesian neural networks differ from infinite ones.
method We analyze the asymptotics of learned feature kernels for various network architectures.
result The leading finite-width corrections to feature kernels have a universal form.
Decentralized Gaussian processes for multi-agent systems.
problem Scalable and flexible learning solutions for multi-agent systems.
method Asymptotically exact decentralized solution to Gaussian processes, with online Bayesian model averaging for hyperparameter selection.
result Asymptotically exact decentralized Gaussian process approximation and online Bayesian model averaging.
This work analyzes Gibbs samplers for Bayesian hierarchical models without dimensionality constraints.
problem Analyzing convergence properties of Gibbs samplers for Bayesian hierarchical models.
method Using Bayesian asymptotics and total variation mixing times, the study provides dimension-free convergence results.
result Dimension-free convergence results for Gibbs samplers targeting hierarchical models under random data-generating assumptions.
TVBO optimizes time-varying functions with asymptotically vanishing regret.
problem Understanding the asymptotic performance of TVBO for time-varying black-box functions.
method Provided upper and lower bounds for cumulative regret of TVBO algorithms.
result TVBO algorithms can achieve asymptotically vanishing regret under certain conditions.
A new method improves robustness and efficiency of Bayesian LOO-CV.
problem Computational expense and unreliability of classical LOO-CV in high-dimensional Bayesian models.
method Proposes a mixture estimator to compute Bayesian LOO-CV criteria with finite asymptotic variance.
result Improved robustness and efficiency in high-dimensional problems.
This paper addresses the issue of model selection for hidden Markov models (HMMs). We generalize factorized asymptotic Bayesian inference (FAB), which has been recently developed for model selection on independent hidden variables (i.e., mixture models), for time-dependent hidden variables. As with FAB in mixture model…
This paper is about index policies for minimizing (frequentist) regret in a stochastic multi-armed bandit model, inspired by a Bayesian view on the problem. Our main contribution is to prove that the Bayes-UCB algorithm, which relies on quantiles of posterior distributions, is asymptotically optimal when the reward dis…
Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilistic model is defined and its marginal likelihoods are evaluated to decide which clusters to merge. While BHC provides a few advantages over traditional distance-based agglomerative clustering algorithms, successive evaluatio…
New method uses diffusion models for Bayesian inverse problems.
problem Solving Bayesian inverse problems with linear-Gaussian models.
method Decoupled Diffusion Sequential Monte Carlo (DDSMC) method.
result Asymptotically exact solution demonstrated on various data types.
Develops fast inference for nonparametric Bayesian LFRM.
problem Inference in LFRM is challenging and slow.
method Small-variance asymptotics framework for nonparametric Bayesian LFRM.
result Deterministic inference algorithms are fast and competitive.
Paper revisits weighted likelihood bootstrap and extends it to loss-likelihood bootstrap.
problem Generating samples from approximate Bayesian posterior of a parametric model.
method Bayesian nonparametric model with minimising expected negative log-likelihood.
result Loss-likelihood bootstrap method for posterior sampling.
Bayesian model explains and improves black-box estimators for class distribution.
problem Calibrating probabilistic classifiers and uncertainty quantification for unlabeled data.
method Introduced a Bayesian model approximating the ground-truth generative process, using efficient MCMC sampling.
result The Bayesian model is competitive and sometimes superior to established point estimators.
Derives TAP approximation for Bayesian linear regression.
problem Log-normalizing constant of posterior distribution in high-dimensional linear regression.
method Variational representation and Thouless-Anderson-Palmer approximation.
result Proves TAP approximation for spherical prior in proportional asymptotic regime.
We extend the Bayesian Information Criterion (BIC), an asymptotic approximation for the marginal likelihood, to Bayesian networks with hidden variables. This approximation can be used to select models given large samples of data. The standard BIC as well as our extension punishes the complexity of a model according to …
Bayesian neural network achieves nearly optimal performance in Besov space.
problem Bayesian neural networks in Besov space.
method Spike-and-slab prior and shrinkage prior for posterior convergence rate.
result The posterior convergence rate is nearly minimax and adaptive to unknown smoothness.
Paper improves Bayesian inference in federated learning with new algorithm VR-FALD*.
problem Bayesian inference in federated learning with communication bottlenecks and statistical heterogeneity.
method Federated Averaging Langevin Dynamics (FALD) and VR-FALD*.
result VR-FALD* corrects client drift due to statistical heterogeneity, improving convergence.
Novel method recursively partitions sample space for density estimation.
problem Estimating complex density functions efficiently and accurately.
method Recursive partitioning of the sample space, asymptotically exact.
result Asymptotically exact approximation of any density function.
New tuning rules for Metropolis algorithms derived from Bayesian large-sample asymptotics.
problem Optimal scaling in random-walk Metropolis algorithms under realistic assumptions.
method Large-sample asymptotics to derive weak convergence results and tuning guidelines.
result Tuning guidelines consistent with previous ones when target density is product form, accounting for correlation structure.
Bayesian method for estimating ATE with robustness to model misspecification.
problem Estimating average treatment effects under unconfoundedness.
method Double robust Bayesian inference using adjusted prior and posterior distributions.
result Bayesian credible sets form asymptotically exact confidence intervals.
The problem of forecasting conditional probabilities of the next event given the past is considered in a general probabilistic setting. Given an arbitrary (large, uncountable) set C of predictors, we would like to construct a single predictor that performs asymptotically as well as the best predictor in C, on any data.…
This paper develops a new method for online density estimation from noisy data.
problem Estimating probability density function from noisy streaming data.
method Quasi-Bayesian sequential deconvolution using Newton's algorithm.
result Sequential deconvolution estimate fn with large sample asymptotic guarantees. Paper calculates the exact error of LDA models.
problem Bayesian generalization error in Latent Dirichlet Allocation (LDA).
method Theoretical analysis of learning coefficient using algebraic geometry.
result Exact asymptotic form of LDA's generalization error.
A new algorithm improves Bayesian federated learning by reducing communication overhead.
problem Bayesian federated learning constraints, including privacy, data ownership, and communication overhead.
method Proposes Quantised Langevin Stochastic Dynamics (QLSD) for Bayesian federated learning, using gradient compression and variance reduction techniques.
result Non-asymptotic and asymptotic convergence guarantees for QLSD and its improved versions.
MINTS uses a minimalist Bayesian framework to tackle multi-armed bandits with structural constraints.
problem Sequential decision-making under uncertainty with complex structural constraints.
method Minimalist Bayesian framework with profile likelihood to eliminate nuisance parameters.
result MINTS achieves near-optimal regret guarantees and adapts to unimodal structure.
The study analyzes when Bayesian averaging over decision trees is reliable.
problem When do Bayesian model averaging weights over decision trees provide reliable information?
method Closed-form solution for Bayesian decision trees with Catalan-exponential priors.
result Established a complete non-asymptotic theory of rational commitment thresholds.
New algorithm speeds Bayesian nonparametric model inference.
problem Slow inference in Bayesian nonparametric models.
method Decompose random measures into finite and infinite sub-measures; use different algorithms for each.
result Hybrid algorithm improves scalability and mixing.
Bayesian models offer great flexibility for clustering applications---Bayesian nonparametrics can be used for modeling infinite mixtures, and hierarchical Bayesian models can be utilized for sharing clusters across multiple data sets. For the most part, such flexibility is lacking in classical clustering methods such a…
Gaussian OBFS proves strong consistency in feature selection with correlations.
problem Feature selection consistency in the presence of correlations.
method Proves strong consistency of Gaussian OBFS under mild conditions.
result Identifies selected features and rates of convergence for different feature types.
New algorithms improve likelihood of finding global optima in Bayesian inference.
problem Finding global optima in Bayesian inference is difficult due to nonconvexity.
method Developed two algorithms: consistent Laplace approximation (CLA) and consistent stochastic variational inference (CSVI).
result Both CSVI and CLA improve likelihood of obtaining global optima compared to standard methods.
Efficient private Bayesian inference for sensitive data.
problem Private analysis of sensitive data.
method Private Bayesian inference in exponential families, using sufficient statistics.
result Properly calibrated posterior beliefs in non-asymptotic data.
A new method combines VI and IS to improve Bayesian inference accuracy.
problem Bayesian inference often underestimates posterior tails, leading to miscalibration and degeneracy.
method Proposes a novel combination of optimization and sampling techniques using the forward KL divergence.
result The method guarantees asymptotic consistency and fast convergence to optimal IS and variational approximations.
Sharp bounds for Dirichlet sums lead to improved Bayesian algorithm analysis.
problem Improving Bayesian algorithm performance through precise deviation bounds.
method Novel integral representation of Dirichlet sum density, Gaussian approximation, complex analysis.
result Significantly sharpened regret bounds for Multinomial Thompson Sampling.
Paper addresses trade-off between robustness and specificity in machine learning.
problem Combating distributional uncertainties in training data compared to population distributions.
method Unified framework that unifies Bayesian, distributionally robust optimization, and regularization methods.
result Reveals the trade-off between robustness and specificity.
Bayesian method improves estimation of unseen species.
problem Estimating unseen species in biological and physical sciences.
method Bayesian nonparametric approach with Pitman-Yor prior and Gaussian credible intervals.
result Improves asymptotic credible intervals for unseen species estimation.
Recent reports have described that learning Bayesian networks are highly sensitive to the chosen equivalent sample size (ESS) in the Bayesian Dirichlet equivalence uniform (BDeu). This sensitivity often engenders some unstable or undesirable results. This paper describes some asymptotic analyses of BDeu to explain the …
The paper proposes methods to estimate MCMC quality with couplings, bounding Wasserstein distance.
problem Improving MCMC efficiency without sacrificing asymptotic consistency.
method Estimators based on couplings of Markov chains to assess quality of asymptotically biased sampling methods.
result Empirical upper bounds of Wasserstein distance for assessing MCMC quality.
Proposes HDBEN for heteroscedastic regression with improved sparsity and variance modeling.
problem Violation of constant error variance in high-dimensional regression.
method HDBEN framework using hierarchical Bayesian priors with ℓ1 and ℓ2 penalties. result Achieves posterior concentration, variable selection consistency, and asymptotic normality.
Bayesian learning made scalable with posteriors library.
problem Computational challenges in Bayesian learning with modern models.
method Introducing posteriors library and tempered MCMC.
result Bayesian approximations are useful and scalable.
Bayesian optimization for composite functions improves efficiency.
problem Optimizing composite functions with expensive derivative-free evaluations.
method Exploits composite structure using multi-output Gaussian process and expected improvement for composite functions.
result Significantly improves sampling efficiency and asymptotically converges to global optimum.
Bayesian neural networks use ridgelet prior for uncertainty quantification.
problem Combining strong predictive performance with uncertainty quantification in Bayesian neural networks.
method Proposes a ridgelet prior that approximates a Gaussian process covariance function in the output space of the network.
result Establishes universality property allowing Bayesian neural networks to approximate any Gaussian process.