This work provides statistical guarantees for VAEs using PAC-Bayesian theory.
problem Theoretical properties of VAEs remain open questions.
method PAC-Bayesian theory to derive statistical guarantees.
result Upper bounds on Wasserstein distance between input and generative model.
Two batch Bayesian optimization algorithms with regret guarantees.
problem Efficiently optimizing multiple objectives in batch feedback settings.
method Gaussian process upper confidence bound and Thompson sampling approaches.
result Frequentist regret guarantees and numerical results.
Nonlinear MCMC improves Bayesian machine learning sampling.
problem Sampling problems in Bayesian machine learning.
method Nonlinear MCMC technique with convergence guarantees.
result Improves sampling in Bayesian neural networks.
Bayesian quadrature improves conformal prediction for better risk assessment.
problem Improving risk assessment for machine learning models.
method Revisiting conformal prediction from a Bayesian perspective and proposing Bayesian quadrature.
result Provides interpretable guarantees and a richer representation of likely losses.
Bayesian neural networks use temperature adjustments to improve predictive performance.
problem Lack of theoretical generalization guarantees for Bayesian neural networks.
method Temperature adjustments to balance likelihood and prior regularization.
result Improved predictive performance through temperature adjustments.
Bayesian optimization enhanced with conformal prediction for better outcome reliability.
problem Uncertainty and model misspecification in Bayesian optimization.
method Conformal prediction to provide coverage guarantees and Bayesian optimization to select queries.
result Significant improvement in query coverage without sacrificing sample-efficiency.
Unified analysis for nonlinear parametric models in Bayesian optimization.
problem Limited theoretical guarantees for nonlinear parametric models in Bayesian optimization.
method Kernel-based framework for analyzing regularized nonlinear parametric models trained on adaptively collected data.
result Unified convergence guarantees for nonlinear acquisition and surrogate models.
New approach for off-policy learning in contextual bandits with performance guarantees.
problem Improving performance of logging policies in contextual bandits.
method PAC-Bayesian analysis of policy mixtures, providing tighter generalization bounds and tractable optimization algorithms.
result Proved tighter generalization bounds and demonstrated effectiveness in practical scenarios.
The paper analyzes distributed Bayesian inference and its Frequentist guarantees.
problem Analyzing large decentralized datasets with distributed Bayesian inference.
method Establishes Frequentist properties for distributed (non-)Bayesian inference.
result Distributed Bayesian inference retains parametric efficiency and enhances robustness.
Bayesian MAML outperforms MAML in meta learning tasks with theoretical guarantees.
problem Theoretical understanding of Bayesian MAML's superiority over MAML.
method Comparison of meta test risks between Bayesian MAML and MAML in meta linear regression.
result Bayesian MAML has provably lower meta test risks than MAML in both distribution agnostic and linear centroid cases.
WBCP improves conformal prediction for distribution shifts using weighted Dirichlet posteriors.
problem Handling distribution shifts in conformal prediction.
method Generalizes Bayesian Quadrature Conformal Prediction (BQ-CP) to arbitrary importance-weighted settings.
result WBCP maintains coverage guarantees while providing richer uncertainty information.
FA-HMC improves Bayesian federated learning with rigorous guarantees.
problem Parameter estimation and uncertainty quantification in non-iid distributed data.
method Federated Averaging stochastic Hamiltonian Monte Carlo (FA-HMC) with convergence guarantees.
result FA-HMC achieves better convergence and communication efficiency than existing methods.
New algorithms minimize PAC-Bayesian C-Bound for majority voting, leading to scalable and accurate predictors.
problem Improving majority vote classifiers using PAC-Bayesian bounds.
method Directly optimizing PAC-Bayesian guarantees on the C-Bound with gradient descent.
result Self-bounding majority vote learning algorithms with scalable and accurate predictors.
Bayesian Beta regression for proportions in high dimensions with theoretical guarantees.
problem Modeling bounded continuous responses in high-dimensional settings with theoretical guarantees.
method Proposes a Bayesian approach using a tempered posterior with Horseshoe prior for shrinkage and variable selection.
result Demonstrates improved estimation accuracy and model interpretability in high-dimensional scenarios.
Pseudo-Bayesian Optimization improves black-box function optimization using simple local regression.
problem Optimizing expensive black-box functions with uncertainty quantification.
method Axiomatic framework for convergence guarantees, combined with simple local regression and randomized prior.
result Empirically superior algorithms outperform state-of-the-art benchmarks in various applications.
Proposes a new PBO method with theoretical guarantees.
problem Optimizing expensive functions in parallel with noisy evaluations.
method Randomized Kriging Believer (KB) based on KB heuristic.
result Randomized KB achieves Bayesian expected regret guarantees.
PACOH improves meta-learning with theoretical guarantees and practical efficiency.
problem Meta-learning's generalization to unseen tasks is poorly understood, especially with limited meta-training tasks.
method PAC-Bayesian framework for deriving generalization bounds and developing PAC-optimal meta-learning algorithms.
result PACOH yields state-of-the-art performance in predictive accuracy and uncertainty estimation.
New PAC-Bayesian bounds provide practical guarantees for neural networks.
problem Loose derandomization step in PAC-Bayesian bounds for deterministic models.
method Introduce disintegrated PAC-Bayesian bounds for deterministic models.
result Significant practical improvement over state-of-the-art bounds.
Bayesian classifier improves robustness with optimistic score ratio.
problem Limited information on class-conditional distribution.
method Optimistic score ratio for robust binary classification.
result Bayesian classifier using optimistic score ratio is robust and computationally tractable.
Efficiently learns polytrees with known skeleton in polynomial time and sample complexity.
problem Learning polytrees with known skeleton structure.
method Proposes an efficient algorithm for learning d-polytrees in polynomial time and sample complexity when the skeleton is known. result Establishes finite-sample guarantees for efficient learning of d-polytrees. Unified approach to Bayesian inference with guarantees on covariance matrices.
problem Approximate Bayesian inference with PSD guarantees.
method Bayes-Newton methods extending Newton's method for optimisation.
result Novel algorithms with PSD covariance matrices.
The study provides statistical guarantees for Bayesian variational boosting.
problem Statistical and convergence issues in variational boosting.
method Proposed a novel variational family and a functional Frank-Wolfe optimization algorithm.
result Demonstrated stochastic boundedness and provided convergence rate for boosting iterates.
BOE reformulates BO as a classifier for scalable batch optimisation.
problem Scalable batch optimisation of expensive functions.
method Reformulates BO as density-ratio estimation, removing need for explicit function prior.
result Theoretical guarantees and improved uncertainty estimates for batch optimisation.
The new field of adaptive data analysis seeks to provide algorithms and provable guarantees for models of machine learning that allow researchers to reuse their data, which normally falls outside of the usual statistical paradigm of static data analysis. In 2014, Dwork, Feldman, Hardt, Pitassi, Reingold and Roth introd…
Survey and compare PAC-Bayes bounds for bandit problems.
problem Designing and evaluating bandit algorithms with strong performance guarantees.
method PAC-Bayes bounds applied to bandit problems.
result PAC-Bayes bounds useful for offline bandit algorithms, but loose for online algorithms.
Bayesian model explains and improves black-box estimators for class distribution.
problem Calibrating probabilistic classifiers and uncertainty quantification for unlabeled data.
method Introduced a Bayesian model approximating the ground-truth generative process, using efficient MCMC sampling.
result The Bayesian model is competitive and sometimes superior to established point estimators.
New risk measures control subgroup imbalances, improving PAC-Bayesian bounds.
problem Insufficient risk bounds for subgroup imbalances in data.
method Introduce constrained f-entropic risk measures and derive PAC-Bayesian bounds.
result First disintegrated PAC-Bayesian guarantees beyond standard risks.
Bayesian framework learns latent preference archetypes for many-objective optimization.
problem Expanding space of trade-offs and context-dependent human values.
method Dirichlet-process mixture model for latent preference archetypes, hybrid queries for efficient information.
result Mixture-aware Bayesian optimization outperforms standard methods on synthetic and real-world benchmarks.
Adaptive Bayesian quadrature (ABQ) is a powerful approach to numerical integration that empirically compares favorably with Monte Carlo integration on problems of medium dimensionality (where non-adaptive quadrature is not competitive). Its key ingredient is an acquisition function that changes as a function of previou…
Traditional differential privacy is independent of the data distribution. However, this is not well-matched with the modern machine learning context, where models are trained on specific data. As a result, achieving meaningful privacy guarantees in ML often excessively reduces accuracy. We propose Bayesian differential…
SBIC improves real-time online binary classification from crowdsourced data.
problem Inferring ground truth from noisy crowdsourced data.
method Streaming Bayesian Inference for Crowdsourcing (SBIC).
result SBIC has low complexity, high accuracy, and provable guarantees.
New algorithm learns Bayesian networks robustly in nearly-linear time.
problem Learning Bayesian networks with adversarially corrupted samples.
method Developed a nearly-linear time algorithm connecting robust learning to robust mean estimation.
result First nearly-linear time algorithm with comparable error guarantees.
Recent developments in differentially private (DP) machine learning and DP Bayesian learning have enabled learning under strong privacy guarantees for the training data subjects. In this paper, we further extend the applicability of DP Bayesian learning by presenting the first general DP Markov chain Monte Carlo (MCMC)…
Novel Bayesian approach for non-stationary linear contextual bandits.
problem Non-stationary linear contextual bandits.
method Weighted Sequential Bayesian (WSB) inference.
result Established frequentist regret guarantees for new algorithms.
Proposes a new algorithm for Sparse Bayesian Learning connected to Stepwise Regression.
problem Sparse Bayesian Learning for probabilistic models.
method Coordinate ascent algorithm (RMP) for SBL, showing connection to Stepwise Regression.
result RMP's noise variance parameter limit connects to Stepwise Regression, with derived guarantees.
Bayesian optimization is a powerful tool for fine-tuning the hyper-parameters of a wide variety of machine learning models. The success of machine learning has led practitioners in diverse real-world settings to learn classifiers for practical problems. As machine learning becomes commonplace, Bayesian optimization bec…
The study analyzes when Bayesian averaging over decision trees is reliable.
problem When do Bayesian model averaging weights over decision trees provide reliable information?
method Closed-form solution for Bayesian decision trees with Catalan-exponential priors.
result Established a complete non-asymptotic theory of rational commitment thresholds.
The automation of posterior inference in Bayesian data analysis has enabled experts and nonexperts alike to use more sophisticated models, engage in faster exploratory modeling and analysis, and ensure experimental reproducibility. However, standard automated posterior inference algorithms are not tractable at the scal…
Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
Safe Bayesian Optimization algorithms are improved to ensure safety in real-world applications.
problem Ensuring safety in Bayesian Optimization algorithms for real-world applications.
method Investigated and improved three safety-related issues of SafeOpt-type algorithms: frequentist uncertainty bounds, RKHS norm assumptions, and discrete search spaces.
result Introduced Real-{eta}-SafeOpt, Lipschitz-only Safe Bayesian Optimization (LoSBO), and Lipschitz-only GP-UCB (LoS-GP-UCB) algorithms that retain safety guarantees and superior performance.
PAC-Bayesian bounds for MLPs with cross entropy loss validated.
problem Generalization bounds for MLPs with cross entropy loss.
method Introduced probabilistic explanations and proved PAC-Bayesian bounds using ELBO.
result MLPs with cross entropy loss inherently guarantee PAC-Bayesian generalization bounds.
A new Bayesian approach to linear system identification has been proposed in a series of recent papers. The main idea is to frame linear system identification as predictor estimation in an infinite dimensional space, with the aid of regularization/Bayesian techniques. This approach guarantees the identification of stab…
New algorithm optimizes Bayesian network learning from Gaussian data.
problem Learning Bayesian networks from Gaussian observational data.
method Proposes a coordinate descent algorithm for ℓ0-penalized maximum likelihood estimation. result The algorithm converges to a coordinate-wise minimum and achieves optimal objective value as sample size increases.
Bayesian inference and Gaussian processes are widely used in applications ranging from robotics and control to biological systems. Many of these applications are safety-critical and require a characterization of the uncertainty associated with the learning model and formal guarantees on its predictions. In this paper w…
This work makes federated Bayesian learning differentially private.
problem Privacy concerns in federated learning with diverse data and computational constraints.
method Modified Partitioned Variational Inference (PVI) to ensure differential privacy.
result Moderately private logistic regression models can be learned in the federated setting with similar performance to non-privately trained models.
Improved VB algorithm for high-dimensional logistic regression with theoretical guarantees.
problem Sparse high-dimensional logistic regression model selection.
method Spike and slab variational Bayes approximation.
result Optimal convergence rates in ℓ2 and prediction loss for sparse truths. Gaussian Processes (GPs) are a generic modelling tool for supervised learning. While they have been successfully applied on large datasets, their use in safety-critical applications is hindered by the lack of good performance guarantees. To this end, we propose a method to learn GPs and their sparse approximations by d…
New framework for Bayesian inference using neural Schrödinger-Föllmer flows.
problem Approximate Bayesian inference in large datasets.
method Stochastic control, Schrödinger bridges, SDE-based models.
result Advocates stochastic control as a finite time and low variance alternative to SGLD.