Bayesian model averaging improves causal effect estimation by averaging over multiple models.
problem Estimating causal effects under linear Structural Causal Models (SCMs).
method Bayesian model averaging using Gaussian scale mixture distributions for computational efficiency.
result Bayesian model averaging is optimal for causal effect estimation.
The study analyzes when Bayesian averaging over decision trees is reliable.
problem When do Bayesian model averaging weights over decision trees provide reliable information?
method Closed-form solution for Bayesian decision trees with Catalan-exponential priors.
result Established a complete non-asymptotic theory of rational commitment thresholds.
FP-BMA improves generalization by encouraging flat posteriors in Bayesian Model Averaging.
problem Lack of flat posterior in approximate Bayesian inference methods hinders effective Bayesian Model Averaging.
method Proposes Flat Posterior-aware Bayesian Model Averaging (FP-BMA) and Flat Posterior-aware Bayesian Transfer Learning schemes.
result FP-BMA successfully captures flat posteriors, improving generalization performance.
Bayesian model estimates treatment effects near cutoffs in regression discontinuity designs.
problem Estimating conditional average treatment effects in regression discontinuity designs.
method Develops a Bayesian additive regression tree (BART) model with linear leaf-level regressions.
result Adapts to different slopes on the running variable near the cutoff, providing interpretable inference.
Bayesian framework mixes imperfect models for improved predictions.
problem Improving predictions of complex computational models in unknown domains.
method Local Bayesian Dirichlet mixing of imperfect models using the Dirichlet distribution.
result Global and local mixtures of models achieve excellent performance in prediction accuracy and uncertainty quantification.
Bayesian model averaging fails under covariate shift, affecting neural networks' performance.
problem Bayesian model averaging's failure in neural networks under covariate shift.
method Explained the issue and proposed novel priors to improve robustness.
result Bayesian model averaging is problematic under covariate shift, especially with linear feature dependencies.
FBMS R package simplifies Bayesian model selection and averaging.
problem Complex regression settings with multi-modal posterior landscapes.
method Efficient MJMCMC and GMJMCMC algorithms for Bayesian model exploration.
result FBMS effectively handles Bayesian generalized linear and nonlinear models.
We study the problem of learning Bayesian network structures from data. We develop an algorithm for finding the k-best Bayesian network structures. We propose to compute the posterior probabilities of hypotheses of interest by Bayesian model averaging over the k-best Bayesian networks. We present empirical results on s…
BayesBlend blends multiple models' predictions for better insurance loss predictions.
problem Improving insurance loss predictions by combining multiple models.
method Pseudo-Bayesian model averaging, stacking, and hierarchical stacking.
result BayesBlend provides a user-friendly way to blend model predictions and estimate weights.
Bayesian stacking improves model performance with varying model weights.
problem Improving model predictions with heterogeneous input performance.
method Bayesian hierarchical stacking with varying model weights inferred via Bayesian inference.
result Hierarchical stacking yields better predictions than linear averaging.
Bayesian model averaging under predictor redundancy
problem Reporting Bayesian model averaging posterior without changing the Bayesian target
method Using hard or soft regions of support space
result Region reports often give shorter and clearer summaries while preserving the main posterior information
clusterBMA combines clustering results from multiple models using Bayesian model averaging.
problem Uncertainty in model selection for clustering.
method Bayesian model averaging to combine results from multiple clustering algorithms.
result ClusterBMA offers probabilistic cluster allocations and quantifies model-based uncertainty.
Revises Bayesian model averaging for foundation models.
problem Ensemble pre-trained and lightly-finetuned foundation models for improved classification performance.
method Introduces trainable linear classifiers and computationally cheaper model averaging scheme (OMA).
result Ensembled models can better predict on various datasets.
In mixture model-based clustering applications, it is common to fit several models from a family and report clustering results from only the `best' one. In such circumstances, selection of this best model is achieved using a model selection criterion, most often the Bayesian information criterion. Rather than throw awa…
In this study, we present a multi-class graphical Bayesian predictive classifier that incorporates the uncertainty in the model selection into the standard Bayesian formalism. For each class, the dependence structure underlying the observed features is represented by a set of decomposable Gaussian graphical models. Emp…
A Bayesian network is a widely used probabilistic graphical model with applications in knowledge discovery and prediction. Learning a Bayesian network (BN) from data can be cast as an optimization problem using the well-known score-and-search approach. However, selecting a single model (i.e., the best scoring BN) can b…
IA-BMA adapts model weights to inputs for better predictions.
problem Predicting with multiple models in heterogeneous settings.
method Input adaptive Bayesian Model Averaging (IA-BMA) with an input adaptive prior and amortized variational inference.
result IA-BMA consistently delivers more accurate and better-calibrated predictions.
New method uses Rashomon sets to improve Bayesian inference in factorial designs.
problem Combustion of model uncertainty in factorial designs leads to multimodal posterior and convergence issues.
method Rashomon-seeded annealing, integrating high-performing models as warm start for AIS.
result Restores full posterior inference without exhaustive enumeration of model space.
Virtually any model we use in machine learning to make predictions does not perfectly represent reality. So, most of the learning happens under model misspecification. In this work, we present a novel analysis of the generalization performance of Bayesian model averaging under model misspecification and i.i.d. data usi…
New methods combine model predictions to avoid linear mixtures' limitations.
problem Combining predictions from different models to avoid linear mixtures' limitations.
method Log-linear pooling (locking) and quantum superposition (quacking) to optimise model weights.
result Demonstrated locking method with illustrative example and practical application.
Bayesian model averaging has become a widely used approach to accounting for uncertainty about the structural form of the model generating the data. When data arrive sequentially and the generating model can change over time, Dynamic Model Averaging (DMA) extends model averaging to deal with this situation. Often in ma…
Meta-learning model predicts intervention effects from uncertain causal graphs.
problem Estimating intervention effects when causal structures are uncertain.
method Model-Averaged Causal Estimation Transformer Neural Process (MACE-TNP) using meta-learning.
result MACE-TNP outperforms Bayesian baselines in predicting intervention distributions.
The motivation for this paper is to apply Bayesian structure learning using Model Averaging in large-scale networks. Currently, Bayesian model averaging algorithm is applicable to networks with only tens of variables, restrained by its super-exponential complexity. We present a novel framework, called LSBN(Large-Scale …
This paper solves aggregation of Pareto optimal models by using Bayesian priors and weighted averaging.
problem How to rationally aggregate Pareto optimal models while preserving Pareto efficiency.
method Four logical steps: 1) Bayesian models, 2) Prior as preference ranking, 3) Consistent aggregation, 4) Weighted average of priors.
result All rational/consistent aggregation rules follow a generalized hierarchical Bayesian model.
We study the information content of nuclear masses from the perspective of global models of nuclear binding energies. To this end, we employ a number of statistical methods and diagnostic tools, including Bayesian calibration, Bayesian model averaging, chi-square correlation analysis, principal component analysis, and …
Bayesian method for estimating ATE with robustness to model misspecification.
problem Estimating average treatment effects under unconfoundedness.
method Double robust Bayesian inference using adjusted prior and posterior distributions.
result Bayesian credible sets form asymptotically exact confidence intervals.
New method predicts spatio-temporal data with short and long-range dependence.
problem Uncertainty in predicting the distribution of mixed moving average fields.
method Theory-guided machine learning approach using generalized Bayesian algorithm.
result Fixed-time and any-time PAC Bayesian bounds for ensemble forecasts.
Bayesian method improves adaptive testing item selection, ensuring full item exposure.
problem Adaptive testing selects items to estimate ability, but must also ensure diverse item exposure.
method Formulated as Bayesian model averaging, deriving optimal item sampling probabilities.
result Stochastic method achieves full item bank exposure without sacrificing accuracy.
This study examines how large language models learn in-context and provides insights into their performance and architecture.
problem Understanding how large language models learn in-context and their performance metrics.
method Bayesian model averaging, parameterization, and statistical analysis of transformer architecture.
result Transformer architecture enables in-context learning through attention mechanisms and fine-grained statistical analysis.
BayesPy is an open-source Python software package for performing variational Bayesian inference. It is based on the variational message passing framework and supports conjugate exponential family models. By removing the tedious task of implementing the variational Bayesian update equations, the user can construct model…
Paper calculates loan loss after default using Bayesian model.
problem Determining loan loss after borrower default.
method Bayesian scheme considering repayment period, volumes, moments, and parameters.
result Allows setting LGD less than or equal to 1 for accurate estimates.
Paper improves Bayesian inference in federated learning with new algorithm VR-FALD*.
problem Bayesian inference in federated learning with communication bottlenecks and statistical heterogeneity.
method Federated Averaging Langevin Dynamics (FALD) and VR-FALD*.
result VR-FALD* corrects client drift due to statistical heterogeneity, improving convergence.
A new Bayesian modeling method is proposed by combining the maximization of the marginal likelihood with a momentum-space renormalization group transformation for Gaussian graphical models. Moreover, we present a scheme for computint the statistical averages of hyperparameters and mean square errors in our proposed met…
We study capital process behavior in the fair-coin game and biased-coin games in the framework of the game-theoretic probability of Shafer and Vovk (2001). We show that if Skeptic uses a Bayesian strategy with a beta prior, the capital process is lucidly expressed in terms of the past average of Reality's moves. From t…
New bound relaxes uniform gradient norm assumptions for PAC-Bayesian bounds.
problem Generalization bounds with strict assumptions like uniformly bounded loss.
method Relax uniform bounds assumptions to on-average bounded loss and gradient norm.
result Proposes a new generalization bound with a surrogate of model complexity.
This paper optimizes Bayesian estimation for log-concave models using Langevin Monte-Carlo.
problem Optimizing Bayesian estimators for log-concave models with Langevin Monte-Carlo.
method Quantitative statistical bounds and numerical approximation of Gibbs measures.
result Established optimal numerical strategy and its cost for Bayesian posterior mean approximation.
A new method for averaging model predictions using minimum divergence.
problem Improving model averaging methods, especially in small samples.
method Minimum divergence framework for model weight calculation.
result Empirically outperforms standard model averaging methods.
New geometric metrics improve Bayesian optimization performance evaluation.
problem Current metrics lack geometric insights and cannot compare algorithms effectively.
method Proposed four geometric metrics: precision, recall, average degree, and average distance.
result Proposed metrics provide more detailed evaluation of Bayesian optimization.
Paper improves BN structure learning from incomplete data.
problem Learning BN structure from incomplete data.
method Node-Average Likelihood (NAL) approach.
result NAL proves consistent and identifiable for conditional Gaussian BNs.
The paper improves error bounds for Bayesian quadrature in noisy settings.
problem Improving error bounds for Bayesian quadrature in noisy settings.
method Develops a two-step meta-algorithm to relate average-case quadrature error to L2-function approximation error. result Provides new average-case results for various kernels and noise settings.
We explore the performance of sample average approximation in comparison with several other methods for stochastic optimization when there is information available on the underlying true probability distribution. The methods we evaluate are (a) bagging; (b) kernel smoothing; (c) maximum likelihood estimation (MLE); and…
Bayesian methods improve DiD analysis for ATT estimation.
problem Estimating ATT in DiD designs with improved accuracy.
method Semiparametric Bayesian outcome regression and doubly robust adjustment.
result Bayesian methods provide strong finite-sample performance.
Bayesian causal inference method improves accuracy over traditional approaches.
problem Bayesian marginalisation over causal models is computationally infeasible.
method Decomposes structure marginalisation into causal orders and DAGs, using Gaussian processes for mechanisms and ARCO for orders.
result Method outperforms state-of-the-art in structure learning and inference.
Bayesian method identifies dynamical models with uncertainty quantification.
problem Uncertainty in selecting governing equations for dynamical systems.
method Bayesian sparse identification with model averaging.
result Accurately recovers sparse interaction structures with uncertainty quantification.
Uncertainty quantification is essential when dealing with ill-conditioned inverse problems due to the inherent nonuniqueness of the solution. Bayesian approaches allow us to determine how likely an estimation of the unknown parameters is via formulating the posterior distribution. Unfortunately, it is often not possibl…
Deep ensembles mimic Bayesian averaging with learned priors.
problem Quantifying uncertainty in neural networks.
method Showed deep ensembles perform exact Bayesian averaging with an implicitly learned data-dependent prior.
result Deep ensembles are Bayesian and provide an explanation for their strong empirical performance.
Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using Bayesian approach: Parameter and prediction uncertainty become easily available, fac…
BayesIMP combines multiple causal graphs to estimate average treatment effects with uncertainty.
problem Uncertainty quantification in causal inference from multiple datasets.
method Bayesian Interventional Mean Processes (BayesIMP) integrating probabilistic integration and kernel mean embeddings.
result Improvements in average treatment effect estimation over state-of-the-art methods.