GBEST model improves survival analysis for small datasets.
problem Challenges in survival analysis, especially with small data.
method Bayesian bootstrap and Beta Stacy bootstrap methods integrated into bagging tree models.
result GBEST model outperforms classical survival models in predictive performance and stability.
Bayesian interpretation of deep ensembles improves uncertainty quantification.
problem Improving uncertainty estimation in deep learning models.
method Viewing deep ensembles as an approximate Bayesian method and specifying corresponding assumptions.
result Improved approximation leads to larger epistemic uncertainty, potentially more reliable predictions.
Bayesian Quadrature improves ensembling for neural networks with dispersed likelihood peaks.
problem Ensembling neural networks struggles with dispersed, narrow peaks in likelihood surfaces.
method Uses Bayesian Quadrature to construct weighted ensembles of architectures.
result Empirically outperforms state-of-the-art baselines in test likelihood, accuracy, and expected calibration error.
Improved Bayesian inference for neuronal ensemble inference reduces computational cost.
problem Efficient inference of neuronal ensembles from activity data.
method Modified MCMC algorithm with simulated annealing for hyperparameter control.
result Our method reduces computational cost while maintaining or improving inference accuracy.
A new stopping rule based on E-values helps efficiently use sampling in Bayesian Deep Ensembles.
problem How long should sampling continue in Bayesian Deep Ensembles to yield significant improvements?
method Formulated as a sequential anytime-valid hypothesis test, using E-values to decide when to stop sampling.
result Only a fraction of the full-chain budget is often required for significant improvements.
Unified theory linking Bayesian and ensemble methods in deep learning.
problem Uncertainty quantification in deep learning.
method Reformulating optimisation as convex optimisation in probability measures, studying Wasserstein gradient flows.
result Unified theory explaining success of deep ensembles over variational inference.
Proposes a new method for ensembling neural subnetworks.
problem Computational expense and limited flexibility of traditional deep ensembles.
method Sequential Bayesian neural subnetwork ensembling.
result Outperforms traditional ensembles in various metrics.
Proposes OBS, a method to adaptively combine Bayesian models online.
problem Learning optimal combinations of Bayesian models in online learning.
method Empirical Bayes lens, Online Bayesian Stacking (OBS).
result Establishes a novel connection between OBS and portfolio selection.
Ensembles of neural networks (NNs) have long been used to estimate predictive uncertainty; a small number of NNs are trained from different initialisations and sometimes on differing versions of the dataset. The variance of the ensemble's predictions is interpreted as its epistemic uncertainty. The appeal of ensembling…
Deep ensembles mimic Bayesian averaging with learned priors.
problem Quantifying uncertainty in neural networks.
method Showed deep ensembles perform exact Bayesian averaging with an implicitly learned data-dependent prior.
result Deep ensembles are Bayesian and provide an explanation for their strong empirical performance.
Proposes a new method for nonlinear Bayesian updates using ensemble kernel regression.
problem Nonlinear and non-Gaussian Bayesian updates for complex systems.
method Combines Kalman filtering for observed components and kernel density estimation for unobserved components, with subsampling and clustering.
result Reduces estimation errors in highly nonlinear scenarios compared to standard linear updates.
Stein variational neural network ensembles improve diversity and uncertainty estimation.
problem Lack of proper Bayesian justification and diversity guarantees in deep neural network ensembles.
method Particle-based inference methods, specifically Stein variational gradient descent (SVGD), operating in weight space, function space, and hybrid settings.
result SVGD methods improve diversity and uncertainty estimation, approaching the true Bayesian posterior more closely.
We propose a novel "tree-averaging" model that utilizes the ensemble of classification and regression trees (CART). Each constituent tree is estimated with a subset of similar data. We treat this grouping of subsets as Bayesian ensemble trees (BET) and model them as an infinite mixture Dirichlet process. We show that B…
BaMANI uses ensemble learning to improve Bayesian network inference.
problem Bayesian network inference's reliance on specific algorithms can obscure causal relationships.
method Developed an ensemble learning approach to marginalize algorithm impact.
result Improved accuracy and reliability of causal network predictions.
Improved neural network ensembles using Stein Variational Newton updates.
problem Lack of efficient second-order information in current ensemble methods.
method Proposes a novel approximate Bayesian inference method integrating Stein Variational Newton updates with scalable Hessian approximations.
result Significantly faster convergence and more accurate posterior distribution approximations.
Decentralized Gaussian processes for multi-agent systems.
problem Scalable and flexible learning solutions for multi-agent systems.
method Asymptotically exact decentralized solution to Gaussian processes, with online Bayesian model averaging for hyperparameter selection.
result Asymptotically exact decentralized Gaussian process approximation and online Bayesian model averaging.
A new clustering method using Bayesian techniques improves robustness and interpretability.
problem Improving clustering techniques for better robustness and interpretability.
method The paper proposes a novel Bayesian clustering method using the proper Bayesian bootstrap, which combines k-means clustering and ensemble clustering.
result The method provides clear indication on the optimal number of clusters and a better representation of the clustered data.
FED distills ensemble predictions efficiently, improving accuracy and uncertainty.
problem Efficiently distilling ensemble predictions from limited data.
method Proposes Functional Ensemble Distillation (FED) using mixup augmentation.
result Significantly boosts performance in accuracy and uncertainty estimation.
Tree ensembles, such as random forests and boosted trees, are renowned for their high prediction performance. However, their interpretability is critically limited due to the enormous complexity. In this study, we present a method to make a complex tree ensemble interpretable by simplifying the model. Specifically, we …
Bayesian inference using stochastic neural networks ensembles.
problem Approximating Bayesian posterior distributions.
method Formulate stochastic ensembles of neural networks, train with variational inference, and evaluate using Monte Carlo dropout.
result Stochastic ensembles provide more accurate posterior estimates than other methods.
This paper examines how to calibrate ensemble members for better prediction accuracy.
problem Improper calibration of deep neural networks leads to unreliable probability estimates.
method Theoretical analysis and empirical evaluation on CIFAR-100 dataset.
result Well-calibrated ensemble members do not guarantee a well-calibrated ensemble prediction, but a well-calibrated ensemble prediction cannot exceed the average performance of its members.
VGE provides a practical approach to uncertainty estimation in ensemble models.
problem Uncertainty estimation in ensemble models using additive decomposition breaks down.
method Variance-Gated Ensembles (VGE) introduces a differentiable framework with a signal-to-noise gate.
result VGE provides a Variance-Gated Margin Uncertainty (VGMU) score and Variance-Gated Normalization (VGN) layer.
This paper connects RND, deep ensembles, and Bayesian inference, providing a unified theoretical perspective.
problem Uncertainty quantification in deep learning models.
method Analysis of Random Network Distillation (RND) within the neural tangent kernel framework.
result The uncertainty signal from RND is equivalent to the predictive variance of a deep ensemble and can be made to mirror the centered posterior predictive distribution of Bayesian inference.
Paper presents a Bayesian-decision-theory framework for long-tailed classification.
problem Heavy imbalance and asymmetric misprediction costs in long-tailed datasets.
method Bayesian-decision-theory perspective, unifying re-balancing and ensemble methods.
result Improves accuracy for all classes, especially tails, with provably optimal decisions.
Paper introduces XBART for nonlinear regression, outperforming XGBoost.
problem Nonlinear regression problems, especially in speed and accuracy.
method Combines Bayesian modeling and recursive partitioning for efficient, accurate predictions.
result XBART provides faster and more accurate predictions than XGBoost.
Ensemble learning is a standard approach to building machine learning systems that capture complex phenomena in real-world data. An important aspect of these systems is the complete and valid quantification of model uncertainty. We introduce a Bayesian nonparametric ensemble (BNE) approach that augments an existing ens…
Bayesian deep ensembles improve prediction accuracy in various settings.
problem Improving prediction accuracy of deep ensembles in out-of-distribution settings.
method Introducing a randomised, untrainable function to each ensemble member, enabling a posterior predictive distribution interpretation.
result Bayesian deep ensembles make more conservative predictions and outperform standard ensembles in various tasks.
MixupMP improves uncertainty quantification in neural networks using data augmentation.
problem Uncertainty quantification in deep learning models.
method MixupMP constructs a more realistic predictive distribution using data augmentation techniques.
result MixupMP achieves superior predictive performance and uncertainty quantification on various image classification datasets.
EDRBO optimizes Bayesian optimization with continuous contexts using ensemble models and robust methods.
problem Bayesian optimization with unknown and continuous contextual distributions leads to suboptimal results.
method EDRBO uses ensemble surrogate models and Wasserstein ball ambiguity sets to handle uncertainty and maintain computational tractability.
result EDRBO achieves sublinear cumulative regret guarantees of order O ( γ T T ) \mathcal{O}(γ_T \sqrt{T}) O ( γ T T ) . Bayesian symbolic regression automates model discovery from data.
problem Learning closed-form mathematical models from data using heuristic methods.
method Probabilistic approach to symbolic regression, connecting to information theory and statistical physics.
result Probabilistic approach provides model plausibility and performance guarantees.
Revises Bayesian model averaging for foundation models.
problem Ensemble pre-trained and lightly-finetuned foundation models for improved classification performance.
method Introduces trainable linear classifiers and computationally cheaper model averaging scheme (OMA).
result Ensembled models can better predict on various datasets.
Bayesian Neural Networks improve geophysical model ensembles with reduced uncertainty.
problem Improving geophysical model projections and uncertainty quantification.
method Developed a Bayesian Neural Network ensemble strategy for geophysical models.
result Bayesian Neural Network ensemble outperforms existing methods in ozone prediction.
Study improves Bayesian optimisation with ensemble transfer learning.
problem Improving sample efficiency in Bayesian optimisation of expensive functions.
method Empirical analysis of ensemble-based transfer learning methods and pipeline components.
result Two components (warm start initialisation and positive weight constraint) improve transfer learning Bayesian optimisation performance.
This paper introduces a privacy-aware Bayesian approach that combines ensembles of classifiers and clusterers to perform semi-supervised and transductive learning. We consider scenarios where instances and their classification/clustering results are distributed across different data sites and have sharing restrictions.…
New algorithm reduces overfitting in neural networks.
problem Overfitting in neural networks.
method Integrates SMC with SGHMC for mini-batch sampling.
result SMCSGHMC outperforms SGD and deep ensembles.
The paper connects neural network ensembles to Bayesian inference using variational methods.
problem Explaining the behavior of ensemble methods in neural networks.
method Deriving conditions for ensemble optimization to reduce divergence to the posterior distribution.
result Ensemble methods can be a valid alternative to approximate Bayesian inference.
Unified predictive uncertainty disentangled using deep split ensembles.
problem Understanding and quantifying uncertainty in NNs for real-world applications.
method Deep split ensemble approach using multivariate Gaussian mixture model.
result Inherently well-calibrated models with high flexibility to group features.
Bayesian EnKF improves sentence comprehension uncertainty modeling.
problem Uncertainty in human language comprehension, especially with ambiguous inputs.
method Bayesian framework using ensemble Kalman filter (EnKF) for uncertainty quantification.
result Enhanced model's ability to approximate human cognitive processing with linguistic ambiguities.
Repulsive deep ensembles improve diversity and Bayesian inference.
problem Challenges in maintaining diversity among ensemble members trained independently.
method Introducing a repulsive term in the update rule of deep ensembles.
result Training dynamics of repulsive ensembles follow a Wasserstein gradient flow of KL divergence with the true posterior.
Fed-ensemble improves FL by averaging predictions from multiple models.
problem Improving generalization in federated learning.
method Random permutations to update K models, averaging predictions.
result Predictions from all K models have the same predictive posterior distribution.
Rodent hippocampal population codes represent important spatial information about the environment during navigation. Several computational methods have been developed to uncover the neural representation of spatial topology embedded in rodent hippocampal ensemble spike activity. Here we extend our previous work and pro…
Bayesian tree ensemble model for estimating treatment effects in high-dimensional survival data.
problem Estimating heterogeneous treatment effects in censored survival data with many covariates.
method Developed a Bayesian tree ensemble model with a horseshoe prior for adaptive shrinkage.
result Accurately estimates treatment effects in high-dimensional covariate spaces and non-linear functions.
Improving Bayesian filtering with strictly proper scoring rules
problem Bayesian filtering of partially and noisily observed dynamical systems
method Proper scoring ensemble filter (PSEF)
result Accurate approximation of challenging filtering distributions
New method uses EKI for efficient Bayesian inference in high-dimensional problems.
problem Efficient inference for high-dimensional posterior distributions in physics-informed neural networks.
method Ensemble Kalman Inversion (EKI) for high-dimensional posterior inference.
result EKI-based inference provides comparable uncertainty estimates to HMC-based methods but with reduced computational cost.
Enhances optimization and sampling methods using ensemble-based gradient inference.
problem Improving ensemble-based methods for optimization and sampling.
method Ensemble-based gradient inference (EGI) to extract higher-order derivatives from particle ensembles.
result Augmented algorithms outperform gradient-free variants, especially in multimodal and non-Gaussian settings.
Simple mode exploration methods do not improve performance in neural networks.
problem Improving predictive probabilities in neural networks.
method Exploring local regions around diverse solutions using simple methods.
result Simple mode exploration methods do not improve performance.
BI-EqNO improves Bayesian inference with flexible neural operators.
problem Inaccurate estimation of marginal likelihoods in approximate Bayesian methods.
method Equivariant neural operator framework for generalized approximate Bayesian inference.
result BI-EqNO enhances both deterministic and stochastic approaches to Bayesian inference.
Understanding the uncertainty of a neural network's (NN) predictions is essential for many purposes. The Bayesian framework provides a principled approach to this, however applying it to NNs is challenging due to large numbers of parameters and data. Ensembling NNs provides an easily implementable, scalable method for …