Bayesian neural networks show good correlation between out-of-sample performance and Bayesian evidence.
problem Improving the out-of-sample performance of Bayesian neural networks.
method Numerical sampling of Bayesian posterior, ensembling over architectures, analysis of evidence vs. model size.
result Good correlation between out-of-sample performance and Bayesian evidence; ensembling improves performance.
Stochastic Bayesian Neural Network improves scalability and performance.
problem Challenges in calculating posterior distribution in Bayesian Neural Networks.
method Maximizes Evidence Lower Bound using Stochastic Evidence Lower Bound objective function.
result Demonstrates improved performance and scalability over previous algorithms.
We propose a Bayesian evidence framework to facilitate transfer learning from pre-trained deep convolutional neural networks (CNNs). Our framework is formulated on top of a least squares SVM (LS-SVM) classifier, which is simple and fast in both training and testing, and achieves competitive performance in practice. The…
Evidence Networks simplify Bayesian model comparison for complex models.
problem Bayesian model comparison challenges with intractable likelihoods or priors.
method Loss functions and neural networks for fast, amortized estimation of Bayes factors.
result Evidence Networks provide accurate and scalable Bayes factor estimation.
Method estimates Bayesian evidence from posterior samples using normalizing flows.
problem Estimating Bayesian evidence from posterior samples.
method Normalizing flows for evidence estimation.
result Method is more robust to sharp features in posterior distributions, especially in higher dimensions.
Bayesian neural networks update beliefs with soft evidence, improving accuracy and calibration.
problem Updating neural network weights with uncertain or soft evidence.
method Developed two algorithms to approximate Jeffrey's rule for updating neural network weights.
result Jeffrey-based methods outperform traditional approaches in accuracy and calibration, especially in noisy data.
We introduce a Bayesian solution for the problem in forensic speaker recognition, where there may be very little background material for estimating score calibration parameters. We work within the Bayesian paradigm of evidence reporting and develop a principled probabilistic treatment of the problem, which results in a…
The paper explores how to handle uncertain evidence in probabilistic models.
problem Handling uncertain evidence in probabilistic models and stochastic simulators.
method The paper considers distributional evidence, Jeffrey's rule, and virtual evidence as methods for interpreting uncertain evidence.
result The paper provides guidelines on how to account for uncertain evidence and highlights the importance of careful consideration.
Bayesian evidence computation revisited for model selection with improper priors.
problem Model selection with improper priors and their impact on Bayesian evidence computation.
method Employing improper priors in model selection problems, distinguishing between Bayesian evidence and fake evidences.
result Diffuse priors asymptotically to infinity do not recover the area under the likelihood.
Parallelized Bayesian quadrature improves sample efficiency and inference.
problem Efficient Bayesian inference and model evidence calculation.
method Batch Bayesian quadrature with kernel recombination for parallel sampling.
result Empirically, outperforms state-of-the-art methods in various datasets.
In this paper, we propose a new stochastic optimization algorithm for Bayesian inference based on multilevel Monte Carlo (MLMC) methods. In Bayesian statistics, biased estimators of the model evidence have been often used as stochastic objectives because the existing debiasing techniques are computationally costly to a…
Bayesian PINNs optimize loss weights for PDEs and data.
problem Optimizing loss weights in physics-informed neural networks.
method Laplace approximation for efficient model evidence computation.
result Unified Bayesian setting for PDEs and noisy measurements.
Bayesian model selection can be misled by ELBO under certain conditions.
problem Misleading model selection when using ELBO for Bayesian inference.
method Analysis of ELBO-based hyperparameter learning in a simple regression model.
result Bayesian model selection can prefer overfit models when ELBO is used, contrary to evidence.
Stochastic variational inference (SVI) plays a key role in Bayesian deep learning. Recently various divergences have been proposed to design the surrogate loss for variational inference. We present a simple upper bound of the evidence as the surrogate loss. This evidence upper bound (EUBO) equals to the log marginal li…
Bayesian inference with deep, weakly nonlinear networks is solved rigorously.
problem Bayesian inference with neural networks of specific structure.
method Perturbative analysis of fully connected neural networks with a shaped nonlinearity.
result Neural network Bayesian inference can be equivalent to kernel methods under certain conditions.
We present a novel technique for tailoring Bayesian quadrature (BQ) to model selection. The state-of-the-art for comparing the evidence of multiple models relies on Monte Carlo methods, which converge slowly and are unreliable for computationally expensive models. Previous research has shown that BQ offers sample effic…
Bayesian approach confirms no return predictability for 1926-2004 data, weak evidence for 1953-2021.
problem Investigating return predictability using Bayesian methods.
method Developed a new shrinkage type prior for a model parameter in a VAR system, compared to other estimation methods.
result Bayesian approach outperforms reduced-bias estimator in terms of size and power.
Bayesian model selection via mean-field variational approximation improves efficiency and accuracy.
problem Bayesian model selection under model mis-specification and latent variables.
method Mean-field variational approximation with non-asymptotic properties and geometric convergence.
result ELBO tends to select models closer to the true model than BIC as sample size increases.
New method corrects Laplace/BIC errors in singular models, revealing effective dimension.
problem Laplace/BIC errors in singular models due to incorrect effective dimension assumption.
method RLCT (real log canonical threshold) to correct effective dimension in linear models.
result Correct evidence slope and effective dimension estimation in linear settings.
In this paper we review the concepts of Bayesian evidence and Bayes factors, also known as log odds ratios, and their application to model selection. The theory is presented along with a discussion of analytic, approximate and numerical techniques. Specific attention is paid to the Laplace approximation, variational Ba…
We consider the forecast aggregation problem in repeated settings, where the forecasts are done on a binary event. At each period multiple experts provide forecasts about an event. The goal of the aggregator is to aggregate those forecasts into a subjective accurate forecast. We assume that experts are Bayesian; namely…
GBEST model improves survival analysis for small datasets.
problem Challenges in survival analysis, especially with small data.
method Bayesian bootstrap and Beta Stacy bootstrap methods integrated into bagging tree models.
result GBEST model outperforms classical survival models in predictive performance and stability.
Adapts linearised Laplace method for deep learning models.
problem Incompatibility of linearised Laplace method with modern deep learning tools.
method Examines and adapts linearised Laplace method for model selection in deep learning.
result Recommendations for better adapting linearised Laplace method to modern deep learning.
SA-BCP combines long-term and local evidence for efficient, adaptive online prediction.
problem Balancing fast adaptation and stable coverage in online prediction.
method State-Adaptive Bayesian Conformal Prediction (SA-BCP) using gated convex combination of temporal inertia and spatial evidence.
result SA-BCP achieves at-or-above-nominal coverage with substantially sharper intervals compared to discounted Bayesian CP.
Bayesian evidence helps compare models but can overfit.
problem Comparing hypotheses consistent with observations.
method Marginal likelihood, Occam's razor, PAC-Bayes bounds.
result Marginal likelihood can negatively correlate with generalization.
Bayesian method reconstructs hidden higher-order interactions from network data.
problem Lack of explicit higher-order interactions in pairwise network data.
method Bayesian approach based on parsimony, infers higher-order structures when statistically supported.
result Demonstrated applicability to various datasets, synthetic and empirical.
Bayesian linear networks reveal optimal depth and width trade-offs.
problem Understanding how depth, width, and dataset size affect model quality in linear networks.
method Zero noise Bayesian inference with Gaussian weight priors and mean squared error.
result Optimal predictions at infinite depth and maximized Bayesian model evidence at infinite depth.
The paper examines how updates to probabilistic models influence behavior based on evidence.
problem Understanding how updates to probabilistic models influence behavior based on evidence.
method Study of KL-regularized soft updates as Bayesian posterior updates within a single probabilistic model.
result Posterior updates determine relative incentives but not absolute rewards, which are ambiguous up to context-specific baselines.
Two data-dependent information metrics are developed to quantify the information of the prior and likelihood functions within a parametric Bayesian model, one of which is closely related to the reference priors from Berger, Bernardo, and Sun, and information measure introduced by Lindley. A combination of theoretical, …
A new framework for efficient sequence maps using Bayesian filtering and covariance.
problem Designing efficient recurrent sequence maps from explicit memory assumptions.
method Design-model framework, exact Bayesian filtering, query-dependent readout, linear-Gaussian instantiation.
result Improved robustness and retrieval performance across various benchmarks.
fiBAG integrates multiplatform genomic data to identify disease markers.
problem Understanding complex mechanisms underlying human diseases from multiplatform genomic data.
method fiBAG uses Gaussian process models and Bayes factors to identify functional evidence and guide variable selection.
result fiBAG improves detection of disease-related markers compared to non-integrative methods.
Simulation-based inference methods can produce unreliable posterior approximations.
problem Reliability of simulation-based inference methods for scientific use cases.
method Benchmarked algorithms including Neural Posterior Estimation, Neural Ratio Estimation, Sequential Neural Likelihood, and Approximate Bayesian Computation.
result Ensembling posterior surrogates provides more reliable approximations.
We offer a novel view of AdaBoost in a statistical setting. We propose a Bayesian model for binary classification in which label noise is modeled hierarchically. Using variational inference to optimize a dynamic evidence lower bound, we derive a new boosting-like algorithm called VIBoost. We show its close connections …
New method improves ABC for Bayesian model comparison.
problem Comparing complex models with observed data.
method Approximate Bayesian Computation with posterior density estimation.
result Efficiently assigns high posterior probabilities to ground-truth models.
OPAA estimates probability densities using functional analysis.
problem Estimating probability density functions efficiently and accurately.
method OPAA uses a parallelizable algorithm based on functional analysis to estimate probability distributions.
result OPAA provides an efficient method to estimate probability density functions and normalizing weights.
Paper proposes SaiyanH to learn BNs with full evidence propagation from dependent variables.
problem Learning disjoint subgraphs that prevent full evidence propagation.
method SaiyanH, a hybrid structure learning algorithm.
result SaiyanH outperforms state-of-the-art algorithms in reconstructing true DAGs.
PAC-Bayesian bounds for MLPs with cross entropy loss validated.
problem Generalization bounds for MLPs with cross entropy loss.
method Introduced probabilistic explanations and proved PAC-Bayesian bounds using ELBO.
result MLPs with cross entropy loss inherently guarantee PAC-Bayesian generalization bounds.
Many probabilistic models of interest in scientific computing and machine learning have expensive, black-box likelihoods that prevent the application of standard techniques for Bayesian inference, such as MCMC, which would require access to the gradient or a large number of likelihood evaluations. We introduce here a n…
A recent paper by Liu et al. combines the topics of adversarial training and Bayesian Neural Networks (BNN) and suggests that adversarially trained BNNs are more robust against adversarial attacks than their non-Bayesian counterparts. Here, I analyze the proposed defense and suggest that one needs to adjust the adversa…
This article introduces a Bayesian nonparametric method for quantifying the relative evidence in a dataset in favour of the dependence or independence of two variables conditional on a third. The approach uses Polya tree priors on spaces of conditional probability densities, accounting for uncertainty in the form of th…
We define the information threshold in Bayesian decision-making.
problem Understanding the optimal amount of information for reliable classification.
method Defining the information threshold as the point of maximum curvature in the prior vs. posterior curve.
result At the information threshold, additional evidence does not significantly improve posterior probability.
Bayesian method for multivariate autoregressive models with exogenous inputs.
problem Estimating uncertainties in autoregressive models with exogenous inputs.
method Recursive Bayesian estimation via message passing in a factor graph.
result Produces full posterior distributions for autoregressive coefficients and noise precision.
The paper tackles approximate unlearning from a subset of training data using variational inference.
problem Unlearning from a small subset of erased training data while maintaining the posterior belief from the full data.
method Formulates unlearning as minimizing KL divergence, equivalent to minimizing an evidence upper bound. Uses variational inference to approximate posterior beliefs and proposes two tricks to handle challenges.
result Demonstrates the effectiveness of the proposed unlearning methods on various Bayesian models.
A new method for CT-DCEGs simplifies inference for asymmetric processes.
problem Inference in asymmetric state space problems with continuous time evolution.
method An extension of CEG propagation for CT-DCEGs, employing junction tree inference.
result CT-DCEGs are preferred over DBNs and continuous time BNs for asymmetric processes.
Unified Bayesian framework for efficient off-policy evaluation and learning in large action spaces.
problem Efficient off-policy evaluation and learning in systems with correlated actions.
method Unified Bayesian framework with structured priors and sDM approach.
result sDM leverages action correlations without compromising computational efficiency.
A new method using normalizing flows speeds up Bayesian model comparison.
problem Computational challenges in calculating Bayesian evidence for complex models.
method Savage-Dickey density ratio with normalizing flows.
result The method scales to high-dimensional settings and provides consistent Bayes factors.
The log-determinant of a kernel matrix appears in a variety of machine learning problems, ranging from determinantal point processes and generalized Markov random fields, through to the training of Gaussian processes. Exact calculation of this term is often intractable when the size of the kernel matrix exceeds a few t…
When using the K-nearest neighbors method, one often ignores uncertainty in the choice of K. To account for such uncertainty, Holmes and Adams (2002) proposed a Bayesian framework for K-nearest neighbors (KNN). Their Bayesian KNN (BKNN) approach uses a pseudo-likelihood function, and standard Markov chain Monte Carlo (…