Paper proposes a compression principle for neural networks using Bayesian optimization.
problem Finding methods for making generalizable predictions in machine learning.
method Compression principle and Bayesian optimization approach.
result Optimal predictive models minimize total compressed message length of data and model definition.
Reinterprets Granger causality with causal Bayesian networks and Reichenbach's principles.
problem Lack of a rigorous causal foundation in Granger causality.
method Reinterpreting Granger causality through Reichenbach's principles and causal Bayesian networks, implementing as c-GC.
result c-GC provides a more principled framework for causal discovery in observational datasets.
Bayesian method detects outliers and uncertain points in data.
problem Detecting outliers and uncertain points in data using Bayesian methods.
method Generative model of data curation for aleatoric uncertainty, combining with epistemic uncertainty and outlier exposure.
result Principled Bayesian approach outperforms methods using aleatoric or epistemic uncertainty alone.
Bayesian methods promise to fix many shortcomings of deep learning, but they are impractical and rarely match the performance of standard methods, let alone improve them. In this paper, we demonstrate practical training of deep networks with natural-gradient variational inference. By applying techniques such as batch n…
Enhances optimization in multi-source settings with causal principles.
problem Optimizing functions with multiple sources of data and causal dependencies.
method Integrates Multi-Source Bayesian Optimization with Causal Bayesian Optimization principles.
result Improves optimization efficiency and reduces computational complexity.
A novel Laplace-approximated Bayesian Tensor Network Kernel Machine (LA-TNKM) provides principled uncertainty estimates.
problem How to provide principled uncertainty estimates for tensor network kernel machines.
method Employing a linearized Laplace approximation for Bayesian inference.
result Consistently matches or surpasses Gaussian Processes and BNNs across diverse UCI regression benchmarks.
Datasets are growing not just in size but in complexity, creating a demand for rich models and quantification of uncertainty. Bayesian methods are an excellent fit for this demand, but scaling Bayesian inference is a challenge. In response to this challenge, there has been considerable recent work based on varying assu…
Bayesian principles improve neural additive models for better feature selection and uncertainty.
problem Lack of calibrated uncertainties and feature selection in neural additive models.
method Augmenting NAMs with Bayesian principles to provide credible intervals, feature selection, and interaction ranking.
result Improved performance on tabular datasets and real-world medical tasks.
Bayesian methods detect clusters in noisy data more reliably.
problem Noisy data distorts traditional clustering methods, leading to unreliable results.
method Bayesian community detection using Minimum Description Length principle.
result Bayesian methods identify more robust clusters in noisy data.
Bayesian principles improve agentic AI decision-making.
problem Decision-making under uncertainty in agentic AI systems.
method Bayesian decision theory applied to the orchestration layer of agentic AI.
result Bayesian principles enhance agentic AI's ability to make decisions under uncertainty.
Bayesian reflex models AI learning like the autonomic nervous system.
problem Online learning in dynamic AI environments.
method Bayesian online algorithms with belief maintenance, sequential updating, and uncertainty-driven action balancing.
result Unified framework for adaptive AI learning.
Active inference minimizes expected free energy for optimal behavior.
problem Understanding and optimizing behavior in complex systems.
method Combines Bayesian decision theory, optimal Bayesian design, and the free energy principle.
result Active inference emerges as a unified framework for information-seeking, utility maximization, and goal-directed behavior.
Bayesian Scattering offers a simple baseline for image data uncertainty.
problem Lack of interpretable, mathematically grounded uncertainty quantification methods for image data.
method Coupling wavelet scattering transform with a simple probabilistic head.
result Bayesian Scattering provides sensible uncertainty estimates under distribution shifts.
Bayesian models use hyperparameters to indirectly assign priors, and this work shows how these priors can be derived from maximum entropy principles.
problem Understanding the assumptions and dependencies in Bayesian hierarchical models.
method Demonstrates how canonical distributions and maximum entropy principles can be used to derive marginal priors in hierarchical models.
result Marginal priors in hierarchical models derived from maximum entropy principles have different constraints compared to the original priors.
Proposes a new Bayesian learning method for optimal treatment regimes.
problem Sub-optimal policies in offline data due to lack of exploration.
method Integrates pessimism principle with Thompson sampling and Bayesian machine learning.
result Derives a credible set that uniformly lower bounds the optimal Q-function.
Unified Bayesian-AI framework improves epidemiological risk prediction and uncertainty quantification.
problem Lack of calibrated uncertainty in machine learning models for epidemiology.
method Combines Bayesian prediction with Bayesian hyperparameter optimization using logistic regression and Gaussian-process Bayesian optimization.
result Unified Bayesian-AI framework provides reliable coverage and improved calibration, enhancing epidemiological decision making.
This paper extends the work in [Suzuki, 1996] and presents an efficient depth-first branch-and-bound algorithm for learning Bayesian network structures, based on the minimum description length (MDL) principle, for a given (consistent) variable ordering. The algorithm exhaustively searches through all network structures…
Bayesian method improves SOM training for dynamic data.
problem Training Self Organizing Maps (SOM) for non-stationary data.
method Variational Bayesian approach with adaptive neighborhood radius.
result Method outperforms other adaptive methods in high dimensions.
Bayesian neural networks with data augmentation show a persistent cold posterior effect.
problem Understanding the cold posterior effect in Bayesian neural networks with data augmentation.
method Developed principled Bayesian neural networks using data augmentation, providing exact likelihoods and tight bounds.
result The cold posterior effect persists even in models incorporating data augmentation, suggesting it's not an artifact.
A neural network derived from first principles using MaxEnt.
problem Developing a neural network from first principles.
method Derived a neural network using the principle of Maximum Entropy, with linear dimension-reducing transformations and conditional mean estimators.
result Unified theoretical justification for activation functions like sigmoid, softplus, and relu.
Proposes MCBO for causal Bayesian optimization with model learning and regret bounds.
problem Maximizing downstream variables in unknown structural models.
method Model-based causal Bayesian optimization (MCBO) that learns full system models and trades off exploration and exploitation.
result First non-asymptotic bounds for CBO and practical implementation showing superior performance.
We present a principled Bayesian framework for signal reconstruction, in which the signal is modelled by basis functions whose number (and form, if required) is determined by the data themselves. This approach is based on a Bayesian interpretation of conventional sparse reconstruction and regularisation techniques, in …
Paper explores SVGD for Bayesian inference, linking deterministic and stochastic dynamics.
problem Bayesian inference and Markov chain Monte Carlo methods.
method Stein variational gradient descent (SVGD) with deterministic and stochastic dynamics.
result Identifies Stein-Fisher information as the leading order contribution in the long-time and many-particle regime.
Bayesian scores improve structure learning in probabilistic circuits.
problem Improper structure learning in probabilistic circuits based on heuristics.
method Developed Bayesian structure scores for deterministic PCs, using them in a greedy cutset algorithm.
result Effective protection against overfitting and fast, almost hyper-parameter-free structure learner.
Consider a Bayesian inference problem where a variable of interest does not take values in a Euclidean space. These "non-standard" data structures are in reality fairly common. They are frequently used in problems involving latent discrete factor models, networks, and domain specific problems such as sequence alignment…
Bayesian learning rule trains binary neural networks effectively.
problem Training binary neural networks is challenging due to discrete optimization.
method Proposes the Bayesian learning rule to estimate Bernoulli weights.
result Obtains state-of-the-art performance and enables uncertainty estimation.
A classic approach for learning Bayesian networks from data is to identify a maximum a posteriori (MAP) network structure. In the case of discrete Bayesian networks, MAP networks are selected by maximising one of several possible Bayesian Dirichlet (BD) scores; the most famous is the Bayesian Dirichlet equivalent unifo…
PACOH improves meta-learning with theoretical guarantees and practical efficiency.
problem Meta-learning's generalization to unseen tasks is poorly understood, especially with limited meta-training tasks.
method PAC-Bayesian framework for deriving generalization bounds and developing PAC-optimal meta-learning algorithms.
result PACOH yields state-of-the-art performance in predictive accuracy and uncertainty estimation.
New method prevents forgetting in learning new tasks.
problem Poor ability of models to solve new problems without forgetting.
method Task-agnostic hierarchical information-theoretic optimality principle with Mixture-of-Variational-Experts layer.
result Demonstrated competitive performance in continual supervised and reinforcement learning.
A central problem in analyzing networks is partitioning them into modules or communities. One of the best tools for this is the stochastic block model, which clusters vertices into blocks with statistically homogeneous pattern of links. Despite its flexibility and popularity, there has been a lack of principled statist…
New approach for off-policy learning in contextual bandits with performance guarantees.
problem Improving performance of logging policies in contextual bandits.
method PAC-Bayesian analysis of policy mixtures, providing tighter generalization bounds and tractable optimization algorithms.
result Proved tighter generalization bounds and demonstrated effectiveness in practical scenarios.
RCaGP improves robustness and computational efficiency in Gaussian processes.
problem Outliers in large datasets corrupt standard Gaussian process models.
method Combines robustness and approximation-awareness in a principled framework.
result Ensures more conservative and reliable uncertainty estimates.
Bayesian neural networks (BNNs) augment deep networks with uncertainty quantification by Bayesian treatment of the network weights. However, such models face the challenge of Bayesian inference in a high-dimensional and usually over-parameterized space. This paper investigates a new line of Bayesian deep learning by pe…
Bayesian networks are a versatile and powerful tool to model complex phenomena and the interplay of their components in a probabilistically principled way. Moving beyond the comparatively simple case of completely observed, static data, which has received the most attention in the literature, in this paper we will revi…
Bayesian method improves star location and flux estimation from coadded images.
problem Statistical analysis of coadded astronomical images is complicated by pixel dependence.
method Bayesian approach that implicitly marginalizes single-exposure pixel intensities.
result Method outperforms single-exposure image training for star parameter estimation.
MissBGM uses AI and Bayesian modeling for better missing data imputation.
problem Missing data imputation in data science, especially with uncertainty quantification.
method AI-powered Bayesian generative modeling with explicit modeling of missingness mechanisms.
result MissBGM provides principled posterior uncertainty over imputations and superior performance.
VES-Gamma adapts EI using information-theoretic principles.
problem Optimizing black-box functions using Bayesian optimization.
method Variational Entropy Search (VES) and VES-Gamma algorithm.
result VES-Gamma improves EI by incorporating information-theoretic concepts.
Sum-product networks (SPNs) are flexible density estimators and have received significant attention due to their attractive inference properties. While parameter learning in SPNs is well developed, structure learning leaves something to be desired: Even though there is a plethora of SPN structure learners, most of them…
Unified framework for arbitrary conditional inference using AI and Bayesian methods.
problem Limited flexibility in existing conditional inference methods.
method Bayesian generative modeling with stochastic iterative algorithm.
result Single learned model for universal conditional prediction with uncertainty quantification.
Bayesian approach optimizes in-context learning for state space models.
problem Optimizing in-context learning for state space models.
method Bayesian optimal sequential prediction over latent sequence tasks.
result Bayesian optimal predictor converges to posterior predictive mean.
Bayesian method corrects bias in imbalanced datasets.
problem Prevalence bias in machine learning datasets.
method Bayesian risk minimization framework, bias-corrected loss function.
result Corrected loss function improves model performance.
We show that a neural network with arbitrary depth and non-linearities, with dropout applied before every weight layer, is mathematically equivalent to an approximation to a well known Bayesian model. This interpretation might offer an explanation to some of dropout's key properties, such as its robustness to over-fitt…
We introduce a Bayesian solution for the problem in forensic speaker recognition, where there may be very little background material for estimating score calibration parameters. We work within the Bayesian paradigm of evidence reporting and develop a principled probabilistic treatment of the problem, which results in a…
We develop the first Bayesian Optimization algorithm, BLOSSOM, which selects between multiple alternative acquisition functions and traditional local optimization at each step. This is combined with a novel stopping condition based on expected regret. This pairing allows us to obtain the best characteristics of both lo…
Unified perspective unites Bayesian optimization and active learning for efficient goal-oriented optimization.
problem Efficiently optimize expensive engineering and scientific problems with limited data.
method Unified framework linking Bayesian infill criteria and active learning criteria.
result Unified approach formalizes Bayesian infill criteria and active learning criteria.
Bayesian optimization improves molecule design by addressing three pitfalls.
problem Bayesian optimization pitfalls cause poor performance in molecule design.
method Identified and addressed three pitfalls: incorrect prior width, over-smoothing, and inadequate acquisition function maximization.
result Basic BO setup achieves highest performance on PMO benchmark.
New rigorous uncertainty bounds for Gaussian Process regression.
problem Need for frequentist uncertainty bounds in applications like learning-based control.
method Introduce new uncertainty bounds that are rigorous and practically useful.
result New bounds are less conservative and more useful for practical applications.
FP-BMA improves generalization by encouraging flat posteriors in Bayesian Model Averaging.
problem Lack of flat posterior in approximate Bayesian inference methods hinders effective Bayesian Model Averaging.
method Proposes Flat Posterior-aware Bayesian Model Averaging (FP-BMA) and Flat Posterior-aware Bayesian Transfer Learning schemes.
result FP-BMA successfully captures flat posteriors, improving generalization performance.