Active inference minimizes expected free energy for optimal behavior.
problem Understanding and optimizing behavior in complex systems.
method Combines Bayesian decision theory, optimal Bayesian design, and the free energy principle.
result Active inference emerges as a unified framework for information-seeking, utility maximization, and goal-directed behavior.
Expands Bayesian experiment design framework to account for model discrepancies.
problem Model misspecification in Bayesian optimal experiment design.
method Introduces Expected General Information Gain and Expected Discriminatory Information criteria.
result Demonstrates improved robustness and detection capabilities in experiment design.
New method improves robustness of Bayesian experimental design.
problem Bayesian experimental design's sensitivity to prior distribution changes.
method Introduces robust expected information gain (REIG) and uses KL-divergence ambiguity sets.
result REIG stabilizes sampling-based EIG estimation and compensates for prior variability.
New method corrects active learning for distribution shifts and outliers.
problem Conventional active learning methods fail to account for test-time distribution.
method JEPIG, a hybrid of BALD and EPIG, maximizes expected predictive information gain.
result JEPIG outperforms conventional methods in active learning with distribution shifts.
Estimates expected information gain using density approximations and dimension reduction.
problem Estimating expected information gain in nonlinear and non-Gaussian settings.
method Flexible transport-based schemes for EIG estimation, optimal sample allocation, and gradient-based upper bounds on mutual information.
result Optimal sample allocation and dimension reduction schemes improve EIG estimation accuracy and convergence rate.
Paper proposes an unbiased optimization method for Bayesian experimental design.
problem Maximizing expected information gain in Bayesian experimental design.
method Randomized multilevel Monte Carlo (MLMC) method combined with stochastic gradient descent.
result An unbiased estimator for the gradient of expected information gain.
We approximate differential entropy for efficient Bayesian experimental design.
problem Efficiently estimating expected information gain in large-scale inference problems.
method Approximate differential entropy using Monte Carlo or quasi-Monte Carlo surrogates.
result Our approach achieves comparable or better convergence rates than state-of-the-art methods.
Active inference selects actions to maximize information gain, aiding structure learning.
problem Learning the structure of underlying world models.
method Active inference selects actions based on expected free energy, which includes information gain and value.
result Actions that maximize information gain help disambiguate among alternative models.
Experimental design is crucial for inference where limitations in the data collection procedure are present due to cost or other restrictions. Optimal experimental designs determine parameters that in some appropriate sense make the data the most informative possible. In a Bayesian setting this is translated to updatin…
Bayesian optimal design of experiments (BODE) has been successful in acquiring information about a quantity of interest (QoI) which depends on a black-box function. BODE is characterized by sequentially querying the function at specific designs selected by an infill-sampling criterion. However, most current BODE method…
Develops methods to estimate gradient of EIG for Bayesian Experimental Design.
problem Optimizing Bayesian inference through efficient experimental design.
method Introduces UEEG-MCMC and BEEG-AP methods for estimating EIG gradient.
result Both methods improve upon existing benchmarks in EIG optimization.
Enhances robustness in experimental design through Generalised Bayesian inference.
problem Poor inference and estimates of information gain when statistical model is incorrectly specified.
method Generalised Bayesian (Gibbs) inference framework applied to experimental design.
result GBOED enhances robustness to outliers and incorrect assumptions about noise distribution.
Gradient-free framework for Bayesian experimental design in complex systems.
problem Optimal experimental design in systems where gradient information is unavailable.
method Combines EKI and ALDI for optimization and sampling, with approximations for scalable utility estimation.
result Demonstrates robust, accurate, and efficient experimental design in various complex systems.
Proposes EPIG for active learning to improve predictive performance.
problem Suboptimal predictive performance of traditional active learning methods.
method Introduces EPIG, a new acquisition function measuring information gain in the space of predictions.
result EPIG leads to stronger predictive performance compared to BALD across various datasets and models.
Bayesian calibration for BCP self-assembly models using image data and measure transport.
problem Calibrating models of BCP self-assembly from image data with aleatory uncertainty.
method Likelihood-free inference via measure transport and summary statistics.
result Expected information gains can be computed efficiently for model calibration.
Upper bound derived for informed traders' gains in a model, akin to thermodynamics.
problem Informed traders' gains in a financial model with finite horizon.
method Bayesian inference and entropic inequality.
result Upper bound for expected gain, analogous to thermodynamics.
Maximizes robustness in Bayesian experimental design under model uncertainty.
problem Brittleness of Bayesian experimental design under model misspecification.
method Formulates as a max--min game, uses Sibson's α-MI, and adopts PAC-Bayes framework.
result Establishes robust belief update and conditional information gain measure.
The optimal selection of experimental conditions is essential to maximizing the value of data for inference and prediction, particularly in situations where experiments are time-consuming and expensive to conduct. We propose a general mathematical framework and an algorithmic approach for optimal experimental design wi…
We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the global maximum. PES codifies this intractable acquisition function in terms of t…
Estimating arbitrary quantities of interest (QoIs) that are non-linear operators of complex, expensive-to-evaluate, black-box functions is a challenging problem due to missing domain knowledge and finite budgets. Bayesian optimal design of experiments (BODE) is a family of methods that identify an optimal design of exp…
Active sampling algorithm improves accuracy of inferred scores from pairwise comparisons.
problem Inference of accurate scores from time-consuming pairwise comparisons.
method Approximate message passing and expected information gain maximization.
result ASAP offers the highest accuracy of inferred scores compared to existing methods.
As Computer Vision moves from a passive analysis of pixels to active analysis of semantics, the breadth of information algorithms need to reason over has expanded significantly. One of the key challenges in this vein is the ability to identify the information required to make a decision, and select an action that will …
We study the problem of causal discovery through targeted interventions. Starting from few observational measurements, we follow a Bayesian active learning approach to perform those experiments which, in expectation with respect to the current model, are maximally informative about the underlying causal structure. Unli…
EAGLE improves reproducibility and stability of model explanations.
problem Creating reliable explanations for opaque machine learning models.
method Formulates perturbation selection as an information-theoretic active learning problem.
result EAGLE learns a linear surrogate model with feature importance scores and uncertainty estimates.
New BED method handles online inference for partially observed dynamical systems.
problem Optimizing data collection for partially observable, partially online dynamical systems.
method Derived estimators of expected information gain and its gradient for SSMs, using nested particle filters.
result Successfully handles both partial observability and online inference in realistic models.
The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.
problem Understanding the reliability and uncertainty of probabilistic predictions.
method Developed decomposition identities for proper losses, quantifying reliability, residual uncertainty, and information gain.
result A three-term identity for classification scores, revealing miscalibration, grouping term, and feature-level uncertainty.
In this article we will propose a completely new point of view for solving one of the most important paradoxes concerning game theory. The solution develop shifts the focus from the result to the strategy s ability to operate in a cognitive way by exploiting useful information about the system. In order to determine fr…
Bayesian optimal experimental design (BOED) is a principled framework for making efficient use of limited experimental resources. Unfortunately, its applicability is hampered by the difficulty of obtaining accurate estimates of the expected information gain (EIG) of an experiment. To address this, we introduce several …
PASOA optimizes Bayesian design by improving SMC samplers and EIG.
problem Sequential design optimization for accurate parameter inference.
method Sequential optimization using contrastive estimation, SMC samplers, and tempering.
result PASOA optimizes design and inference with improved consistency.
FairBED: A Bayesian Experimental Design Approach to Gathering Fairer Data
problem Ensuring fairness in machine learning
method Bayesian Experimental Design
result Improved fairness-accuracy trade-offs
New method boosts BOED using SBI and neural likelihood.
problem Maximizing EIG in BOED with intractable likelihood.
method Neural likelihood estimation, multi-start gradient ascent.
result Significantly improved BOED performance over state-of-the-art.
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on Shannon's concept of entropy include realization information, Kullback-Leibler divergenc…
Transformers for binary decisions are sensitive to evidence order, leading to unreliable outcomes.
problem Order sensitivity in Transformers for binary decisions leads to unreliable outcomes.
method Formalized an expectation-realization gap and developed QMV and EDFL bounds.
result Uniform permutation mixtures reduce dispersion and improve reliability.
New active learning strategy improves decision-making accuracy.
problem Maximizing decision-making accuracy in sequential data acquisition.
method Introduces a novel active learning criterion that maximizes expected information gain on the posterior decision distribution.
result Improved performance in decision-making accuracy compared to existing alternatives.
We introduce an interactive market setup with sequential auctions where agents receive variegated signals with a known deadline. The effects of differential information and mutual learning on the allocation of overall profit \& loss (P\&L) and the pace of price discovery are analysed. We characterise the signal-based e…
Securely evaluates the benefits of merging datasets for causal estimation.
problem Challenges in assessing the value of merging datasets for causal treatment effect estimation.
method Cryptographically secure multi-party computation to evaluate Expected Information Gain (EIG) while ensuring privacy.
result Demonstrates the first privacy-preserving method for dataset acquisition tailored to causal estimation.
Meta-active learning optimizes control of safety-critical systems by efficiently learning dynamics and configurations.
problem Efficiently learning system dynamics and optimal configurations for safety-critical systems like deep brain stimulation.
method Meta-learning an acquisition function using LSTM, cast as meta-learning, with a mixed-integer linear program policy.
result Achieved a 46% increase in information gain and a 20% speedup in computation time over baselines.
Introduces relative information gain for improving Gaussian process regression rates.
problem Improving the sample complexity of estimating or maximizing unknown functions.
method Introduces relative information gain, interpolates between effective dimension and information gain, and proves PAC-Bayesian bounds.
result Obtains minimax-optimal rates of convergence through the relative information gain.
Unified framework for portfolio optimization using gain PDF.
problem Optimizing portfolios with control over high profits.
method Unified approach incorporating various PO methods using gain PDF.
result Directly matching target PDF for maximal control over PO.
This paper extends SLS controllers to two stocks, proving the RPE property with cross-coupling.
problem Extending SLS controllers to two stocks without exploiting correlations.
method Developed a novel architecture for cross-coupling two SLS controllers, derived a closed-form expected value, and proved the RPE property.
result Guaranteed RPE property with cross-coupling for a large class of stock dynamics.
Financial market dynamics compared to thermodynamics.
problem Understanding the dynamics of financial markets through thermodynamic principles.
method Analogy with Szilárd information engine to derive market temperature and information extraction.
result Informed traders' gains are bounded by market temperature and information.
GO-CBED optimizes experiments for specific causal queries, improving efficiency.
problem Efficiently infer causal relationships with limited resources.
method Goal-oriented Bayesian framework that maximizes expected information gain on user-specified causal quantities.
result GO-CBED outperforms existing methods in various causal tasks, especially with limited budgets.
It is of increasing importance to develop learning methods for ranking. In contrast to many learning objectives, however, the ranking problem presents difficulties due to the fact that the space of permutations is not smooth. In this paper, we examine the class of rank-linear objective functions, which includes popular…
We study the gain of an insider having private information which concerns the default risk of a counterparty. More precisely, the default time τis modelled as the first time a stochastic process hits a random barrier L. The insider knows this barrier (as it can be the case for example for the manager of the counterpart…
There are (at least) three approaches to quantifying information. The first, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as the length of the shortest program producing it. The second, Shannon information…
This paper treats prediction markets as Bayesian inverse problems to quantify uncertainty and identify event outcomes.
problem Uncertainty and identifiability in prediction market outcomes from price-volume histories.
method Formulates prediction markets as Bayesian inverse problems, introduces a log-odds observation model, and derives posterior uncertainty quantification and identifiability criteria.
result Explicit diagnostics for informative and stable inference regimes, and validation through synthetic data experiments.
Bayesian design improves by reducing policy training cost.
problem Double intractability in expected information gain limits policy learning.
method Score matching to isolate EIG, then train policy singly intractably.
result Reduced computational burden for policy training, allowing multiple iterations.
C-IP improves LLMs' query selection for interactive tasks by estimating uncertainty robustly.
problem Minimizing the number of queries for interactive LLMs.
method Conformal Information Pursuit (C-IP) using conformal prediction sets.
result C-IP achieves better predictive performance and shorter query-answer chains.