New decision-theoretic characterization separates belief and decision posteriors.
problem Understanding the conditions under which loss-based updating coincides with Bayesian updating.
method Decision-theoretic approach to distinguish belief and decision posteriors.
result Generalized Bayes coincides with ordinary Bayesian updating only if the loss is proportional to negative log-likelihood.
Paper extends transfer learning for decision rules, improving treatment rule estimation.
problem Estimating optimal individualized treatment rules under changing conditions.
method Bayes decision rules and low-dimensional empirical risk minimization.
result Consistent estimators and risk bounds established under mild conditions.
Bayes classifier cannot be learned from noisy labels without knowing noise distribution.
problem Learning a Bayes classifier from noisy labels when the noise distribution is unknown.
method Demonstrates the identifiability issues and proposes a simple algorithm for learning the Bayes decision rule.
result The Bayes decision rule is generally unidentified and cannot be learned without knowing the noise distribution.
This work bounds classification error in machine learning for low Bayes error conditions.
problem Understanding the error mismatch between Bayes error and model-based classification error.
method Applying classification error bounds to study the relationship with Kullback-Leibler divergence and proposing a linear approximation for low Bayes error conditions.
result A linear approximation of the classification error bound for low Bayes error conditions is proposed.
This paper establishes the asymptotic consistency of the {\it loss-calibrated variational Bayes} (LCVB) method. LCVB was proposed in~\cite{LaSiGh2011} as a method for approximately computing Bayesian posteriors in a `loss aware' manner. This methodology is also highly relevant in general data-driven decision-making con…
Bayesian framework for policy learning in decision problems.
problem Maximizing expected welfare in decision-making problems.
method Loss-based Bayesian updating and squared-loss surrogate for welfare maximization.
result General Bayes posterior over decision rules with Gaussian pseudo-likelihood interpretation.
This article introduces a framework to estimate the value of evidence-based decision making.
problem Lack of empirical tools to assess the value of evidence-based decision making and optimize statistical precision.
method Empirical framework using parametric and nonparametric empirical Bayes methods.
result The value of statistical evidence depends on how organizations translate it into policy decisions.
Study optimizes best-arm identification with minimax and Bayes strategies.
problem Efficiently identifying the best arm in fixed-budget scenarios.
method Adaptive procedure with two stages: pilot phase and minimax game.
result Single strategy is asymptotically minimax and Bayes optimal.
Kernel Bayes' rule has been proposed as a nonparametric kernel-based method to realize Bayesian inference in reproducing kernel Hilbert spaces. However, we demonstrate both theoretically and experimentally that the prediction result by kernel Bayes' rule is in some cases unnatural. We consider that this phenomenon is i…
Bayesian neural networks are shown to be minimax and admissible under certain conditions.
problem Optimality of Bayesian neural networks in deep learning models.
method Analysis of decision rules induced by BNNs in the normal location model under quadratic loss.
result A hyperprior on the effective output variance yields a minimax and admissible decision rule.
A new procedure for learning cost-sensitive SVM(CS-SVM) classifiers is proposed. The SVM hinge loss is extended to the cost sensitive setting, and the CS-SVM is derived as the minimizer of the associated risk. The extension of the hinge loss draws on recent connections between risk minimization and probability elicitat…
New GAN formulation addresses mode collapse issue.
problem Mode collapse in GANs.
method Randomized decision rules, empirical Bayes, stochastic gradient MCMC.
result Proposed method converges to Nash equilibrium.
Study on limits of LLM-based multi-agent planning reliability.
problem Reliability limits of LLM-based multi-agent planning.
method Modeling LLM-based multi-agent architecture as a decision network, showing dominance by centralized Bayes decision maker.
result Optimizing multi-agent directed acyclic graphs under communication budget is equivalent to choosing a constrained experiment.
Test-time training adapts a pretrained model to each prompt via parameter updates, improving accuracy under pretraining-to-test distribution shifts.
problem Improving accuracy of pretrained models under distribution shifts.
method Explaining TTT behavior through a decision-theoretic lens.
result TTT reduces prediction error when updates are spectrally matched to the prompt's signal-to-noise ratio and aligned with query-relevant eigen-directions.
Study post-hoc Learning to Defer using density-ratio losses.
problem Optimizing decision-making between models and experts.
method Density-ratio losses for post-hoc L2D scorers, derived from class-probability estimation.
result The approach recovers known results and introduces new connections to expert comparison and anomaly detection.
The paper proposes multicalibration to improve matching in graphs with imperfect predictors.
problem Finding the best matching in graphs with imperfect predictors.
method Introduces multicalibration as a fairness notion to ensure unbiasedness on protected sets of contexts.
result Constructing a multicalibrated predictor that outperforms standard optimal rules in matching algorithms.
Paper proves fair classification can be done via simple thresholding.
problem Achieving fair binary classification subject to group fairness constraints.
method Proves Bayes optimal fair learning rule is a group-wise thresholding rule over the Bayes regressor with randomization.
result Proposes an efficient unconstrained optimization algorithm for post-processing fair classification.
We propose a new procedure for inference on optimal treatment regimes in the model-free setting, which does not require to specify an outcome regression model. Existing model-free estimators for optimal treatment regimes are usually not suitable for the purpose of inference, because they either have nonstandard asympto…
VB uses natural gradients in information geometry.
problem Estimating or computing natural gradients in VB.
method Natural-gradient descent algorithm and Bayesian Learning Rule.
result Simplification of Bayes' rule and generalization of quadratic surrogates.
A nonparametric kernel-based method for realizing Bayes' rule is proposed, based on representations of probabilities in reproducing kernel Hilbert spaces. Probabilities are uniquely characterized by the mean of the canonical map to the RKHS. The prior and conditional probabilities are expressed in terms of RKHS functio…
We present a particle flow realization of Bayes' rule, where an ODE-based neural operator is used to transport particles from a prior to its posterior after a new observation. We prove that such an ODE operator exists. Its neural parameterization can be trained in a meta-learning framework, allowing this operator to re…
CT compares two distributions using Bayes' theorem and chain rule.
problem Measuring the difference between two probability distributions.
method Conditional transport (CT) using chain rule and Bayes' theorem.
result CT strikes a good balance between mode-covering and mode-seeking behaviors.
Paper explores Bayes rule for Gaussian mixtures with missing data, outperforming supervised classifiers.
problem Improving classification accuracy in partially classified samples with missing data.
method Generative model framework with missing-data mechanism, Bayes rule allocation.
result Bayes rule classifier with missing-data mechanism outperforms fully supervised classifiers in various conditions.
We consider using an ensemble of binary classifiers for transductive prediction, when unlabeled test data are known in advance. We derive minimax optimal rules for confidence-rated prediction in this setting. By using PAC-Bayes analysis on these rules, we obtain data-dependent performance guarantees without distributio…
The paper optimizes A/B tests by balancing lift and cost in large-scale settings.
problem Balancing lift and cost in A/B tests for large-scale experimentation.
method Empirical Bayes approach using a greedy knapsack algorithm to rank experiments based on lift-to-cost ratio, incorporating local false discovery rate (lfdr).
result The proposed method maximizes expected profit while controlling false discovery rate, demonstrating superior performance in large-scale settings.
This paper explores minimax-Bayes solutions for reinforcement learning problems.
problem How to select appropriate priors for decision making under uncertainty in sequential decision making.
method Study of minimax-Bayes solutions for various reinforcement learning problems.
result Minimax policies are more robust than standard priors.
A new method learns proper multiclass losses and probabilities.
problem Learning proper multiclass losses for complex classification tasks.
method Extends monotonicity to multiclass problems using convex functions.
result Consistently outperforms natural multiclass baseline on up to 1,000 class datasets.
Differentially-private Bayes consistency rule for binary classification and density estimation.
problem Privacy constraints limit private learning in the distribution-free PAC model.
method Constructs a universally Bayes consistent learning rule that satisfies differential privacy.
result Private learning is possible for arbitrary distributions, even with a single algorithm.
In order to interact intelligently with objects in the world, animals must first transform neural population responses into estimates of the dynamic, unknown stimuli which caused them. The Bayesian solution to this problem is known as a Bayes filter, which applies Bayes' rule to combine population responses with the pr…
The paper tackles fair classification with multiple sensitive features.
problem Existing fair classification methods often consider a single sensitive feature, but in practice, individuals are defined by multiple sensitive features.
method Characterizes Bayes-optimal fair classifiers for multiple sensitive features under various fairness measures, proposing in-processing and post-processing algorithms.
result Bayes-optimal fair classifiers for multiple sensitive features are instance-dependent thresholding rules that rely on a weighted sum of group membership probabilities.
A new sequential method estimates Poisson means in streaming data, achieving optimality and efficiency.
problem Estimating Poisson means in a streaming, or online, framework.
method A quasi-Bayesian approach based on Newton's algorithm for a sequential estimate.
result Established frequentist guarantees including consistency and asymptotic optimality.
A tutorial on VAEs explaining its derivation and applications.
problem Understanding the VAE model and its limitations.
method Explains VAE through probabilistic and information theoretic perspectives.
result Identifies two common misconceptions and their practical consequences.
This research improves binary classification by balancing overfitting and generalization with a novel Bayesian approach.
problem Improving binary classification models to avoid overfitting and generalize well.
method Introduces a PAC-Bayes type learning rule with a balancing parameter λ to balance training error and KL divergence to a prior.
result A choice of λ ensures uniformly vanishing excess loss, even in the agnostic case, by under-regularizing or over-regularizing appropriately.
A novel Bayesian computation method using importance weighting improves numerical stability and performance.
problem Bayesian computation stability and performance issues.
method Nonparametric approach via feature means, importance weighting, and kernel Bayes' rule.
result Importance weighted kernel Bayes' rule yields superior numerical stability and performance.
MARLA uses deep reinforcement learning for multi-agent AHT, reducing Bayes risk.
problem Decentralized active hypothesis testing among multiple agents.
method MARLA algorithm using deep multi-agent reinforcement learning.
result MARLA outperforms single-agent learning in AHT problems.
Optimizes portfolio construction using Bayesian methods and variational techniques.
problem Balancing reward and risk in portfolio construction.
method Bayesian decision-theoretic formulation, saddle-point problem, variational Bayes relaxation, efficient algorithm, provable convergence.
result Proves statistical consistency of proposed decision with optimal Bayesian decision.
Reward augmented maximum likelihood (RAML), a simple and effective learning framework to directly optimize towards the reward function in structured prediction tasks, has led to a number of impressive empirical successes. RAML incorporates task-specific reward by performing maximum-likelihood updates on candidate outpu…
Bayesian optimization is a sample-efficient approach to global optimization that relies on theoretically motivated value heuristics (acquisition functions) to guide its search process. Fully maximizing acquisition functions produces the Bayes' decision rule, but this ideal is difficult to achieve since these functions …
New priors can update posteriors without re-estimating likelihoods.
problem Degradation of classification approaches when class priors change.
method Recompute posteriors using recovered likelihoods from original posteriors and new priors.
result Dynamic update of original posteriors is possible without re-estimating likelihoods.
We develop a class of rules spanning the range between quadratic discriminant analysis and naive Bayes, through a path of sparse graphical models. A group lasso penalty is used to introduce shrinkage and encourage a similar pattern of sparsity across precision matrices. It gives sparse estimates of interactions and pro…
Proposes a method for selecting important variables in high-dimensional data.
problem High-dimensional classification problems with many noise variables.
method Probability-based nonparametric multiple-class classification method with variable selection.
result The method can have prediction power similar to Bayes rule and retains interpretability.
This paper uses deep learning to improve network threat detection in finance.
problem Detecting unknown threats in large-scale data applications.
method Uses deep learning for advanced threat detection.
result Improves protective measures in the financial industry.
This paper argues against using calibration metrics for assessing posterior probabilities and proposes expected proper scoring rules instead.
problem The assessment of posterior probabilities generated by machine learning classifiers using calibration metrics is flawed and should be replaced with expected proper scoring rules.
method The paper reviews proper scoring rules from a practical perspective, explains why expected PSRs are a principled measure of posterior quality, and introduces a new calibration metric called calibration loss.
result Calibration loss is superior to expected calibration error and expected score divergence calibration metrics for assessing posterior probabilities.
New classifiers account for context-specific independences.
problem Restrictions in generative models for classification.
method Staged tree classifiers that account for context-specific independences.
result Staged tree classifiers achieve competitive classification accuracy.
A method for selecting pseudo-labeled data in semi-supervised learning using generalized Bayes and soft revision.
problem Selecting pseudo-labeled data for semi-supervised learning with robustness to uncertainty.
method Using credal sets and the Gamma-Maximin method with soft revision to update priors and select pseudo-labeled data.
result The Gamma-Maximin method with soft revision can achieve promising results, especially in scenarios with low labeled data proportions.
Solves open problem on universally consistent online learning with unbounded losses.
problem Open problem on universally consistent online learning with unbounded losses.
method Constructs random measurable partitions of the instance space.
result Simple memorization rule is optimistically universal for any unbounded loss.
A new framework for consistent segmentation evaluation reduces operating losses.
problem Inconsistent thresholding-based segmentation methods lead to suboptimal solutions.
method Developed a consistent ranking-based framework (RankDice/RankIoU) using Bayes rules and Dice-/IoU-calibration.
result The proposed framework is Dice-/IoU-calibrated and provides excess risk bounds and convergence rates.
A new method learns interpretable decision rules using submodular optimization.
problem Learning interpretable decision rules from data.
method Submodular optimization approach for selecting rules from a large set.
result The method effectively learns interpretable rule sets from real datasets.