Kernel Bayes' rule has been proposed as a nonparametric kernel-based method to realize Bayesian inference in reproducing kernel Hilbert spaces. However, we demonstrate both theoretically and experimentally that the prediction result by kernel Bayes' rule is in some cases unnatural. We consider that this phenomenon is i…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Bayes rule”
Bayes classifier cannot be learned from noisy labels without knowing noise distribution.
problem Learning a Bayes classifier from noisy labels when the noise distribution is unknown.
method Demonstrates the identifiability issues and proposes a simple algorithm for learning the Bayes decision rule.
result The Bayes decision rule is generally unidentified and cannot be learned without knowing the noise distribution.
Paper extends transfer learning for decision rules, improving treatment rule estimation.
problem Estimating optimal individualized treatment rules under changing conditions.
method Bayes decision rules and low-dimensional empirical risk minimization.
result Consistent estimators and risk bounds established under mild conditions.
When Is Generalized Bayes Bayesian? A Decision-Theoretic Characterization of Loss-Based Updatingstat.ME
New decision-theoretic characterization separates belief and decision posteriors.
problem Understanding the conditions under which loss-based updating coincides with Bayesian updating.
method Decision-theoretic approach to distinguish belief and decision posteriors.
result Generalized Bayes coincides with ordinary Bayesian updating only if the loss is proportional to negative log-likelihood.
Paper proves fair classification can be done via simple thresholding.
problem Achieving fair binary classification subject to group fairness constraints.
method Proves Bayes optimal fair learning rule is a group-wise thresholding rule over the Bayes regressor with randomization.
result Proposes an efficient unconstrained optimization algorithm for post-processing fair classification.
This work bounds classification error in machine learning for low Bayes error conditions.
problem Understanding the error mismatch between Bayes error and model-based classification error.
method Applying classification error bounds to study the relationship with Kullback-Leibler divergence and proposing a linear approximation for low Bayes error conditions.
result A linear approximation of the classification error bound for low Bayes error conditions is proposed.
VB uses natural gradients in information geometry.
problem Estimating or computing natural gradients in VB.
method Natural-gradient descent algorithm and Bayesian Learning Rule.
result Simplification of Bayes' rule and generalization of quadratic surrogates.
Kernel Bayes' rulestat.ML
A nonparametric kernel-based method for realizing Bayes' rule is proposed, based on representations of probabilities in reproducing kernel Hilbert spaces. Probabilities are uniquely characterized by the mean of the canonical map to the RKHS. The prior and conditional probabilities are expressed in terms of RKHS functio…
We present a particle flow realization of Bayes' rule, where an ODE-based neural operator is used to transport particles from a prior to its posterior after a new observation. We prove that such an ODE operator exists. Its neural parameterization can be trained in a meta-learning framework, allowing this operator to re…
CT compares two distributions using Bayes' theorem and chain rule.
problem Measuring the difference between two probability distributions.
method Conditional transport (CT) using chain rule and Bayes' theorem.
result CT strikes a good balance between mode-covering and mode-seeking behaviors.
Paper explores Bayes rule for Gaussian mixtures with missing data, outperforming supervised classifiers.
problem Improving classification accuracy in partially classified samples with missing data.
method Generative model framework with missing-data mechanism, Bayes rule allocation.
result Bayes rule classifier with missing-data mechanism outperforms fully supervised classifiers in various conditions.
We consider using an ensemble of binary classifiers for transductive prediction, when unlabeled test data are known in advance. We derive minimax optimal rules for confidence-rated prediction in this setting. By using PAC-Bayes analysis on these rules, we obtain data-dependent performance guarantees without distributio…
Differentially-private Bayes consistency rule for binary classification and density estimation.
problem Privacy constraints limit private learning in the distribution-free PAC model.
method Constructs a universally Bayes consistent learning rule that satisfies differential privacy.
result Private learning is possible for arbitrary distributions, even with a single algorithm.
In order to interact intelligently with objects in the world, animals must first transform neural population responses into estimates of the dynamic, unknown stimuli which caused them. The Bayesian solution to this problem is known as a Bayes filter, which applies Bayes' rule to combine population responses with the pr…
This paper establishes the asymptotic consistency of the {\it loss-calibrated variational Bayes} (LCVB) method. LCVB was proposed in~\cite{LaSiGh2011} as a method for approximately computing Bayesian posteriors in a `loss aware' manner. This methodology is also highly relevant in general data-driven decision-making con…
The paper tackles fair classification with multiple sensitive features.
problem Existing fair classification methods often consider a single sensitive feature, but in practice, individuals are defined by multiple sensitive features.
method Characterizes Bayes-optimal fair classifiers for multiple sensitive features under various fairness measures, proposing in-processing and post-processing algorithms.
result Bayes-optimal fair classifiers for multiple sensitive features are instance-dependent thresholding rules that rely on a weighted sum of group membership probabilities.
A tutorial on VAEs explaining its derivation and applications.
problem Understanding the VAE model and its limitations.
method Explains VAE through probabilistic and information theoretic perspectives.
result Identifies two common misconceptions and their practical consequences.
This research improves binary classification by balancing overfitting and generalization with a novel Bayesian approach.
problem Improving binary classification models to avoid overfitting and generalize well.
method Introduces a PAC-Bayes type learning rule with a balancing parameter λ to balance training error and KL divergence to a prior.
result A choice of λ ensures uniformly vanishing excess loss, even in the agnostic case, by under-regularizing or over-regularizing appropriately.
A novel Bayesian computation method using importance weighting improves numerical stability and performance.
problem Bayesian computation stability and performance issues.
method Nonparametric approach via feature means, importance weighting, and kernel Bayes' rule.
result Importance weighted kernel Bayes' rule yields superior numerical stability and performance.
Study optimizes best-arm identification with minimax and Bayes strategies.
problem Efficiently identifying the best arm in fixed-budget scenarios.
method Adaptive procedure with two stages: pilot phase and minimax game.
result Single strategy is asymptotically minimax and Bayes optimal.
New priors can update posteriors without re-estimating likelihoods.
problem Degradation of classification approaches when class priors change.
method Recompute posteriors using recovered likelihoods from original posteriors and new priors.
result Dynamic update of original posteriors is possible without re-estimating likelihoods.
We develop a class of rules spanning the range between quadratic discriminant analysis and naive Bayes, through a path of sparse graphical models. A group lasso penalty is used to introduce shrinkage and encourage a similar pattern of sparsity across precision matrices. It gives sparse estimates of interactions and pro…
Bayesian framework for policy learning in decision problems.
problem Maximizing expected welfare in decision-making problems.
method Loss-based Bayesian updating and squared-loss surrogate for welfare maximization.
result General Bayes posterior over decision rules with Gaussian pseudo-likelihood interpretation.
A method for selecting pseudo-labeled data in semi-supervised learning using generalized Bayes and soft revision.
problem Selecting pseudo-labeled data for semi-supervised learning with robustness to uncertainty.
method Using credal sets and the Gamma-Maximin method with soft revision to update priors and select pseudo-labeled data.
result The Gamma-Maximin method with soft revision can achieve promising results, especially in scenarios with low labeled data proportions.
Solves open problem on universally consistent online learning with unbounded losses.
problem Open problem on universally consistent online learning with unbounded losses.
method Constructs random measurable partitions of the instance space.
result Simple memorization rule is optimistically universal for any unbounded loss.
A new framework for consistent segmentation evaluation reduces operating losses.
problem Inconsistent thresholding-based segmentation methods lead to suboptimal solutions.
method Developed a consistent ranking-based framework (RankDice/RankIoU) using Bayes rules and Dice-/IoU-calibration.
result The proposed framework is Dice-/IoU-calibrated and provides excess risk bounds and convergence rates.
Estimates classification rules from partially classified data.
problem Estimating Bayes' rule for unclassified observations in partially classified data.
method Fitting a g-component mixture model by maximum likelihood (ML) via the EM algorithm.
result Asymptotic relative efficiency (ARE) of Bayes' rule estimated from partially classified samples.
Proposes a method to reconcile count time series forecasts.
problem No formal framework for probabilistic reconciliation of count time series.
method Generalizes Bayes' rule for reconciling real-valued and count variables.
result Improves forecast accuracy for count variables compared to Gaussian reconciliation.
Study post-hoc Learning to Defer using density-ratio losses.
problem Optimizing decision-making between models and experts.
method Density-ratio losses for post-hoc L2D scorers, derived from class-probability estimation.
result The approach recovers known results and introduces new connections to expert comparison and anomaly detection.
Bayesian model-based reinforcement learning is a formally elegant approach to learning optimal behaviour under model uncertainty, trading off exploration and exploitation in an ideal way. Unfortunately, finding the resulting Bayes-optimal policies is notoriously taxing, since the search space becomes enormous. In this …
We analyze a model of learning and belief formation in networks in which agents follow Bayes rule yet they do not recall their history of past observations and cannot reason about how other agents' beliefs are formed. They do so by making rational inferences about their observations which include a sequence of independ…
Develops methods for fair classification under linear disparity constraints.
problem Disparate impacts of machine learning algorithms on protected groups.
method Bayes-optimal fair classification methods via pre-, in-, and post-processing.
result Explicit forms of Bayes-optimal fair classifiers under linear disparity measures.
Refinement revisited with connections to Bayes error, conditional entropy and calibrated classifiersstat.ML
The concept of refinement from probability elicitation is considered for proper scoring rules. Taking directions from the axioms of probability, refinement is further clarified using a Hilbert space interpretation and reformulated into the underlying data distribution setting where connections to maximal marginal diver…
A new method predicts true classes from positive and unlabeled data with additional labeled observations.
problem Predicting true classes from positive and unlabeled data with selection bias.
method Introduces augmented PU prediction, allowing feature-dependent labeling, and compares various empirical Bayes rules.
result The variational autoencoder-based method performs similarly or better than other methods and improves accuracy for unlabeled samples.
A new procedure for learning cost-sensitive SVM(CS-SVM) classifiers is proposed. The SVM hinge loss is extended to the cost sensitive setting, and the CS-SVM is derived as the minimizer of the associated risk. The extension of the hinge loss draws on recent connections between risk minimization and probability elicitat…
Naive Bayes can be used as a discriminative classifier, matching the definition of logistic regression.
problem The definition of generative and discriminative classifiers.
method Comparing Naive Bayes and logistic regression, showing they can be used in either generative or discriminative ways.
result Naive Bayes can be used as a discriminative classifier.
RoPAWS improves semi-supervised learning on uncurated data.
problem Efficiency of semi-supervised learning with real-world unlabeled data.
method Reinterprets PAWS as a generative classifier and calibrates predictions using densities of labeled and unlabeled data.
result Significant improvement in performance for uncurated data.
Resampling-based Confidence Intervals for Model-free Robust Inference on Optimal Treatment Regimesstat.ME
We propose a new procedure for inference on optimal treatment regimes in the model-free setting, which does not require to specify an outcome regression model. Existing model-free estimators for optimal treatment regimes are usually not suitable for the purpose of inference, because they either have nonstandard asympto…
This article introduces a framework to estimate the value of evidence-based decision making.
problem Lack of empirical tools to assess the value of evidence-based decision making and optimize statistical precision.
method Empirical framework using parametric and nonparametric empirical Bayes methods.
result The value of statistical evidence depends on how organizations translate it into policy decisions.
New bounds prevent degradation in high-dimensional signal estimation.
problem Statistical learning bounds degradation with increasing dimensionality.
method Investigates linear prediction rules under structural assumptions.
result Derives upper and lower bounds on generalization error.
Bayesian unlearning uses Bayes' rule to remove data from a model, but faces challenges in obtaining the exact posterior.
problem Removing data from a trained model while maintaining model accuracy.
method Uses Laplace approximation and Variational Inference to approximate the updated posterior.
result Insights on the applicability of Bayesian unlearning in practical scenarios for neural networks.
New method computes discriminative classifiers from generative models.
problem Discriminative vs generative classifiers are often seen as distinct, but this work shows they can be equivalent.
method General theoretical result showing generative classifiers can be computed discriminatively.
result Bayesian Maximum Posterior classifier from generative models matches discriminative classifier definition.
Test-time training adapts a pretrained model to each prompt via parameter updates, improving accuracy under pretraining-to-test distribution shifts.
problem Improving accuracy of pretrained models under distribution shifts.
method Explaining TTT behavior through a decision-theoretic lens.
result TTT reduces prediction error when updates are spectrally matched to the prompt's signal-to-noise ratio and aligned with query-relevant eigen-directions.
We presented Bayesian portfolio selection strategy, via the factor asset pricing model. If the market is information efficient, the proposed strategy will mimic the market; otherwise, the strategy will outperform the market. The strategy depends on the selection of a portfolio via Bayesian multiple testing methodol…
A new optimization algorithm for Gaussian Variational Inference on precision matrices.
problem Complex models with positive definite constraints on covariance matrices.
method Manifold Gaussian Variational Bayes (MGVBP) with natural gradient updates.
result Empirically validated as a feasible and efficient solution for VI in complex models.
A new concept of confidence in learning is defined and analyzed.
problem Understanding and quantifying trust in learning processes.
method Formal axioms, continuum measures, vector fields, loss functions.
result Confidence can be represented and optimized in learning.
A new generative classification strategy outperforms existing methods in class-incremental learning.
problem Incrementally training deep neural networks to recognize new classes is challenging.
method Proposes learning the joint distribution p(x,y) and performing classification using Bayes' rule, implemented with variational autoencoders and importance sampling.
result Performs very well on continual learning benchmarks, outperforming existing baselines.
This work investigates the case of a network of agents that attempt to learn some unknown state of the world amongst the finitely many possibilities. At each time step, agents all receive random, independently distributed private signals whose distributions are dependent on the unknown state of the world. However, it m…