Neural Empirical Bayes estimates source distributions from noisy simulations.
problem Estimating source distributions from noisy, simulated data.
method Uses neural density estimators to estimate a prior or source distribution over uncorrupted samples, then performs posterior inference.
result Recovering ground truth source distributions up to symmetries.
VBphenoR uses variational Bayes for EHR-based patient phenotyping.
problem Phenotyping patients from EHR data for targeted treatments.
method Variational Bayes Gaussian Mixture Model (GMM) and logistic regression.
result Closed-form inference for efficient patient phenotype determination.
Deep learning improves Bayes factor computation for likelihood-free models.
problem Computing Bayes factors for likelihood-free models is challenging.
method Proposes a deep learning estimator of Bayes factors using simulated data.
result Establishes consistency of the Deep Bayes Factor estimator.
New PAC-Bayes training method improves model generalization for unbounded loss.
problem Improving generalization of complex models under unbounded loss.
method Established new PAC-Bayes bound for unbounded loss, jointly training prior and posterior.
result Outperforms existing PAC-Bayes training algorithms and matches ERM accuracy.
A new method estimates Bayes error for deep networks, suggesting they may have reached the limit.
problem Evaluating the performance of deep learning models and detecting overfitting.
method A simple and direct Bayes error estimator based on uncertainty of class assignments.
result Deep networks may have reached the Bayes error limit for benchmark datasets.
This paper tackles non-vacuous generalization bounds in ReLU networks by resolving rescaling invariances.
problem Non-vacuous generalization guarantees for ReLU networks with rescaling invariances.
method Proposes a lifted representation to resolve rescaling invariances and studies KL-based rescaling-invariant PAC-Bayes bounds.
result KL-based rescaling-invariant PAC-Bayes bounds provide tighter guarantees and resolve discrepancies in network complexity.
Bayesian Empirical Bayes extends EB to complex structures using probabilistic symmetry.
problem Improving simultaneous inference in complex settings like arrays and graphs.
method Generalized empirical Bayes approach based on probabilistic symmetry.
result BEB outperforms existing methods in denoising arrays and spatial data.
A new sequential method estimates Poisson means in streaming data, achieving optimality and efficiency.
problem Estimating Poisson means in a streaming, or online, framework.
method A quasi-Bayesian approach based on Newton's algorithm for a sequential estimate.
result Established frequentist guarantees including consistency and asymptotic optimality.
Unified approach improves accuracy in private estimation.
problem Protecting sensitive information in statistical estimation.
method Unified Bayes Private Point (UBaPP) approach to Bayes point estimation under differential privacy constraints.
result Achieves better accuracy-privacy trade-off than traditional methods.
New method for high-dimensional linear regression using empirical Bayes.
problem Estimating prior in high-dimensional linear regression.
method Variational empirical Bayes approach with NPMLE and mean field approximation.
result Established asymptotic consistency and computational efficiency of the method.
Bayesian model-based reinforcement learning is a formally elegant approach to learning optimal behaviour under model uncertainty, trading off exploration and exploitation in an ideal way. Unfortunately, finding the resulting Bayes-optimal policies is notoriously taxing, since the search space becomes enormous. In this …
Study shows how classifiers can approach Bayes error in high-dimensional settings.
problem Generalization error in high-dimensional perceptrons.
method Proved a formula for generalization error using convex optimization and observed that logistic and hinge regression can approach Bayes error closely.
result Logistic and hinge regression can approach Bayes-optimal generalization error closely in high-dimensional settings.
Smart Bayes integrates generative and discriminative features for improved classification.
problem Improving classification performance by combining generative and discriminative modeling.
method Integrates generative likelihood-ratio features into a logistic-regression-style classifier.
result Often outperforms logistic regression and Naive Bayes in simulations and real data.
This paper introduces Bayes Hilbert spaces for efficient posterior approximation.
problem Efficient posterior approximation in Bayesian models for large datasets.
method Develops Bayes Hilbert spaces for posterior approximation and connects them to Bayesian coresets and kernel-based distances.
result Bayes Hilbert spaces provide a novel framework for posterior approximation that is computationally efficient.
Efficiently computes tree-Wasserstein barycenter for large-scale multilevel clustering and scalable Bayes.
problem Large-scale multilevel clustering and scalable Bayes problems.
method Proposes an efficient algorithm for tree-Wasserstein barycenter and variants.
result Significantly improves efficiency in computation and memory usage for large-scale applications.
ContraBAR uses contrastive learning to learn Bayes-optimal policies in RL.
problem Learning optimal policies for unknown tasks sampled from a known distribution.
method Proposes ContraBAR, a meta RL algorithm using contrastive predictive coding (CPC) for belief inference.
result ContraBAR achieves comparable performance to state-of-the-art methods and is computationally efficient.
The paper develops a method to accurately estimate the Bayes misclassification error rate.
problem Estimating the best achievable classifier performance without learning a Bayes-optimal classifier.
method Learning to benchmark using an ensemble of ε-ball estimators and Chebyshev approximation.
result The proposed method achieves an optimal mean squared error rate of O(N^(-1)) under a smoothness assumption.
New PAC-Bayes bounds for unbounded losses using Cramér-Chernoff techniques.
problem Developing bounds for unbounded losses in PAC-Bayesian settings.
method Introducing a new PAC-Bayes oracle bound using Cramér-Chernoff bounds and controlling random variable tails.
result Our bounds generalize and improve upon previous results, providing more informative and potentially tighter bounds.
Unified approach for fair classification with overlapping groups.
problem Ensuring fairness across multiple overlapping groups in prediction problems.
method Probabilistic population analysis leading to Bayes-optimal classifier, unifying existing methods.
result Outperforms baselines in fairness-performance tradeoff on real datasets.
Neural Bayes methods simplify fitting complex bivariate extremal models.
problem Inference on complex multivariate extremal dependence models with computationally expensive likelihood functions.
method Use neural networks to approximate Bayes estimators and classifiers for model selection.
result Proposed neural Bayes methods enable routine implementation of complex extreme-value dependence models.
Bayes-TrEx finds in-distribution examples for model inspection.
problem Challenges in interpreting neural networks, especially high-confidence failures and ambiguous classifications.
method Bayesian sampling approach to find in-distribution examples with specified prediction confidence.
result Bayes-TrEx enables more flexible holistic model analysis than just inspecting the test set.
We present a family of expectation-maximization (EM) algorithms for binary and negative-binomial logistic regression, drawing a sharp connection with the variational-Bayes algorithm of Jaakkola and Jordan (2000). Indeed, our results allow a version of this variational-Bayes approach to be re-interpreted as a true EM al…
We compare in this paper several feature selection methods for the Naive Bayes Classifier (NBC) when the data under study are described by a large number of redundant binary indicators. Wrapper approaches guided by the NBC estimation of the classification error probability out-perform filter approaches while retaining …
Flexible empirical Bayes for large-scale multiple linear regression.
problem Large-scale multiple linear regression with flexible priors and efficient computation.
method Adaptive shrinkage priors combined with variational approximations for hyperparameter estimation.
result The posterior mean from the empirical Bayes method solves a penalized regression problem.
Parameter estimation for model-based clustering using a finite mixture of normal inverse Gaussian (NIG) distributions is achieved through variational Bayes approximations. Univariate NIG mixtures and multivariate NIG mixtures are considered. The use of variational Bayes approximations here is a substantial departure fr…
We show how to compute lower bounds for the supremum Bayes error if the class-conditional distributions must satisfy moment constraints, where the supremum is with respect to the unknown class-conditional distributions. Our approach makes use of Curto and Fialkow's solutions for the truncated moment problem. The lower …
Paper compares two multi-label classification methods: Bayes metaclassifier and soft-confusion-matrix.
problem Comparing multi-label classification methods under the same framework.
method Bayes metaclassifier combined with problem-transformation approach, and soft-confusion-matrix classifier.
result Both methods perform similarly, but statistical analysis provides insights.
Paper proves fair classification can be done via simple thresholding.
problem Achieving fair binary classification subject to group fairness constraints.
method Proves Bayes optimal fair learning rule is a group-wise thresholding rule over the Bayes regressor with randomization.
result Proposes an efficient unconstrained optimization algorithm for post-processing fair classification.
Researchers compute Bayes error for classification models using normalizing flows.
problem Evaluating the inherent difficulty of classification problems.
method Invertible transformations and Gaussian base distributions to compute Bayes error.
result State-of-the-art models can achieve near-optimal accuracy but not always.
Unified approach to neural network learning with PAC-Bayes bounds.
problem Overfitting and extrapolation issues in neural networks.
method Combines variational inference and PAC-Bayes for scalable learning.
result Validates theory and shows better generalization on high-dimensional tasks.
Paper tightens PAC-Bayes bounds using coin-betting for better estimates.
problem Estimating mean of random elements with possibly S-dependent parameters.
method Refined PAC-Bayes proof strategy based on coin-betting framework.
result Derives tighter concentration inequalities for all sample sizes.
A scalable method for accurate inference of low-dimensional parameters in high-dimensional linear regression.
problem Statistical inference for low-dimensional parameters in high-dimensional linear regression models.
method Mean-field variational Bayes approach, focusing on nuisance parameters and conditional distributions.
result Competitive numerical performance and theoretical guarantees for estimation and uncertainty quantification.
The paper proves statistical consistency and fairness guarantees for a plug-in algorithm.
problem Establishing statistical guarantees for fairness-aware binary classification.
method Proves statistical consistency and derives finite sample guarantees for the plug-in algorithm.
result The plug-in algorithm is statistically consistent and guarantees fairness and differential privacy.
We propose an empirical Bayes estimator based on Dirichlet process mixture model for estimating the sparse normalized mean difference, which could be directly applied to the high dimensional linear classification. In theory, we build a bridge to connect the estimation error of the mean difference and the misclassificat…
Bayes-CPACE optimally explores continuous BAMDPs.
problem Model uncertainty in continuous state and action spaces.
method Covering state-belief-action space with samples, exploiting Lipschitz continuity.
result Near-optimal value function computed efficiently.
Paper develops PAC-Bayes bounds for unknown linear systems.
problem Learning controllers for unknown stochastic linear discrete-time systems.
method PAC-Bayes framework for data-dependent high probability bounds.
result Proposes efficient learning algorithms with theoretical guarantees.
PAC-Bayes bounds have been proposed to get risk estimates based on a training sample. In this paper the PAC-Bayes approach is combined with stability of the hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting is used with a Gaussian prior centered at the expected output. Thus a novelty of our …
Proposes MOPED method for choosing priors in Bayesian DNNs.
problem Challenges in specifying meaningful priors for deep neural networks.
method Two-stage hierarchical modeling with empirical Bayes.
result MOPED enables scalable variational inference and reliable uncertainty quantification.
Develops a fast variational approximation for high-dimensional empirical Bayes posteriors.
problem Optimal posterior computation in high-dimensional settings with prior tails effect.
method Variational approximation of empirical Bayes posterior with data-driven centers and thin-tailed conjugate priors.
result Retains optimal concentration rate properties and superior performance compared to existing methods.
New PAC-Bayes meta-learning method improves few-shot learning accuracy and calibration.
problem Few-shot learning with limited data.
method PAC-Bayes framework extended to meta-learning, estimating task-specific posteriors.
result State-of-the-art calibration and classification results on benchmarks.
Unified framework for Bayes-optimal classifiers under group fairness.
problem Mitigating disparate impacts from algorithmic predictions in high-stakes decision-making.
method Unified framework based on Neyman-Pearson argument for deriving Bayes-optimal classifiers under group fairness constraints.
result Proposes FairBayes method that directly controls disparity and achieves optimal fairness-accuracy tradeoff.
New method improves Bayesian inference for parametric models, robust to misspecification.
problem Inference can be untrustworthy when parametric models are wrong.
method Adaptive nonparametric corrections for parametric Bayesian models using generalized Bayes.
result The method achieves robustness and efficiency, converging fast when the parametric model is close to true.
A framework uses variational Bayes for solving inverse problems efficiently.
problem Solving inverse problems in various dimensions with flexibility and accuracy.
method Variational Bayes approximations with message passing and factor graph approach.
result Efficient algorithm updates for higher dimensions and computational advantage over MCMC.
PROBE algorithm efficiently solves sparse high-dimensional linear regression.
problem Sparse high-dimensional linear regression models with complex parameter spaces.
method Partitioned empirical Bayes ECM algorithm for computationally efficient MAP estimation.
result PROBE algorithm provides robust and efficient coordinate-wise optimization.
Optimal projections enhance Naive Bayes classification.
problem Improving Naive Bayes classification accuracy.
method Projection pursuit to find optimal linear projections.
result The approach substantially outperforms other models.
Proposes a sparse Naïve Bayes classifier to improve performance and interpretability.
problem Naïve Bayes assumes feature independence, which is violated in real data.
method Integrates feature correlation and performance measures for feature selection.
result Competitive results in accuracy, sparsity, and running times for balanced datasets.
Bayesian supertrees method uses variational Bayes for overlapping taxon subsets.
problem Inference of phylogenetic tree topologies for overlapping taxon sets.
method Variational Bayesian approach.
result Demonstrates effectiveness of variational Bayes for Bayesian supertrees.
New approach to meta-learning with variational Bayes for unlabeled data.
problem Improving machine learning systems' adaptability to small data.
method Generative meta-learning using AEVB and mean-field variational Bayes.
result Relevant VB updates do not depend on generative neural networks for certain models.