This work investigates training infinite mixtures with maximum likelihood for improved uncertainty quantification.
problem Improving uncertainty quantification in neural networks.
method Investigates training infinite mixtures with maximum likelihood instead of variational inference.
result The proposed method leads to stochastic networks with increased predictive variance, improved robustness, and higher entropy on out-of-distribution data.
The log-likelihood loss in heteroscedastic neural networks can lead to poor parameter estimates.
problem Capturing aleatoric uncertainty in deep learning models.
method Examine the log-likelihood loss in conjunction with gradient-based optimizers and propose an alternative formulation, β-NLL. result Using an appropriate β largely mitigates the issue of poor parameter estimates. We extend Bayes' theorem for upper probabilities considering likelihood uncertainty.
problem Addressing uncertainty in likelihood for upper probability bounds.
method Generalization of Wasserman and Kadane's result, considering both prior and likelihood uncertainty.
result A sufficient condition for the upper bound to become an equality.
ACNML method improves uncertainty estimation for deep networks.
problem Uncertainty estimation and calibration for deep neural networks under distribution shift.
method Approximate Bayesian inference to approximate CNML distribution.
result ACNML compares favorably to prior techniques for uncertainty estimation.
Proposes a method for credal prediction using relative likelihood.
problem Representing epistemic uncertainty with sets of probability distributions.
method Credal prediction based on relative likelihood and ensemble learning techniques.
result Superior uncertainty representation without compromising predictive performance.
DeepLR constructs confidence intervals for neural networks with asymmetric expansions.
problem Uncertainty estimation for neural network predictions.
method Likelihood-ratio-based approach for constructing asymmetric confidence intervals.
result DeepLR offers asymmetric intervals expanding in regions with limited data.
This work improves neural network calibration using explicit regularization.
problem Improving predictive uncertainty in neural networks.
method Introducing a probabilistic calibration measure and exploring explicit regularization techniques.
result Explicit regularization improves log-likelihood and predictive uncertainty.
We define a generalized likelihood function based on uncertainty measures and show that maximizing such a likelihood function for different measures induces different types of classifiers. In the probabilistic framework, we obtain classifiers that optimize the cross-entropy function. In the possibilistic framework, we …
The paper extends explainability methods to uncertainty-aware models, revealing feature impacts on predictive entropy and likelihood.
problem Understanding the factors contributing to uncertainty in probabilistic models.
method Adapting permutation feature importance, partial dependence plots, and individual conditional expectation plots to measure feature impacts on predictive entropy and likelihood.
result Novel insights into model behaviour and feature impacts on uncertainty are obtained.
Develops an empirical likelihood framework for random forests and ensembles.
problem Quantifying the statistical uncertainty of random forests and ensembles.
method Empirical likelihood framework exploiting the incomplete U-statistic structure of ensemble predictions. result Modified empirical likelihood statistic achieves accurate coverage and practical reliability.
We evaluate the uncertainty quality in neural networks using anomaly detection. We extract uncertainty measures (e.g. entropy) from the predictions of candidate models, use those measures as features for an anomaly detector, and gauge how well the detector differentiates known from unknown classes. We assign higher unc…
Neural networks estimate spatial process likelihoods efficiently.
problem Challenges in estimating spatial processes with slow or intractable likelihoods.
method Convolutional neural networks trained on a classification task to learn likelihood function.
result Neural likelihood surfaces provide fast and accurate parameter estimation.
Monotonic improvement in uncertainty estimation with Gaussian processes as dimension increases.
problem Uncertainty quantification in machine learning models, especially with Gaussian processes, is challenging and poorly understood.
method Analyzing the behavior of marginal likelihood and cross-validation metrics as input dimension increases, and exploring the effects of cold posteriors.
result The marginal likelihood improves monotonically with input dimension, while cross-validation metrics exhibit double descent behavior.
We propose a new scalable multi-class Gaussian process classification approach building on a novel modified softmax likelihood function. The new likelihood has two benefits: it leads to well-calibrated uncertainty estimates and allows for an efficient latent variable augmentation. The augmented model has the advantage …
In decision-making systems, it is important to have classifiers that have calibrated uncertainties, with an optimisation objective that can be used for automated model selection and training. Gaussian processes (GPs) provide uncertainty estimates and a marginal likelihood objective, but their weak inductive biases lead…
GenUQ uses generative models to estimate uncertainty in operator learning.
problem Uncertainty quantification in stochastic operator models.
method Introduces a measure-theoretic approach with a generative hyper-network.
result Outperforms other UQ methods in various example problems.
Bayesian method infers contextual bandit policies robustly.
problem Inference of contextual bandit policies in small sample sizes.
method Empirical likelihood for Bayesian inference.
result Accurate uncertainty measurements and policy comparison.
This work optimizes statistical inference with neural networks for high-energy physics data.
problem Optimal dimensionality reduction with minimal loss of information in the presence of systematic uncertainties.
method Neural network optimization based on binned Poisson likelihoods with nuisance parameters.
result Estimates of parameters of interest close to optimal.
Ribbon: Scalable Approximation and Robust Uncertainty Quantification
problem Reliably quantifying predictive uncertainty for complex models
method Ribbon, a scalable approximation to Dirichlet-reweighted bootstrap uncertainty
result Asymptotically equivalent to a flat-prior Laplace approximation under correct likelihood specification, recovers robust sandwich covariance under misspecification
This paper proposes a new method to approximate posterior distributions using generative neural networks trained via scoring rule minimization.
problem Bayesian Likelihood-Free Inference for models with intractable likelihood.
method Approximate posterior with generative neural networks trained via scoring rule minimization, avoiding the instability of adversarial training.
result Scoring Rule minimization leads to better performance and uncertainty quantification compared to adversarial training.
SMURF-THP improves Transformer Hawkes process models by providing uncertainty quantification.
problem Uncertainty quantification for Transformer Hawkes process predictions.
method Score matching for learning the score function of event arrival times.
result SMURF-THP outperforms likelihood-based methods in confidence calibration.
Revises logistic-softmax likelihood for Bayesian meta-learning in few-shot classification.
problem Inherent uncertainty in logistic-softmax leads to suboptimal performance in meta-learning.
method Redesigns logistic-softmax likelihood with a temperature parameter for better control of prior confidence.
result Achieves well-calibrated uncertainty estimates and comparable/superior performance on benchmark datasets.
A method for profiling systematic uncertainties in SBI using Factorizable Normalizing Flows.
problem Computational cost and limited applicability of current SBI methods for realistic analyses.
method Simulation-Based Inference with Factorizable Normalizing Flows to model systematic variations.
result Efficient profiling of nuisance parameters and multivariate DoI in complex analyses.
Bayesian autoencoders improve OOD detection by addressing Bernoulli likelihood issues.
problem Out-of-distribution (OOD) detection fails with Bernoulli likelihood for certain datasets.
method Proposes Bayesian autoencoders and alternative likelihood models to fix the issue.
result Bayesian autoencoders and alternative likelihood models improve OOD detection accuracy.
The paper proposes a method to construct confidence sets using likelihood ratios for sequential decision-making.
problem Constructing valid uncertainty estimates for unknown quantities in sequential decision-making.
method The method uses likelihood ratios to create any-time valid confidence sequences without specialized treatment for each application.
result The proposed confidence sets maintain the prescribed coverage in a model-agnostic manner and their size depends on the choice of estimator sequence.
CBDL uses credal sets to improve uncertainty quantification in deep learning.
problem Uncertainty in predictions and robustness to distribution shifts in deep learning.
method Train an infinite ensemble of Bayesian Neural Networks using credal sets.
result CBDL distinguishes between aleatoric and epistemic uncertainties and quantifies them better than single BNNs.
When using the K-nearest neighbors method, one often ignores uncertainty in the choice of K. To account for such uncertainty, Holmes and Adams (2002) proposed a Bayesian framework for K-nearest neighbors (KNN). Their Bayesian KNN (BKNN) approach uses a pseudo-likelihood function, and standard Markov chain Monte Carlo (…
Bayesian neural networks enhance uncertainty estimation in graph contrastive learning.
problem Uncertainty in graph contrastive learning when labeled data is scarce.
method Variational Bayesian neural networks for uncertainty estimation.
result Improved uncertainty estimation and downstream performance on semi-supervised node-classification tasks.
Optimizes sampling for faster convergence in Bayesian experimental design and uncertainty quantification.
problem Efficiently selecting samples for faster convergence in Bayesian experimental design and uncertainty quantification.
method Output-weighted acquisition functions leveraging likelihood ratio to guide sampling towards relevant regions.
result Superiority of the proposed method in uncertainty quantification and rare event identification.
VJE learns latent representations without contrastive learning, providing probabilistic semantics.
problem Learning latent representations without contrastive signals.
method VJE maximizes a symmetric conditional evidence lower bound (ELBO) on paired encoder embeddings, using a Student-t distribution on a polar representation.
result VJE outperforms standard non-contrastive baselines in ImageNet-1K, CIFAR-10/100, and STL-10.
Sparse Gaussian process hyperparameters optimized using MCMC.
problem Hyperparameter uncertainty leads to biased estimates and underestimation of predictive uncertainty.
method Proposes an MCMC algorithm to sample from the hyperparameter posterior in sparse Gaussian process regression.
result Significantly improves sampling efficiency in the Gaussian likelihood case.
Deep Learning (DL) methods have been transforming computer vision with innovative adaptations to other domains including climate change. For DL to pervade Science and Engineering (S&E) applications where risk management is a core component, well-characterized uncertainty estimates must accompany predictions. However, S…
Coherent uncertainty quantification is a key strength of Bayesian methods. But modern algorithms for approximate Bayesian posterior inference often sacrifice accurate posterior uncertainty estimation in the pursuit of scalability. This work shows that previous Bayesian coreset construction algorithms---which build a sm…
A new framework for clustering with uncertainty quantification.
problem Lack of uncertainty quantification in clustering methods.
method Generalized Bayes framework using Gibbs posteriors and loss functions.
result Efficient algorithms for clustering and uncertainty quantification.
New method corrects biased predictions and uncertainty estimates in classification with nuisance parameters.
problem Tackles biased predictions and invalid uncertainty estimates in classification with nuisance parameters.
method Proposes a method that estimates ROC across the entire nuisance parameter space to devise invariant cutoffs.
result Demonstrates effective domain adaptation and valid prediction sets with high power.
A new framework for PPLS combines noise estimation, optimization, and calibration.
problem Probabilistic PLS models need interpretable latent factors and calibrated uncertainty.
method End-to-end pipeline combining noise estimation, constrained optimization, and prediction calibration.
result Achieves near-nominal coverage and native calibrated uncertainty across benchmarks.
We suggest a general approach to quantification of different forms of aleatoric uncertainty in regression tasks performed by artificial neural networks. It is based on the simultaneous training of two neural networks with a joint loss function and a specific hyperparameter λ>0 that allows for automatically detecting …
Paper quantifies uncertainty in pairwise comparison models.
problem Uncertainty quantification in sparse Bradley-Terry-Luce models.
method Unified proof strategy for MLE and spectral estimator.
result Sharp and uniform non-asymptotic expansions for estimators.
This paper benchmarks Bayesian models' ability to estimate predictive correlations, especially for active learning.
problem Benchmarking how accurately Bayesian models estimate predictive correlations, especially in active learning.
method Considered transductive active learning as a benchmark, introduced meta-correlations and cross-normalized likelihoods.
result Meta-correlations and cross-normalized likelihoods can efficiently evaluate predictive correlations and are consistent with TAL performance.
Research shows deep generative models' likelihoods are unreliable for anomaly detection.
problem Anomaly detection using deep generative models' likelihoods is unreliable.
method Examined the behavior of distribution densities through reparametrization.
result The likelihoods used for anomaly detection rely on strong and implicit hypotheses.
Review of diffusion models for SBI in non-ideal data scenarios.
problem Inference of parameters from complex simulation outputs with intractable likelihoods.
method Diffusion models for likelihood-free inference, addressing model misspecification, unstructured observations, and missing data.
result Improved robustness and efficiency in SBI methods for non-ideal data scenarios.
Reconstructing the position of an interaction for any dual-phase time projection chamber (TPC) with the best precision is key to directly detecting Dark Matter. Using the likelihood-free framework, a new algorithm to reconstruct the 2-D (x; y) position and the size of the charge signal (e) of an interaction is presente…
Embeddings in machine learning are low-dimensional representations of complex input patterns, with the property that simple geometric operations like Euclidean distances and dot products can be used for classification and comparison tasks. The proposed meta-embeddings are special embeddings that live in more general in…
FedGVI improves FL robustness to model misspecification.
problem Limited robustness in FL approaches to model misspecification.
method Probabilistic Federated Learning framework that generalizes previous methods.
result FedGVI provides robust and calibrated predictions under model misspecification.
Proposes a neural network loss function for better uncertainty estimation.
problem Challenges in estimating predictive uncertainty of neural networks.
method Bayesian Validation Metric (BVM) framework with ensemble learning.
result Competitive and robust uncertainty estimation on in-distribution and out-of-distribution data.
A new method improves uncertainty quantification in Bayesian inference.
problem Poor uncertainty quantification in traditional Gibbs posteriors.
method Sequential Gibbs posteriors with a Bernstein-von Mises theorem.
result Sequential Gibbs posteriors provide better frequentist coverage.
New method selects equivariant models using uncertainty metrics.
problem Selecting equivariant models among pretrained ones with varying symmetry biases.
method Uncertainty-aware model selection using frequentist, Bayesian, and calibration-based measures.
result Bayesian model evidence often misaligns with predictive performance.
Variational auto-encoders (VAEs) are a popular and powerful deep generative model. Previous works on VAEs have assumed a factorized likelihood model, whereby the output uncertainty of each pixel is assumed to be independent. This approximation is clearly limited as demonstrated by observing a residual image from a VAE …