Bayesian uncertainty quantification is flawed, according to new research.
problem Flawed interpretation of Bayesian uncertainty quantification.
method Discussion of Bayesian updating and optimization-based perspective, proposing measures of quality.
result Bayesian uncertainty quantification is not coherent with optimization-based perspective.
In chemistry, deep neural network models have been increasingly utilized in a variety of applications such as molecular property predictions, novel molecule designs, and planning chemical reactions. Despite the rapid increase in the use of state-of-the-art models and algorithms, deep neural network models often produce…
Normalizing flows improve ptychography reconstruction quality and uncertainty quantification.
problem Challenges in ptychography due to large-scale nonlinear and non-convex inverse problems and photon statistics.
method Use of normalizing flows to model the posterior distribution and quantify reconstruction uncertainty.
result Normalizing flows enable better characterization and uncertainty quantification in ptychography reconstructions.
Measures DNA quality degradation effects.
problem Identifying degraded DNA sequence data.
method Novel quality quantification based on intentional degradation effects.
result Quantified measures of degradation can be used for multiple purposes.
The paper highlights the importance of model misspecification in uncertainty estimation.
problem The reliability of uncertainty estimates in machine learning models under model misspecification.
method Thought experiments and literature review.
result Model misspecification should be given more attention in uncertainty estimation.
Bayesian Neural Networks (BNNs) place priors over the parameters in a neural network. Inference in BNNs, however, is difficult; all inference methods for BNNs are approximate. In this work, we empirically compare the quality of predictive uncertainty estimates for 10 common inference methods on both regression and clas…
Study evaluates quality of uncertainty estimates for neural networks.
problem Lack of principled assessment methods for evaluating uncertainty quality in deep learning.
method Statistical methods of frequentist interval coverage, interval width, and expected calibration error.
result Different UQ methods produce markedly different quality uncertainty estimates.
This work tackles uncertainty quantification in tomography reconstruction.
problem Ill-posed nature of tomographic reconstruction leading to no unique solution.
method Gaussian process modeling to incorporate prior knowledge and experimental noises.
result Efficient uncertainty quantification in tomographic reconstruction.
Uncertainty of labels in clinical data resulting from intra-observer variability can have direct impact on the reliability of assessments made by deep neural networks. In this paper, we propose a method for modelling such uncertainty in the context of 2D echocardiography (echo), which is a routine procedure for detecti…
Bayesian autoencoders quantify anomaly uncertainty for safer machine learning.
problem Lack of uncertainty quantification in autoencoders for anomaly detection.
method Formulated Bayesian autoencoders to quantify epistemic and aleatoric anomalies.
result Demonstrated effectiveness of BAEs on benchmark and real datasets.
BayesIMP combines multiple causal graphs to estimate average treatment effects with uncertainty.
problem Uncertainty quantification in causal inference from multiple datasets.
method Bayesian Interventional Mean Processes (BayesIMP) integrating probabilistic integration and kernel mean embeddings.
result Improvements in average treatment effect estimation over state-of-the-art methods.
Improving software quality through effective organizational learning.
problem Lack of reliable quantification methods for software evolution.
method Leveraging application lifecycle management data to identify and address managerial practices.
result Effective learning from past processes improves software quality indirectly.
Generative Score Inference improves uncertainty quantification for multimodal data.
problem Accurate uncertainty quantification in multimodal learning tasks.
method Generative Score Inference (GSI) uses synthetic samples to approximate conditional score distributions.
result GSI achieves state-of-the-art performance in hallucination detection and image captioning uncertainty estimation.
Optimizes a small set of centroid points to approximate bootstrap distribution.
problem Computational inefficiency of standard bootstrap methods in large-scale machine learning.
method Explicitly optimizes a small set of high quality centroid points to approximate the ideal bootstrap distribution.
result Accurately estimates uncertainty with a small number of bootstrap centroids, outperforming i.i.d. sampling.
This paper improves uncertainty quantification in ELM models.
problem Uncertainty in ELM predictions due to data assumptions and randomness.
method Analytical derivations and variance estimates under various conditions.
result Improved understanding and estimation of ELM variability.
Novel unsupervised scheme for highly imbalanced and overlapping datasets.
problem Highly imbalanced and overlapping classes in medical datasets.
method Unsupervised domain adaptation scheme based on Quantification.
result High quality results for Quantification and Domain Adaptation.
This paper introduces VI for physics-informed deep learning, enhancing uncertainty quantification.
problem Uncertainty quantification in physics-informed deep learning.
method Variational inference for generative and inverse problems.
result VI provides a flexible and scalable approach for physics-based inference.
This paper considers the generation of prediction intervals (PIs) by neural networks for quantifying uncertainty in regression tasks. It is axiomatic that high-quality PIs should be as narrow as possible, whilst capturing a specified portion of data. We derive a loss function directly from this axiom that requires no d…
This work uses conformal prediction to quantify uncertainty in large language models for multiple-choice questions.
problem Ensuring robustness and reliability of large language models in high-stakes applications.
method Conformal prediction applied to multi-choice question answering.
result Uncertainty estimates from conformal prediction are closely related to prediction accuracy.
RegVar quantifies uncertainty in deep learning networks by measuring sensitivity to regularization.
problem Uncertainty quantification in deep learning networks, especially for large networks.
method RegVar method based on variation due to regularization, implemented during fine-tuning phase.
result RegVar provides rigorous uncertainty estimates that recover Bayesian deep learning approximations.
This paper presents a method to automatically generate high-quality prediction intervals for neural networks.
problem Accurate uncertainty quantification for deep learning models in real-world applications.
method Dual neural network approach with a novel loss function to balance prediction interval width and coverage.
result Our method produces significantly narrower prediction intervals with higher probability coverage compared to state-of-the-art methods.
Proposes PSCs for UQ in deep nets without retraining.
problem Estimating uncertainty in deep nets with a single pass.
method Identifies sensitive, smooth intermediate layer, fits probabilistic model.
result PSCs achieve UQ and OOD detection performance matching existing methods.
Single neural networks can match deep ensembles' benefits without the complexity.
problem The effectiveness and necessity of deep ensembles in neural network models.
method Demonstrated limitations of ensemble diversity and OOD performance in deep ensembles compared to a single larger model.
result A single neural network can replicate deep ensembles' benefits in uncertainty quantification and robustness.
Develops a deterministic method to approximate NSDEs for better uncertainty quantification.
problem Computational infeasibility of obtaining well-calibrated uncertainty from NSDEs.
method Bidimensional moment matching algorithm for approximating NSDE transition kernel.
result Deterministic approximation improves uncertainty calibration and prediction accuracy.
This study evaluates uncertainty quantification methods for deep learning in predictive maintenance.
problem Uncertainty quantification for reliable decision-making in predictive maintenance.
method State-of-the-art variational inference algorithms for Bayesian neural networks (BNN), Monte Carlo Dropout (MCD), deep ensembles (DE), and heteroscedastic neural networks (HNN) were tested.
result No method clearly outperforms others in all situations, but DE and MCD provide more conservative uncertainty estimates.
Enhances uncertainty estimation in neural networks using Dirichlet-based MC Dropout.
problem Deterministic predictions without uncertainty estimates in neural networks.
method Integrates Dirichlet-based framework within Monte Carlo Dropout.
result Improves quality of uncertainty estimates in deep learning models.
ELUQuant quantifies uncertainties in DIS events using BNNs and MNFs.
problem Uncertainty quantification in Deep Inelastic Scattering (DIS) events.
method Physics-informed Bayesian Neural Network with flow approximated posteriors.
result Effective extraction of kinematic variables x, Q2, and y with detailed event-level uncertainty. CP4SBI improves the calibration of credible sets in SBI models.
problem Inaccurate credible sets in SBI models lead to underestimation of true parameters.
method Develops a local conformal calibration framework for SBI models.
result Improves the quality of uncertainty quantification for neural posterior estimators.
Fast algorithm solves BVPs in linear time with probabilistic uncertainty.
problem Solving boundary value problems efficiently and accurately.
method Gauss--Markov prior tailored to BVPs, linear-time computation.
result Probabilistic solution with linear time complexity and comparable quality.
This work investigates uncertainty quantification for black-box large language models in natural language generation.
problem Lack of trustworthiness in responses generated by black-box large language models.
method Differentiated uncertainty vs confidence, proposed and compared several confidence/uncertainty measures, applied to selective NLG.
result A simple measure for semantic dispersion can predict the quality of LLM responses.
DCK improves air quality index prediction with probabilistic spatial models.
problem Non-Gaussian, complex spatial structure of air quality index.
method Deep classifier kriging (DCK) for non-Gaussian, nonlinear spatial prediction.
result DCK outperforms conventional methods in predictive accuracy and uncertainty quantification.
ESRL uses uncertainty quantification to learn safe, optimal policies in offline RL.
problem Challenges in interpreting and measuring uncertainty of learned policies in offline RL.
method Expert-Supervised Reinforcement Learning (ESRL) framework that uses hypothesis testing and posterior distributions.
result The framework can learn safe and optimal policies with theoretical guarantees and independent sample efficiency.
A method for estimating signal distributions from inverse problems using normalizing flows.
problem Estimating the distribution of the underlying signal from observations in inverse problems.
method A framework for approximate inference on a pre-trained unconditional flow model, using a composition of two flow models for stable variational inference.
result Our method produces high-quality samples with uncertainty quantification and can be amortized for zero-shot inference.
Enhances clinical trial predictions by quantifying uncertainty.
problem Uncertainty in medical diagnosis and drug discovery predictions.
method Selective classification integrated with Hierarchical Interaction Network (HINT).
result Significant improvement in PR-AUC, F1, ROC-AUC, and overall accuracy.
The paper uses statistics to improve the explainability of models.
problem Subjective human assessment of explanations and lack of theoretical guarantees.
method Leveraging statistical estimators for proper definition and evaluation of explanations.
result Statistical tools provide theoretical guarantees and evaluation metrics for explanations.
This paper studies uncertainty quantification in deep spatiotemporal forecasting.
problem Uncertainty quantification in deep spatiotemporal forecasting models.
method Analysis of UQ methods from Bayesian and frequentist perspectives, including statistical decision theory.
result Different UQ methods have different strengths and weaknesses, with Bayesian methods being more robust in mean prediction and frequentist methods providing more extensive coverage.
New method certifies risks of LLM outputs, improving accuracy and reliability.
problem Uncertain and incorrect outputs from large language models.
method Information-lift certificates using PAC-Bayes bounds and skeleton design.
result Achieves 77.0% coverage at 2% risk, outperforming baselines.
SGMs are robust to practical errors via uncertainty quantification.
problem Robustness of SGMs to practical implementation errors.
method Wasserstein uncertainty propagation (WUP) theorem and Bernstein estimates.
result SGMs are provably robust to multiple sources of error.
This paper explores Bayesian Neural Network posteriors, uncovering symmetries and their impact.
problem Understanding the complex posterior distribution of deep Bayesian Neural Networks.
method Investigates optimal approaches for approximating posteriors, analyzes modes, and explores visualizations.
result Uncovered weight-space symmetries and their impact on the posterior, particularly scaling symmetries.
A new method uses conformal prediction to create reliable confidence masks for image super-resolution.
problem Uncertainty quantification in image super-resolution using generative models.
method Conformal prediction techniques applied to a confidence mask for reliable uncertainty communication.
result Strong theoretical guarantees and empirical solid performance in image super-resolution.
Develops optimal uncertainty quantification for risk-averse decision makers.
problem Quantifying prediction uncertainty for risk-sensitive domains.
method Decision-theoretic foundations connecting uncertainty quantification with risk-averse decision-making.
result Risk-Averse Calibration (RAC) algorithm provides optimal prediction sets for risk-averse decision makers.
Efficient neural network ensembles improve image classification reliability and uncertainty quantification.
problem Uncertainty in neural network predictions for industrial image classification.
method Investigated efficient neural network ensembles (snapshot, batch, multi-input multi-output) for image classification reliability and uncertainty quantification.
result Batch ensemble is a cost-effective and competitive alternative to deep ensembles, offering savings in training and test time.
DVGS identifies low-quality data quickly and accurately.
problem Identifying and filtering mislabeled or noisy data in machine learning.
method Data Valuation with Gradient Similarity (DVGS) algorithm.
result DVGS outperforms baseline methods in identifying low-value data across various domains.
This paper improves SVM prediction uncertainty quantification methods.
problem Lack of comprehensive UQ methods for SVM predictions.
method Developed SSVQR model for sparse PI estimation and feature selection algorithm.
result Proposed SSVQR model achieves sparse solutions and improves PI quality.
Hybrid AI and rule-based framework de-identifies medical imaging data.
problem De-identifying medical imaging data to protect PHI and PII.
method Combines rule-based and AI techniques with uncertainty quantification.
result Robust performance across benchmark datasets and regulatory standards.
Study evaluates uncertainty quantification for atomistic neural networks, revealing complex relationships between error and uncertainty.
problem Uncertainty quantification for predictions of atomistic neural networks.
method Modified PhysNet NN architecture, evaluated with various metrics, analyzed QM9 and tautomerization reaction databases.
result Error and uncertainty are not linearly related; redundancy and noise complicate predictions, especially for small changes.
Estimates density ratio for two-sample comparison using tree models.
problem Comparing two distributions given i.i.d. observations.
method Additive tree models with balancing loss for density ratio estimation.
result Bayesian inference provides uncertainty quantification for density ratio.
Efficiently samples conformal boundaries in high dimensions using flows.
problem Difficulty in interpreting and using prediction sets in high-dimensional or structured output spaces.
method Flow-based approach using differentiable nonconformity scores to induce deterministic flows on the output space.
result Sampling conformal boundaries in arbitrary dimensions becomes computationally efficient and training-free.