New ML methods improve physical system understanding by quantifying uncertainty across diverse regimes.
problem Capturing multi-regime physical systems with standard ML techniques.
method Coverage-oriented uncertainty quantification (UQ) methods.
result Coverage-oriented UQ models deliver physically consistent uncertainty estimates.
Bayesian UQ matches frequentist UQ for adaptively collected data.
problem Uncertainty quantification for adaptive data collection.
method Extends Bernstein-von Mises theorem to adaptively collected data.
result Bayesian UQ asymptotically matches Wald-type frequentist UQ.
CoT-UQ improves LLM uncertainty quantification by integrating reasoning steps.
problem LLMs' overconfidence and lack of response-wise uncertainty quantification.
method Integrates LLMs' reasoning steps into uncertainty estimation.
result Significantly improves uncertainty quantification accuracy (5.9% AUROC improvement).
PCS-UQ framework improves uncertainty quantification for machine learning models.
problem Ensuring trustworthy uncertainty quantification for machine learning models in high-stakes domains.
method PCS-UQ framework based on Predictability, Computability, and Stability principles, integrating prediction-checking, bootstrap samples, and multiplicative calibration.
result PCS-UQ maintains target coverage while outperforming or matching conformal methods in interval width and subgroup coverage.
This study revisits UQ validation methods based on consistency and adaptivity concepts.
problem Lack of comprehensive validation methods for UQ metrics across input feature ranges.
method Revisit and extend common validation methods for UQ metrics based on consistency and adaptivity concepts.
result Improved understanding and capabilities of UQ metrics validation methods.
This paper surveys UQ methods for deep learning.
problem Overconfident predictions in deep learning models.
method Categorizes UQ methods by uncertainty sources.
result Identifies strengths and limitations of each category.
A new method combines SciML and UQ with physical constraints.
problem Uncertainty quantification in scientific machine learning tasks.
method Physics-constrained polynomial chaos expansion.
result Effective uncertainty quantification and SciML integration.
Last-layer approximation improves UQ performance without sacrificing computational efficiency.
problem Epistemic uncertainty quantification for deep neural networks.
method Comparison of full-network and last-layer linearization using theoretical and empirical approaches.
result Last-layer approximation yields comparable UQ performance with improved computational efficiency.
A decision-theoretic bootstrapping method for robust uncertainty quantification.
problem Uncertainty in finite data sets and distributional shift between training and testing data.
method Partition data, train models, sample UQ subsets, define adversarial game, identify optimal mixed strategies.
result Optimal model mixtures and UQ estimates for robust uncertainty quantification.
Global sensitivity analysis improves BNN hyperparameter selection for accurate uncertainty quantification.
problem Difficulties in obtaining accurate uncertainty quantification with Bayesian Neural Networks (BNNs).
method Global sensitivity analysis of BNN performance under varying hyperparameter settings.
result Many hyperparameters interact to affect both predictive accuracy and uncertainty quantification.
Beam search improves UQ in LLMs by reducing duplicates and variance.
problem Peaked distributions in multinomial sampling lead to duplicates and high variance in uncertainty estimates.
method Employ beam search to generate candidates for consistency-based UQ, providing a theoretical lower bound and empirical evaluation.
result Beam search achieves smaller error than multinomial sampling, leading to state-of-the-art UQ performance.
Study evaluates machine learning methods for uncertainty quantification in complex systems.
problem Accurately quantify epistemic and aleatoric uncertainties in complex dynamical systems.
method Examined Gaussian processes, UQ-augmented neural networks (ENN, BNN, D-NN, G-NN) on two model data sets.
result Concluded on model architecture and hyperparameter tuning for improved UQ accuracy.
DVE uses GPs on DNN outputs to provide UQ without retraining.
problem Feature collapse in DNNs affects UQ methods.
method Deep Vecchia ensemble (DVE) of GPs on DNN hidden layers.
result Deterministic UQ possible in feature-collapsed DNNs.
This research improves model interpretability and uncertainty estimation for deep learning models on non-iid data.
problem Improving interpretability and uncertainty estimation for deep learning models on non-iid data.
method 4 UQ approaches (BNN, SWAG, MC dropout, ensemble) applied to ARMED MEDL models.
result Ensemble approaches, especially with 90% subsampling, provide best performance in prediction and uncertainty estimation.
Improves inverse uncertainty quantification for time-dependent data using PCA and deep neural networks.
problem Efficiently quantify model input uncertainties from time-dependent experimental data.
method Functional PCA for dimensionality reduction, deep neural networks for surrogate modeling, Bayesian neural networks for uncertainty estimation.
result The proposed method reduces the computational cost and improves the agreement with experimental data.
Flexible evidential deep learning improves uncertainty quantification in machine learning.
problem Overconfident predictions in machine learning models can lead to serious consequences.
method Proposes flexible evidential deep learning (F-EDL) to model uncertainty over class probabilities using a flexible Dirichlet distribution.
result Empirically demonstrates state-of-the-art uncertainty quantification performance across diverse scenarios.
MC-CP combines adaptive MC dropout with conformal prediction for robust uncertainty quantification.
problem Deploying deep learning models in safety-critical applications requires reliable confidence estimates.
method MC-CP integrates adaptive Monte Carlo dropout with conformal prediction to improve model performance.
result MC-CP significantly outperforms state-of-the-art UQ methods in both classification and regression tasks.
UQE uses LLMs to analyze unstructured data efficiently.
problem Efficient analytics on unstructured data.
method Proposes UQE, a query engine that uses LLMs to interpret UQL queries.
result Demonstrates efficient analytics on various unstructured data types.
The paper tackles uncertainty quantification for classification under label shift without assuming i.i.d. data.
problem Uncertainty quantification for classification under label shift in non-i.i.d. settings.
method The paper uses conformal prediction and post-hoc binning for distribution-free UQ, and reweights these methods for label shift.
result The reweighted methods improve UQ performance under label shift, preserving coverage and calibration.
Study evaluates uncertainty quantification methods for molecular property prediction.
problem Uncertainty in neural models for molecular property prediction.
method Systematically evaluated several UQ methods on five benchmark datasets.
result No single method is unequivocally superior, and none provides reliable error ranking across datasets.
Study evaluates quality of uncertainty estimates for neural networks.
problem Lack of principled assessment methods for evaluating uncertainty quality in deep learning.
method Statistical methods of frequentist interval coverage, interval width, and expected calibration error.
result Different UQ methods produce markedly different quality uncertainty estimates.
Surrogate models help predict complex systems with less computational cost.
problem Uncertainty in complex systems due to variability and external loads.
method Surrogate models trained on limited simulations to approximate full time-dependent response.
result Efficient surrogate models reduce computational expense for UQ in nonlinear dynamics.
Δ-UQ uses anchoring to estimate uncertainty in models.
problem Estimating uncertainty in predictive models.
method Anchoring input into a tuple for uncertainty estimation.
result Δ-UQ outperforms baselines in various use-cases.
Study uncovers uncertainty in traffic prediction models across cities.
problem Lack of interpretability in deep learning models for traffic prediction.
method Investigated uncertainty quantification methods for image-based traffic prediction.
result Meaningful uncertainty estimates can be recovered for traffic prediction.
Comparison of UQ methods in deep learning for a simple physical system.
problem Uncertainty quantification in deep learning for physical systems.
method Bayesian Neural Networks (BNN), Concrete Dropout (CD), Deep Ensembles (DE), and Analytic Error Propagation.
result Pitfalls in using UQ methods, especially Bayesian Neural Networks and Concrete Dropout.
A new method ranks uncertainty vectors from multiple measures for robust prediction.
problem Single scalar measures of model reliability are insufficient for comprehensive uncertainty quantification.
method Optimal transport ranks vectors of uncertainty measures, supporting flexible fusion of aleatoric and epistemic uncertainties.
result The method provides a robust ranking of uncertainty that supports various downstream tasks.
OOD-trained Bayesian neural networks perform similarly to frequentist methods in uncertainty quantification.
problem Bayesian neural networks struggle in out-of-distribution (OOD) detection tasks.
method Incorporated out-of-distribution data into Bayesian inference through four different methods.
result OOD-trained Bayesian neural networks are competitive with frequentist baselines.
Decision-alignment evaluates uncertainty quantification for decision-relevant UQ
problem Evaluation of uncertainty quantification metrics
method Introduce decision-alignment
result Proper scoring rules align with decision utility
Uncertainty Toolbox aids in assessing and improving uncertainty quantification in machine learning.
problem Disparate evaluation metrics and implementations hinder direct comparison of uncertainty quantification results.
method Provides an open-source Python library for assessing, visualizing, and improving uncertainty quantification.
result Facilitates more accurate and comparable uncertainty quantification across different works.
This paper emphasizes the need for uncertainty quantification in data-driven ML models for nuclear engineering.
problem Uncertainty in ML predictions due to data noise, model architecture, and stochastic training.
method Explains and compares uncertainties in physics-based and data-driven models, and presents techniques to quantify ML prediction uncertainties.
result The importance of uncertainty quantification in ML models for nuclear engineering applications.
New algorithm speeds up Bayesian UQ for high-dimensional inverse problems.
problem Computational inefficiency in Bayesian inference for high-dimensional inverse problems.
method Deep neural network-based autoencoder for dimension reduction and emulation phase.
result Computational efficiency up to three orders of magnitude with scalable Bayesian UQ.
New method for neural network uncertainty quantification using empirical Neural Tangent Kernel.
problem Accurately quantify uncertainty in neural network predictions.
method Post-hoc, sampling-based approach using gradient-descent on linearized networks.
result Method effectively approximates Gaussian process posterior and outperforms existing methods in efficiency and accuracy.
Novel framework improves GNN uncertainty estimates under distribution shifts.
problem Improving reliability of GNN uncertainty estimates under distribution shifts.
method Adapting stochastic data centering to graph data through novel graph anchoring strategies.
result G-ΔUQ leads to better calibrated GNNs for node and graph classification. This paper proposes an active learning-based Gaussian process (AL-GP) metamodelling method to estimate the cumulative as well as complementary cumulative distribution function (CDF/CCDF) for forward uncertainty quantification (UQ) problems. Within the field of UQ, previous studies focused on developing AL-GP approaches…
Novel approach uses ENN for UQ in gust predictions, reducing RMSE and improving confidence.
problem Reducing bias and uncertainty in wind gust predictions.
method Evidential Neural Network (ENN) with Explainable AI.
result 47% reduction in RMSE, 95% coverage of observed gusts at 179 out of 266 stations.
A new method decomposes subjective risk into epistemic and aleatoric uncertainties.
problem Uncertainty quantification in modeling decisions.
method Subjective risk decomposition using strictly proper loss.
result Recovery of classic uncertainty measures and new learning-theoretic connections.
As neural networks have begun performing increasingly critical tasks for society, ranging from driving cars to identifying candidates for drug development, the value of their ability to perform uncertainty quantification (UQ) in their predictions has risen commensurately. Permanent dropout, a popular method for neural …
BODE enhances deep neural network predictions and uncertainty quantification in safety modeling.
problem Uncertainty in deep neural network predictions for safety-critical applications.
method Bayesian optimization combined with deep ensembles (BODE).
result BODE reduces total uncertainty by over 30% compared to a manually tuned baseline ensemble.
Study validates ML-UQ calibration statistics using simulated reference values.
problem Validation of ML-UQ calibration statistics is lacking due to lack of predefined reference values.
method Proposed validation workflow using simulated reference values derived from synthetic datasets.
result Some statistics, like CC and ENCE, are overly sensitive to generative distribution choice.
Tabular FMs struggle with reliable uncertainty quantification.
problem Uncertainty quantification in tabular foundation models.
method Compared TabPFN and Gaussian processes (GPs) across various regression tasks.
result GP outperforms TabPFN in data-scarce settings and when kernels are good priors.
Researchers use active subspaces to quantify uncertainty in deep generative models for molecular design.
problem Uncertainty quantification in deep generative models for molecular design due to high parameter space.
method Leveraging active subspaces to approximate posterior distribution over low-dimensional parameters.
result The proposed UQ scheme effectively estimates epistemic uncertainty in high-dimensional parameter space without altering model architecture.
New framework models uncertainty in classification debates.
problem Weak interpretability of existing uncertainty quantification methods.
method Courtroom analogy and Mixture of Dirichlet Experts (MoDEX) model.
result MoDEX achieves state-of-the-art uncertainty quantification performance.
AI helps HEP measure uncertainties, but needs better interpretation.
problem Uncertainty interpretation in AI for HEP measurements.
method Discussing existing AI methods for inference, simulation, and control/decision-making.
result Need for trustworthy AI UQ methods for widespread usage.
Single model estimates uncertainty via biased data shifts.
problem Estimating uncertainties in deep neural networks.
method Trivial input transformation to approximate ensemble behavior.
result Single model uncertainty estimates are superior to current methods.
This paper studies uncertainty quantification in deep spatiotemporal forecasting.
problem Uncertainty quantification in deep spatiotemporal forecasting models.
method Analysis of UQ methods from Bayesian and frequentist perspectives, including statistical decision theory.
result Different UQ methods have different strengths and weaknesses, with Bayesian methods being more robust in mean prediction and frequentist methods providing more extensive coverage.
Proposes PSCs for UQ in deep nets without retraining.
problem Estimating uncertainty in deep nets with a single pass.
method Identifies sensitive, smooth intermediate layer, fits probabilistic model.
result PSCs achieve UQ and OOD detection performance matching existing methods.
Wireless traffic prediction is a fundamental enabler to proactive network optimisation in beyond 5G. Forecasting extreme demand spikes and troughs due to traffic mobility is essential to avoiding outages and improving energy efficiency. Current state-of-the-art deep learning forecasting methods predominantly focus on o…
CMCO provides robust uncertainty estimates for neural operators without retraining.
problem Uncertainty quantification in deep learning for real-time virtual sensing.
method Unified Monte Carlo dropout and split conformal prediction in DeepONet.
result Near-nominal empirical coverage in diverse applications.