This paper tackles uncertainty in deep learning for construction of prediction intervals.
problem Deep learning models lack the ability to provide reliable prediction intervals for high-risk tasks.
method The authors design a special loss function to learn both aleatory and epistemic uncertainties without requiring uncertainty labels.
result The method constructs prediction intervals that are competitive with state-of-the-art methods on publicly available datasets.
This work introduces a method to decompose uncertainty in in-context learning for large language models.
problem Understanding the sources of uncertainty in in-context learning for large language models.
method Variational uncertainty decomposition framework without sampling from latent parameter posterior.
result Quantitative and qualitative validation of decomposed epistemic and aleatoric uncertainties.
This paper examines sources of uncertainty in machine learning from a statistical perspective.
problem Quantifying uncertainty in supervised machine learning models.
method A conceptual, basic science approach examining aleatoric and epistemic uncertainty.
result Sources of uncertainty are diverse and cannot always be decomposed into aleatoric and epistemic.
Survey on uncertainty in ML and DL, covering sources, quantification, and decision-making.
problem Understanding and quantifying uncertainty in ML and DL for risk-sensitive applications.
method Structured review of literature, categorizing uncertainty, assessing uncertainty quantification techniques.
result Broadened scope of uncertainty discussion and updated DL uncertainty quantification methods.
Enhances reinforcement learning uncertainty estimation with a generalized Gaussian error model.
problem Inaccurate error representations and compromised uncertainty estimation in conventional uncertainty-aware TD learning.
method Introduces a novel framework for generalized Gaussian error modeling in deep reinforcement learning, incorporating higher-order moments, particularly kurtosis, to improve uncertainty estimation and mitigation.
result Significant performance gains in policy gradient algorithms with the proposed framework.
Uncertainty modeling for dynamical systems
problem Uncertainty modeling for dynamical systems
method Discussing sources of uncertainty, their nature, and task-specific objectives
result Identifying the types of uncertainty needed for dynamical systems
Reinforcement learning agents are faced with two types of uncertainty. Epistemic uncertainty stems from limited data and is useful for exploration, whereas aleatoric uncertainty arises from stochastic environments and must be accounted for in risk-sensitive applications. We highlight the challenges involved in simultan…
Proposes linking energy and force uncertainty in deep learning potentials.
problem Uncertainty in predicted energies and forces in machine learning models.
method Introduces a spatially correlated noise process to link energy and force uncertainty.
result Demonstrates the approach on molecular datasets, linking energy and force uncertainties.
Teaches uncertainty in ML through practical examples.
problem Lack of uncertainty teaching in ML curricula.
method Developed a curriculum and use cases.
result Motivates adoption of uncertainty concepts in AI courses.
Uncertainty Toolbox aids in assessing and improving uncertainty quantification in machine learning.
problem Disparate evaluation metrics and implementations hinder direct comparison of uncertainty quantification results.
method Provides an open-source Python library for assessing, visualizing, and improving uncertainty quantification.
result Facilitates more accurate and comparable uncertainty quantification across different works.
Proposes a new criterion for reliable uncertainty estimation in deep neural networks.
problem Inability of existing approaches to provide reliable uncertainty estimates for deep neural networks.
method Develops a density uncertainty layer architecture that satisfies the proposed criterion.
result Density uncertainty layers provide more reliable uncertainty estimates and robust out-of-distribution detection.
Fortuna simplifies uncertainty quantification in deep learning.
problem Improving uncertainty estimates in deep learning models.
method Supports various calibration techniques including conformal prediction and scalable Bayesian inference.
result Simplifies benchmarking and builds robust AI systems.
Proposes a method to quantify and explain deep learning model uncertainties.
problem Deep learning model predictions are sensitive to perturbations and adversarial attacks.
method Gradient-based uncertainty attribution method to identify problematic regions and propose mitigation strategies.
result Proposed UA-Backprop method achieves competitive accuracy and efficiency compared to existing methods.
Bayesian neural networks enhance uncertainty estimation in graph contrastive learning.
problem Uncertainty in graph contrastive learning when labeled data is scarce.
method Variational Bayesian neural networks for uncertainty estimation.
result Improved uncertainty estimation and downstream performance on semi-supervised node-classification tasks.
Model separates overall uncertainty into aleatoric and epistemic components for active learning.
problem Active learning with uncertainty quantification.
method Non-stationary Heteroscedastic Gaussian process model.
result Model separates overall uncertainty into aleatoric and epistemic components.
URL benchmark evaluates uncertainty quantification in pretrained models.
problem Need for reliable uncertainty estimates in transferable pretrained models.
method Proposes URL benchmark to measure transferability of representations and uncertainty estimates.
result Transferable uncertainty quantification remains challenging but not contradictory to traditional goals.
DER uses neural nets to better handle uncertainty in machine learning.
problem Need for principled uncertainty reasoning in safety-critical domains.
method Uncertainty-aware regression-based neural networks (NNs) with evidential distributions.
result DER shows promise over traditional methods but is a heuristic.
New method quantifies uncertainty in reinforcement learning models.
problem Quantifying uncertainty over expected cumulative rewards in reinforcement learning.
method Proposes a new uncertainty Bellman equation to more accurately estimate value function variance.
result Our method converges to the true posterior variance over values and improves sample-efficiency.
Bayesian parametric matrix models provide uncertainty quantification for spectral learning.
problem Uncertainty quantification in spectral learning for safety-critical applications.
method Bayesian parametric matrix models (B-PMMs) that extend PMMs to provide uncertainty estimates.
result B-PMMs achieve exceptional uncertainty calibration (ECE < 0.05) while maintaining favorable scaling.
New method improves uncertainty estimation in Bayesian deep learning models.
problem Underestimation of predictive uncertainty in Neural Linear Models (NLMs).
method Proposes a novel training method to capture useful predictive uncertainties and incorporate domain knowledge.
result Traditional training procedures for NLMs can drastically underestimate uncertainty in data-scarce regions.
Improved asset pricing using uncertainty-adjusted sorting in machine learning models.
problem Ignoring asset-specific estimation uncertainty in portfolio construction.
method Uncertainty-adjusted prediction bounds for sorting assets.
result Improves portfolio performance across various ML models and equity panels.
The paper argues that uncertainty quantification in ML is application-specific and proposes a flexible family of measures.
problem The need for proper uncertainty quantification in machine learning for safety-critical applications.
method A flexible family of uncertainty measures tailored to specific applications, using proper scoring rules to control characteristics.
result Different uncertainty measures are more suitable for different tasks (e.g., selective prediction, out-of-distribution detection, active learning).
TRUST improves structure learning with tractable uncertainty.
problem Capturing uncertainty in structure learning for causal DAGs.
method Probabilistic circuits for posterior inference.
result Probabilistic circuits enhance structure learning quality and uncertainty.
A new method estimates uncertainty without explicit prediction models.
problem Costly data acquisition in machine learning.
method Distance-weighted Class Impurity method for uncertainty estimation.
result Distance-weighted Class Impurity effectively estimates uncertainty without prediction models.
Paper quantifies epistemic uncertainty in deep learning.
problem Uncertainty in deep learning models, especially epistemic uncertainty.
method Dissects epistemic uncertainty into procedural and data variability, proposes estimation methods.
result Demonstrates how proposed methods overcome computational challenges and provide guidance for modeling and data collection.
The paper introduces new measures for quantifying uncertainty in machine learning.
problem Uncertainty representation and quantification in machine learning.
method Proper scoring rules for aleatoric and epistemic uncertainty quantification.
result Established a natural bridge between credal set and second-order distribution representations of uncertainty.
QUAM improves uncertainty quantification in deep learning models.
problem Estimating epistemic uncertainty in deep learning models.
method QUAM identifies regions with high divergence between predictions and a reference model.
result QUAM has lower approximation error of epistemic uncertainty compared to previous methods.
This study introduces axioms to assess regression uncertainty measures.
problem Limited formal justification and evaluations of uncertainty measures in regression settings.
method Introduces axioms and analyzes entropy- and variance-based measures in a predictive exponential family context.
result Provides a principled foundation for reliable uncertainty assessment in regression.
Study improves AI's handling of uncertainty.
problem Uncertainty in AI models, especially with limited data.
method Integrates theories, latest developments, and practical applications.
result Novel definition of total uncertainty in AI.
Unified Uncertainty Calibration improves AI predictions by combining different types of uncertainty.
problem AI classifiers struggle with uncertainty, leading to miscalibrated predictions and poor performance.
method Unified Uncertainty Calibration (U2C) combines aleatoric and epistemic uncertainties to improve prediction quality.
result U2C outperforms traditional reject-or-classify methods across various ImageNet benchmarks.
Paper analyzes uncertainty metrics in ensemble learning for healthcare AI.
problem Selecting appropriate uncertainty metrics for ensemble learners in healthcare AI.
method Rigorous analysis of two uncertainty metrics: ensemble mean and variance.
result Ensemble mean is preferable to ensemble variance for decision making in healthcare AI.
Unified taxonomy for ML uncertainty in physics, validated.
problem Uncertainty quantification in machine learning for physics.
method Unified taxonomy, principled validation tools.
result Illustrated validation tools with examples.
Various strategies for active learning have been proposed in the machine learning literature. In uncertainty sampling, which is among the most popular approaches, the active learner sequentially queries the label of those instances for which its current prediction is maximally uncertain. The predictions as well as the …
This paper studies directed exploration for reinforcement learning agents by tracking uncertainty about the value of each available action. We identify two sources of uncertainty that are relevant for exploration. The first originates from limited data (parametric uncertainty), while the second originates from the dist…
Paper introduces uncertainty injection for deep learning robust optimization.
problem Uncertainty in input data affects deep learning model performance in optimization problems.
method Uncertainty injection scheme for training deep learning models to produce robust solutions.
result Proposed scheme improves robustness of solutions in wireless communications applications.
Survey and framework for consistent uncertainty quantification in deep learning.
problem Partial uncertainty coverage and inconsistencies in deep learning uncertainty quantification.
method Bayes' theorem and conditional probability densities applied to all major sources of uncertainty.
result Improved robustness and reliability of neural network predictions in real-world scenarios.
Bayesian autoencoders quantify anomaly uncertainty for safer machine learning.
problem Lack of uncertainty quantification in autoencoders for anomaly detection.
method Formulated Bayesian autoencoders to quantify epistemic and aleatoric anomalies.
result Demonstrated effectiveness of BAEs on benchmark and real datasets.
Enhances uncertainty estimation in neural networks using Dirichlet-based MC Dropout.
problem Deterministic predictions without uncertainty estimates in neural networks.
method Integrates Dirichlet-based framework within Monte Carlo Dropout.
result Improves quality of uncertainty estimates in deep learning models.
Bayesian analysis reveals epistemic uncertainty as a key diagnostic for delayed generalization in in-context learning.
problem Delayed generalization in in-context learning from few examples.
method Bayesian perspective, modular arithmetic tasks, approximate Bayesian techniques, spectral mechanism analysis.
result Epistemic uncertainty collapses sharply when the model groks, indicating a practical diagnostic of generalization.
The volume of a credal set correlates with epistemic uncertainty in binary classification but not in multi-class.
problem Representing and quantifying epistemic uncertainty in machine learning.
method Examined the geometric representation of credal sets as d-dimensional polytopes and their volume as a measure of uncertainty. result The volume of a credal set is a meaningful measure of epistemic uncertainty in binary classification but not in multi-class.
Simple neural net outperforms complex uncertainty methods.
problem Reliable uncertainty estimation from deterministic models.
method A simple baseline using a single softmax neural net with residual connections and spectral normalization.
result Simple neural net outperforms DUQ and SNGP on uncertainty prediction.
Develops a framework to quantify uncertainties in multiple ML models.
problem Uncertainty in ML model predictions and model inputs.
method Develops a theoretical framework to decouple and transform uncertainties.
result Generates joint distribution of ML predictions considering uncertainties.
Second-order methods fail to fully quantify epistemic uncertainty, leading to biased predictions.
problem Incomplete quantification of epistemic uncertainty in machine learning models.
method Analysis of existing second-order uncertainty estimation methods.
result Current methods overestimate aleatoric uncertainty and underestimate epistemic uncertainty, leading to biased predictions.
Ultrasonic guided waves are commonly used to localize structural damage in infrastructures such as buildings, airplanes, bridges. Damage localization can be viewed as an inverse problem. Physical model based techniques are popular for guided wave based damage localization. The performance of these techniques depend on …
Bayesian meta learning improves uncertainty quantification in regression.
problem Trusting uncertainty quantification in Bayesian regression.
method Trust-Bayes framework for Bayesian meta learning, optimizing for trustworthy uncertainty quantification.
result Lower bounds and sample complexity for trustworthy uncertainty quantification are characterized.
MAPIE provides uncertainty quantification for ML models.
problem Estimating uncertainties in ML model predictions.
method Conformal prediction methods for single-output regression and multi-class classification.
result Strong theoretical guarantees on marginal coverages.
Proposes a method to enhance exploration in RL using temporal difference uncertainties.
problem Challenges in estimating uncertainty in non-tabular reinforcement learning settings.
method Estimates uncertainty over value function using temporal difference errors and incorporates it as an intrinsic reward.
result Demonstrates improved exploration in various tasks, including Deep Sea and Atari 2600 environments.
A new method improves uncertainty estimation in deep learning, especially for hard-to-label samples.
problem Improving uncertainty estimation for hard-to-label samples in deep learning.
method Introduces Fisher Information Matrix (FIM) to dynamically reweight objective loss terms.
result Consistently outperforms traditional evidential neural networks in uncertainty estimation tasks.