Survey of uncertainty in neural networks.
problem Understanding and quantifying uncertainty in neural networks predictions.
method Review of recent advances, challenges, and potential opportunities in uncertainty estimation.
result Comprehensive overview of uncertainty sources and approaches.
Proposes a new method for uncertainty estimation in neural networks.
problem Estimating uncertainty in neural networks.
method Samples outputs from Gaussian distributions parametrized by mean and variance sub-layers.
result Achieves better uncertainty quality than other methods.
Bayesian neural network models improve uncertainty quantification in multivariate regression.
problem Uncertainty quantification in multivariate regression models with heteroscedastic noise.
method Proposes Bayesian Last Layer neural network models and EM algorithms for parameter learning.
result Capable of disentangling aleatoric and epistemic uncertainty.
Probabilistic deep learning uses neural networks and models to handle uncertainty.
problem Handling uncertainty in deep learning models.
method Two approaches: probabilistic neural networks and deep probabilistic models.
result TensorFlow Probability library supports both approaches.
Enhances neural network regression performance by modeling weight and variance uncertainty.
problem Improving predictive performance of neural networks for regression tasks.
method Extended Blundell's framework to include variance uncertainty, using a full posterior distribution over variance parameters.
result Explicitly modeling variance uncertainty improves generalization of Bayesian neural networks.
Bayesian inference improves neural network predictions by separating aleatoric and epistemic uncertainties.
problem Improving prediction accuracy of neural networks by quantifying and separating uncertainties.
method Approximated posterior distributions using deep ensembles for various neural network architectures.
result Prediction accuracy depends on both aleatoric and epistemic uncertainties, not just marginalized uncertainty.
Cooperative model disentangles data uncertainties.
problem Disentangling aleatoric and epistemic uncertainties in real-world data.
method Cooperatively trains a variance estimation network with a Bayesian neural network.
result Improves mean estimation and disentangles uncertainties.
Paper improves uncertainty quantification in PINNs using error bounds and solution bundles.
problem Uncertainty quantification in PINNs for differential equation systems.
method Two-step procedure with Bayesian Neural Networks and heteroscedastic variance.
result Improved uncertainty estimation over PINNs solutions in differential equation systems.
Overparametrized neural networks retain significant epistemic uncertainty even with sufficient data.
problem Epistemic uncertainty in overparametrized neural networks persists despite model identifiability.
method Analysis of non-identifiability and characterization of residual uncertainty in one-hidden-layer ReLU networks.
result Substantial parameter uncertainty remains even when the underlying function is fully identified.
New BNN architectures reduce computational cost for uncertainty quantification.
problem High computational cost in Bayesian neural networks.
method Partial trace-class Bayesian neural networks (PaTraC BNNs).
result Comparable uncertainty quantification with fewer parameters.
Delta Variances efficiently estimate epistemic uncertainty in neural networks.
problem Mitigating uncertainty in neural networks with limited data.
method Delta Variances, a computationally efficient algorithm for epistemic uncertainty quantification.
result Empirically competitive results with a single gradient computation.
Bayesian neural networks help quantify prediction uncertainties in neural models.
problem Uncertainty in neural network predictions due to model randomness and lack of knowledge.
method Bayesian statistical framework to categorize uncertainty.
result Errors in neural network predictions can be obtained and characterized.
The paper proposes a method for better uncertainty estimation in neural networks.
problem Estimating predictive uncertainty in neural networks is crucial but challenging.
method The paper proposes a function-space variational inference method to infer a posterior distribution over functions.
result The proposed method leads to state-of-the-art uncertainty estimation and predictive performance.
Bayesian Neural Networks help quantify uncertainty in deep learning predictions.
problem Uncertainty quantification in deep learning predictions.
method Bayesian statistics applied to neural networks.
result Design, implementation, training, and evaluation of Bayesian Neural Networks.
Bayesian neural networks enhance uncertainty estimation in graph contrastive learning.
problem Uncertainty in graph contrastive learning when labeled data is scarce.
method Variational Bayesian neural networks for uncertainty estimation.
result Improved uncertainty estimation and downstream performance on semi-supervised node-classification tasks.
Enhances uncertainty estimation in neural networks using Dirichlet-based MC Dropout.
problem Deterministic predictions without uncertainty estimates in neural networks.
method Integrates Dirichlet-based framework within Monte Carlo Dropout.
result Improves quality of uncertainty estimates in deep learning models.
Qualitative analysis of MC dropout for NN model uncertainty.
problem Measuring uncertainty in neural network models.
method Mathematical formulation of Monte Carlo dropout and its benefits/costs in NN models.
result Potential benefits and associated costs of using MC dropout in NN models.
For many applications it is critical to know the uncertainty of a neural network's predictions. While a variety of neural network parameter estimation methods have been proposed for uncertainty estimation, they have not been rigorously compared across uncertainty measures. We assess four of these parameter estimation m…
Proposes a new criterion for reliable uncertainty estimation in deep neural networks.
problem Inability of existing approaches to provide reliable uncertainty estimates for deep neural networks.
method Develops a density uncertainty layer architecture that satisfies the proposed criterion.
result Density uncertainty layers provide more reliable uncertainty estimates and robust out-of-distribution detection.
New framework quantifies uncertainties in neural network explanations.
problem Lack of methods to quantify uncertainties in neural network explanations.
method Converts any explanation method into a Bayesian neural network method, modeling uncertainties.
result Allows quantification of explanation uncertainties and appropriate confidence levels.
Simplifies neural regression by combining two sub-networks for predictions and uncertainties.
problem Neural networks underestimate uncertainty, leading to overly confident predictions.
method Extends IRLS to a two-sub-network approach with shared representations and complementary loss functions.
result Proposed network is simpler to implement and more robust to uncertainty variations.
Bayesian neural networks simplified with input augmentation.
problem Uncertainty in deep learning models.
method Layer-wise input augmentation to induce uncertainty distributions.
result State-of-the-art performance in uncertainty representation.
USNRT uses tree-structured learning to improve uncertainty quantification of variance networks.
problem Improving uncertainty quantification of variance networks.
method Tree-structured local neural network model that partitions feature space into regions for training region-specific neural networks to predict mean and variance.
result USNRT shows superior performance in estimating uncertainty with variances on UCI datasets compared to recent methods.
Single-pass method estimates neural network uncertainty.
problem Uncertainty estimation in deep learning requires multiple passes.
method Probabilistic reasoning over neural network depths.
result Single forward pass for uncertainty estimation.
This paper connects noise injection to Bayesian inference for neural networks, improving model uncertainty.
problem Improving the reliability and confidence of neural network predictions through uncertainty quantification.
method Introducing noise into neural network parameters during training and inference to estimate prediction uncertainty.
result The MCNI method outperforms baseline models in regression and classification tasks.
Improved neural network regression uncertainty estimation.
problem Neural networks lack classical uncertainty due to finite data.
method Bootstrapped Deep Ensembles, incorporating parametric bootstrap.
result Significantly improved uncertainty estimation compared to standard Deep Ensembles.
Post-hoc uncertainty quantification improves on pre-trained neural networks without underfitting.
problem Uncertainty quantification in neural networks is underfitting or computationally demanding.
method Gaussian Process Activation function (GAPA) for neuron-level uncertainty, with two methods: GAPA-Free and GAPA-Variational.
result GAPA-Variational outperforms Laplace approximation on most datasets in uncertainty quantification metrics.
New method improves estimation of neural network aleatoric uncertainty.
problem Existing methods overestimate aleatoric uncertainty in neural networks.
method Proposes a new de-noising method to estimate data uncertainty more accurately.
result Demonstrates better approximation of actual data uncertainty.
The paper introduces a method to quantify uncertainty in neural networks without parametric assumptions.
problem Uncertainty quantification for neural network predictions.
method Nonparametric estimation of conditional label distribution using Nadaraya-Watson kernel.
result The method effectively disentangles aleatoric and epistemic uncertainties.
Bayesian neural networks outperform calibrated neural networks for tabular data.
problem Uncertainty in neural network predictions for tabular data.
method Bayesian neural networks vs. post-hoc calibration methods.
result Bayesian neural networks yield competitive performance compared to calibrated neural networks.
Proposes a method to identify critical regions in neural networks using adversarial attacks.
problem Capturing uncertainty in neural networks near decision boundaries.
method Adversarial attack method to derive uncertainty from input perturbations.
result The proposed method outperforms other uncertainty methods in capturing model uncertainty.
K-StoNet improves neural networks by avoiding local minima and assessing uncertainty.
problem Local minima and prediction uncertainty in deep neural networks.
method Combines SVR with latent variable model, using RBF kernel for feature space mapping and IRO algorithm for training.
result The model asymptotically converges to the global optimum and assesses prediction uncertainty easily.
A new method to predict uncertainties in trained neural networks.
problem Reliability of trained neural networks outside their training domain.
method Prediction rigidities as a constrained optimization problem.
result Cheap uncertainties without modifying neural networks.
Efficiently quantifies uncertainty in subsurface flow using neural networks guided by theory.
problem Uncertainty in dynamic subsurface flow predictions.
method Theory-guided Neural Network (TgNN) for efficient uncertainty quantification.
result TgNN surrogate improves efficiency of uncertainty quantification compared to MC method.
New method extracts aleatoric and epistemic uncertainties from regression-based neural networks.
problem Need for principled uncertainty reasoning in machine learning systems.
method Learning evidential distributions for aleatoric and epistemic uncertainties.
result Allows for the simultaneous extraction of both uncertainties without sampling or out-of-distribution data.
New method calibrates neural network uncertainty for medical images.
problem Uncalibrated probabilistic outputs from deep neural networks in medical diagnosis.
method Functional space variational inference for Bayesian neural networks.
result Better calibrated uncertainty estimates at lower computational cost.
BBNN improves neural network accuracy and uncertainty quantification.
problem Overfitting and lack of interpretability in probabilistic neural networks.
method Boosted Bayesian Neural Networks (BBNN) using Boosting Variational Inference (BVI).
result BBNN achieves ~5% higher accuracy and superior uncertainty quantification.
Wasserstein Dropout improves uncertainty estimation in neural networks.
problem Estimating neural uncertainties for safe machine learning.
method A purely non-parametric approach using dropout-based sub-network distributions and Wasserstein distance.
result Wasserstein Dropout outperforms state-of-the-art methods in uncertainty estimation.
Bayesian inference for neural networks improves uncertainty quantification.
problem Improving predictive uncertainty in neural networks.
method Ensemble Kalman filter extensions and interacting particle systems.
result Effective methods for quantifying predictive uncertainty in neural networks.
Bayesian Neural Networks improve geophysical model ensembles with reduced uncertainty.
problem Improving geophysical model projections and uncertainty quantification.
method Developed a Bayesian Neural Network ensemble strategy for geophysical models.
result Bayesian Neural Network ensemble outperforms existing methods in ozone prediction.
The paper evaluates and improves uncertainty estimates in neural networks for safety-critical applications.
problem Quantifying uncertainty in neural networks for safety-critical systems.
method Proposes a statistical test for evaluating uncertainty realism in neural networks and transfers a classification architecture to image-to-image tasks.
result The variational U-Net architecture significantly improves uncertainty realism in image-to-image tasks compared to a plain model.
Bayesian Neural Networks improve uncertainty reasoning in NNs.
problem Frequentist implementation of NNs cannot reason about uncertainty in predictions.
method Introduces Bayesian Neural Networks and compares approximate inference methods.
result Future research can improve on current methods of inference.
The paper presents methods to improve uncertainty calibration in Bayesian Neural Networks.
problem Uncalibrated Bayesian Neural Networks often lead to overconfidence.
method The paper uses alpha-divergences from Information Geometry for calibration.
result Calibration using alpha-divergences provides better uncertainty estimates and is more efficient.
Unified method for input, data, and model uncertainty in neural networks.
problem Uncertainty in neural network inputs and outputs.
method Propagating uncertainty through inputs using a unified formulation.
result More stable decision boundaries with input noise, and propagation of input uncertainty to model outputs.
New methods for uncertainty in neural networks with leaky ReLU activations.
problem Uncertainty in feed-forward neural networks with random input perturbations.
method Analytical expressions for PDF and moments of neural network output, linearization of leaky ReLU, Gaussian copula surrogate models.
result Accurate statistical results for large input perturbations, excellent agreement with Monte Carlo simulations.
Method converts neural networks to function space for better uncertainty quantification.
problem Lack of uncertainty estimates and difficulty in incorporating new data in deep neural networks.
method Dual parameterization to convert from weight space to function space, enabling sparse representation.
result Compact and principled way to capture uncertainty and incorporate new data.
Study on Monte-Carlo dropout for uncertainty estimation in neural networks.
problem Variability in uncertainty estimation quality through Monte-Carlo dropout.
method Analysis of dropout behavior in neural networks.
result Observation of interesting properties for uncertainty estimation.
This work improves neural network trustworthiness through uncertainty estimation.
problem Overconfident neural networks lead to poor performance under distribution shifts.
method Develops a general uncertainty framework for neural networks, including classification with rejection.
result Improves model trustworthiness and robustness in decision-making tasks.