Qualitative analysis of MC dropout for NN model uncertainty.
problem Measuring uncertainty in neural network models.
method Mathematical formulation of Monte Carlo dropout and its benefits/costs in NN models.
result Potential benefits and associated costs of using MC dropout in NN models.
A fast single-shot MC dropout method for neural networks.
problem Inability of DNNs to provide uncertainty measures for new situations.
method Analytically approximates MC dropout for fully connected networks.
result Approach preserves BDNN advantages while being faster.
MC-CP combines adaptive MC dropout with conformal prediction for robust uncertainty quantification.
problem Deploying deep learning models in safety-critical applications requires reliable confidence estimates.
method MC-CP integrates adaptive Monte Carlo dropout with conformal prediction to improve model performance.
result MC-CP significantly outperforms state-of-the-art UQ methods in both classification and regression tasks.
MC Dropout is re-evaluated as not Bayesian, affecting predictive posterior and multimodality.
problem MC Dropout's Bayesian properties and predictive posterior are questioned.
method Re-evaluation of MC Dropout's properties, including a new VI engine in pytorch.
result MC Dropout does not produce a faithful Bayesian predictive posterior.
Enhances uncertainty estimation in medical image segmentation.
problem Frequency-related noise in medical imaging leads to biased uncertainty estimates.
method Extends MC-Dropout to the frequency domain for better uncertainty estimation.
result MC-Frequency Dropout improves calibration and uncertainty in semantic segmentation.
Enhances uncertainty estimation in neural networks using Dirichlet-based MC Dropout.
problem Deterministic predictions without uncertainty estimates in neural networks.
method Integrates Dirichlet-based framework within Monte Carlo Dropout.
result Improves quality of uncertainty estimates in deep learning models.
A fast method for neural networks that provides uncertainty measures.
problem Inability of neural networks to provide reliable uncertainty measures for new situations.
method Moment propagation (MP) for analytically approximating MC dropout.
result Single-shot MC dropout approximation as fast as NNs with similar uncertainty measures.
Batch normalisation doesn't affect variational inference but fails for larger batch sizes.
problem Failure of Monte Carlo Batch Normalisation (MCBN) for capturing epistemic uncertainty in larger batch sizes.
method Investigated MCBN as an approximate inference technique for Bayesian neural networks, showing its limitations and providing insights for improvement.
result For larger batch sizes, MCBN fails to capture epistemic uncertainty, requiring the batch size to be a variational parameter.
Study on MC dropout in wide neural networks and its convergence to Gaussian processes.
problem Understanding the behavior of Monte Carlo dropout in wide neural networks.
method Rigorously studied the limiting distribution of wide untrained NNs under dropout, proving convergence to Gaussian processes. Investigated correlations and non-Gaussian behavior in finite width NNs.
result Wide untrained neural networks under dropout converge to Gaussian processes for fixed sets of weights and biases.
Bayesian Neural Networks improve uncertainty modeling in facial emotion recognition.
problem High aleatoric uncertainty and visual ambiguity in facial emotion recognition.
method Bayesian Neural Networks approximated using MC-Dropout, MC-DropConnect, or Ensemble methods.
result Bayesian Neural Networks produce more human-like output probabilities.
Develops a neural surrogate for proton dose calculation using Monte Carlo dropout uncertainty.
problem Computational demand in proton therapy workflows requiring repeated evaluations.
method Integrates Monte Carlo dropout into a neural network surrogate for fast, differentiable dose predictions and uncertainty quantification.
result Shows significant speedups over MC while retaining uncertainty information.
Early detection of incipient faults is of vital importance to reducing maintenance costs, saving energy, and enhancing occupant comfort in buildings. Popular supervised learning models such as deep neural networks are considered promising due to their ability to directly learn from labeled fault data; however, it is kn…
Develops a multilevel Monte Carlo framework with dropout for efficient uncertainty quantification.
problem Efficiently quantify uncertainty in complex models using dropout.
method Integrates multilevel Monte Carlo with Monte Carlo dropout, creating coupled estimators to reduce variance.
result Demonstrates significant variance reduction and efficiency gains over single-level Monte Carlo dropout.
We show that dropout training is best understood as performing MAP estimation concurrently for a family of conditional models whose objectives are themselves lower bounded by the original dropout objective. This discovery allows us to pick any model from this family after training, which leads to a substantial improvem…
Radial BNNs offer a scalable, continuous weight distribution for Bayesian deep learning.
problem Discrete support in Bayesian deep learning methods like MC dropout.
method Radial BNNs with full support over weight-space.
result Radial BNNs outperform discrete-support methods in real-world applications.
Study on Monte-Carlo dropout for uncertainty estimation in neural networks.
problem Variability in uncertainty estimation quality through Monte-Carlo dropout.
method Analysis of dropout behavior in neural networks.
result Observation of interesting properties for uncertainty estimation.
PGA neural network improves uncertainty quantification in lake temperature modeling.
problem Quantifying uncertainties in lake temperature models while maintaining physical consistency.
method Integrates physical constraints into neural networks using Monte Carlo Dropout.
result Ensures better generalizability and physical consistency in MC estimates.
Simplifies neural regression by combining two sub-networks for predictions and uncertainties.
problem Neural networks underestimate uncertainty, leading to overly confident predictions.
method Extends IRLS to a two-sub-network approach with shared representations and complementary loss functions.
result Proposed network is simpler to implement and more robust to uncertainty variations.
A cubing strategy identifies stable hyperparameter regions for uncertainty quantification in spatial deep learning.
problem Uncertainty quantification in spatial deep learning models.
method Cubing-based diagnostic framework to recursively partition hyperparameter space and evaluate regions using scoring rules.
result Our approach produces competitive or superior predictive intervals compared to a statistical baseline model.
QUAM improves uncertainty quantification in deep learning models.
problem Estimating epistemic uncertainty in deep learning models.
method QUAM identifies regions with high divergence between predictions and a reference model.
result QUAM has lower approximation error of epistemic uncertainty compared to previous methods.
Two main obstacles preventing the widespread adoption of variational Bayesian neural networks are the high parameter overhead that makes them infeasible on large networks, and the difficulty of implementation, which can be thought of as "programming overhead." MC dropout [Gal and Ghahramani, 2016] is popular because it…
We prove, under two sufficient conditions, that idealised models can have no adversarial examples. We discuss which idealised models satisfy our conditions, and show that idealised Bayesian neural networks (BNNs) satisfy these. We continue by studying near-idealised BNNs using HMC inference, demonstrating the theoretic…
The paper introduces a method to decompose variance in twin networks for better treatment effect estimation.
problem Accurate treatment effect estimation requires reliable uncertainty measures to locate model failures.
method Layer-wise variance decomposition using Monte Carlo Dropout in twin networks.
result The encoder component dominates under distributional shift, providing a practical diagnostic for data collection.
We consider active learning of deep neural networks. Most active learning works in this context have focused on studying effective querying mechanisms and assumed that an appropriate network architecture is a priori known for the problem at hand. We challenge this assumption and propose a novel active strategy whereby …
Deep active inference agents learn complex environments using Monte-Carlo methods.
problem Understanding and modeling biological intelligence in complex, continuous state-spaces.
method Neural architecture for deep active inference agents using multiple forms of Monte-Carlo sampling.
result Deep active inference agents can learn environmental dynamics and plan future actions.
Deep neural networks (DNNs) provide state-of-the-art results for a multitude of applications, but the approaches using DNNs for multimodal audiovisual applications do not consider predictive uncertainty associated with individual modalities. Bayesian deep learning methods provide principled confidence and quantify pred…
This paper evaluates scalable uncertainty estimation methods for DNN-based molecular property prediction.
problem Uncertainty quantification in DNN models for molecular property prediction.
method Quantitative comparison of MC-Dropout, deep ensembles, and bootstrapping on the QM9 dataset.
result Ensembling and bootstrapping consistently outperform MC-Dropout, with different context-specific pros and cons.
Paper confirms MCS spaces are equivalent to CS sets.
problem Understanding the equivalence of MCS and CS sets.
method Analyzing the MCS stratification and intrinsic stratification.
result MCS spaces are equivalent to CS sets with respect to their stratification.
Bayesian active learning improves stress and affect detection on wearable devices.
problem Handling unlabeled data in real-time for stress and affect detection.
method Bayesian Neural Networks with Monte-Carlo Dropout and suitable acquisition functions.
result Framework achieves significant efficiency boost and low number of acquired pool points.
Bayesian approach improves image captioning quality metrics.
problem Improving image captioning quality metrics.
method Bayesian Self-Critical Sequence Training (B-SCST) with Monte Carlo dropout approximate variational inference.
result B-SCST improves CIDEr-D scores on various image captioning datasets.
Measuring uncertainty is a promising technique for detecting adversarial examples, crafted inputs on which the model predicts an incorrect class with high confidence. But many measures of uncertainty exist, including predictive en- tropy and mutual information, each capturing different types of uncertainty. We study th…
Let X be a non-compact geometrically finite hyperbolic 3-manifold without cusps of rank 1. The deformation space $\mc{H}$ of X can be identified with the Teichmüller space $\mc{T}$ of the conformal boundary of X as the graph of a section in $T^*\mc{T}$. We construct a Hermitian holomorphic line bundle $\mc{L}$ on…
New algebraic structures help categorify link invariants.
problem Classifying and distinguishing links and virtual links.
method Introducing mc-biquandles and categorifying homsets.
result New link invariants defined via mc-biquandle coloring quivers.
Ensemble approaches for uncertainty estimation have recently been applied to the tasks of misclassification detection, out-of-distribution input detection and adversarial attack detection. Prior Networks have been proposed as an approach to efficiently \emph{emulate} an ensemble of models for classification by paramete…
We determine the homogeneous Kähler diffeomorphism FC which expresses the Kähler two-form on the Siegel-Jacobi ball $\mc{D}^J_n=\C^n\times \mc{D}_n$ as the sum of the Kähler two-form on $\C^n$ and the one on the Siegel ball $\mc{D}_n$. The classical motion and quantum evolution on $\mc{D}^J_n$ determined by a hermiti…
We give a homological characterization of n-manifolds whose universal covering $\Wi M$ has Gromov's macroscopic dimension $\dim_{mc}\Wi M<n$. As the result we distinguish dimmc from the macroscopic dimension dimMC defined by the author \cite{Dr}. We prove the inequality $\dim_{mc}\Wi M<\dim_{MC}\Wi M=n$ f…
Paper proposes a method to improve prediction intervals for neural networks.
problem Improving prediction intervals for neural network models.
method Adapting extremely randomized trees to neural networks to create ensembles.
result The method yields gains in out-of-sample accuracy and is superior to existing methods.
We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning. Stochastic Weight Averaging (SWA), which computes the first moment of stochastic gradient descent (SGD) iterates with a modified learning rate schedule, has recently been sho…
Favour speeds up variance estimation in BNNs, making them practical for performance-critical tasks.
problem Slow inference in Bayesian Neural Networks (BNNs) hinders their adoption.
method A fast algorithm for updating a diagonal-plus-low-rank matrix approximation under various operations.
result Favour is as fast as 2-3 inference samples but matches the performance of 10-100 samples.
Advanced Dropout improves DNN performance without requiring model-specific dropout techniques.
problem Overfitting in deep neural networks due to lack of data.
method Model-free, end-to-end training with adaptive dropout rate optimization.
result Advanced Dropout outperforms other techniques on various datasets.
UTS improves DNN uncertainty calibration without labels, robust to domain shift.
problem Improving uncertainty calibration of DNNs under domain shift.
method UTS uses unlabeled test samples and a novel weighted NLL loss function.
result UTS outperforms other methods in domain shift scenarios.
Paper detects gradual changes in cluster structure using MC fusion.
problem Detecting gradual changes in cluster structure over time.
method MC fusion for multiple mixture numbers, examining MC transition.
result Accurately captures cluster structure during transitional periods.
We study the structure of classical groups of equivalences for smooth multigerms f:(N,S)→(P,y), and extend several known results for monogerm equivalences to the case of mulitgerms. In particular, we study the group $\A$ of source- and target diffeomorphism germs, and its stabilizer $\A_f$. For monogerms $…
In many real-world learning scenarios, features are only acquirable at a cost constrained under a budget. In this paper, we propose a novel approach for cost-sensitive feature acquisition at the prediction-time. The suggested method acquires features incrementally based on a context-aware feature-value function. We for…
ADRL improves participant selection in MCS systems.
problem Designing a participant selection algorithm for different MCS systems with multiple goals.
method Auxiliary-task based deep reinforcement learning (ADRL) using transformers and pointer networks.
result ADRL outperforms other baselines in various MCS settings.
Proposes mCS for multivariate selection with FDR control.
problem Selecting high-quality candidates from multivariate datasets.
method Introduces regional monotonicity and multivariate nonconformity scores.
result Significantly improves selection power with FDR control.
Compressed Monte Carlo improves efficiency in Bayesian inference.
problem Efficiently approximating posterior distributions in Bayesian models.
method Introduces Compressed Monte Carlo (C-MC) to compress statistical information.
result C-MC schemes outperform traditional methods in particle filtering and adaptive IS algorithms.
BayesDLL offers a PyTorch library for Bayesian deep learning with large models.
problem Bayesian inference for large-scale deep networks.
method Variational inference, MC-dropout, stochastic-gradient MCMC, Laplace approximation.
result BayesDLL can handle Vision Transformers and pre-trained model weights as priors.