Method combines LD and Fermat Distance for neural network uncertainty.
problem Measuring uncertainty in neural network predictions.
method Statistical Depth (LD) combined with Fermat Distance.
result Effective uncertainty estimation without impacting original model performance.
Single-pass method estimates neural network uncertainty.
problem Uncertainty estimation in deep learning requires multiple passes.
method Probabilistic reasoning over neural network depths.
result Single forward pass for uncertainty estimation.
Depth uncertainty networks don't improve with bias correction, contrary to expectations.
problem Improving performance in active learning with overparameterised models like NNs.
method Depth uncertainty networks, compared to underparameterised models, show no improvement in performance with bias correction.
result Depth uncertainty networks do not improve with bias correction, unlike underparameterised models.
DUNs improve active learning by dynamically adjusting model complexity.
problem Active learning datasets grow, leading to model bias or overfitting.
method Depth Uncertainty Networks (DUNs) infer model depth for flexibility and complexity.
result DUNs outperform other BNN variants on active learning tasks, reducing overfitting.
Paper introduces MCSD, a method for uncertainty estimation in deep learning.
problem Need for reliable uncertainty quantification in deep neural networks.
method Theoretical connection to variational inference and empirical benchmarking of MCSD.
result MCSD achieves competitive predictive accuracy and improves uncertainty ranking.
New algorithm quantifies uncertainty in regression models for complex data types.
problem Uncertainty quantification in regression models for complex data types.
method Model-free uncertainty quantification algorithm based on conditional depth measures and kernel mean embeddings.
result Provides faster convergence rates and non-asymptotic guarantees for prediction regions.
New method improves reliability of depth estimation models.
problem Uncertainty quantification in large-scale vision models.
method Parameter-efficient Bayesian neural networks with PEFT methods.
result Combining PEFT methods with Bayesian inference enhances predictive performance.
Extends partitioned local depth concept with probabilistic considerations.
problem Uncertain, variable, and conflicting information in data.
method Partitioned local depth with probabilistic concepts of local relevance and support division.
result Extends original ideas to handle uncertain data.
One-shot neural architecture search allows joint learning of weights and network architecture, reducing computational cost. We limit our search space to the depth of residual networks and formulate an analytically tractable variational objective that allows for obtaining an unbiased approximate posterior over depths in…
Measures difficulty of predictions to improve deep learning models.
problem Understanding and improving deep learning models.
method Introduces a measure called prediction depth to assess example difficulty.
result Surprising relationships between prediction depth and model performance.
Regression Prior Networks improve ensemble performance on regression tasks.
problem Improving ensemble performance on regression tasks.
method Extending Prior Networks and Ensemble Distribution Distillation (EnD2) to regression tasks using the Normal-Wishart distribution. result Regression Prior Networks yield performance competitive with ensemble approaches on regression tasks.
We develop a scalable method for Bayesian neural networks with stochastic differential equations.
problem Uncertainty quantification in deep neural networks.
method Gradient-based stochastic variational inference in continuous-depth Bayesian neural networks.
result Gradient estimator with zero variance as the approximation improves.
Real-time uncertainty estimation for computer vision tasks.
problem Real-time inference of uncertainty in deep learning models.
method Uncertainty-Aware Distribution Distillation method for fast inference.
result Significantly reduced inference time with improved uncertainty and predictive performance.
DBU models struggle with robust uncertainty estimates under adversarial attacks.
problem Robustness of DBU models in adversarial settings.
method Investigated robustness of DBU models under adversarial attacks; proposed median smoothing approach.
result DBU models are not robust in indicating correctly and wrongly classified samples, detecting adversarial examples, and distinguishing ID and OOD data.
We present a sampling-free approach for computing the epistemic uncertainty of a neural network. Epistemic uncertainty is an important quantity for the deployment of deep neural networks in safety-critical applications, since it represents how much one can trust predictions on new data. Recently promising works were pr…
Proposes a method to model uncertainty in neural ordinary differential equations.
problem Lack of uncertainty modeling and robustness in neural ordinary differential equations.
method Introduces a novel approach to model uncertainty by considering a distribution over the end-time of the ODE solver.
result Demonstrates the effectiveness of the proposed approaches in modelling uncertainty and robustness through experiments.
This paper explores BDL hyperparameters for robust polynomial mapping with noise.
problem Designing BDL hyperparameters for robust function mapping with uncertainty quantification.
method Mapping Bayesian connectionist representations to polynomials of varying orders and noise types.
result Optimal network depth and ensemble size for prediction and uncertainty quantification.
Proposes a method to quantify uncertainty in graph neural networks for node classification.
problem Uncertainty in graph neural networks for node classification.
method Bayesian uncertainty propagation (BUP) method embedding GNNs in a Bayesian framework.
result Demonstrates superior performance of the proposed method on benchmark datasets.
Density-Regression improves deep uncertainty estimation with faster inference.
problem Efficient uncertainty estimation under distribution shifts with modern deep models.
method Leverages density function for fast inference and distance-aware feature space.
result Density-Regression achieves competitive uncertainty estimation performance.
Improved VAE model enhances uncertainty estimation for out-of-distribution samples.
problem VAEs assign higher likelihood to out-of-distribution inputs.
method INCPVAE integrates noise contrastive prior into VAEs for reliable uncertainty estimation.
result INCPVAE outperforms standard VAEs in uncertainty estimation for OOD inputs.
We show that a neural network with arbitrary depth and non-linearities, with dropout applied before every weight layer, is mathematically equivalent to an approximation to a well known Bayesian model. This interpretation might offer an explanation to some of dropout's key properties, such as its robustness to over-fitt…
Proposes a new method for robust uncertainty quantification in regression tasks.
problem Robust uncertainty estimation for deep neural networks in regression tasks.
method Generalized Auxiliary Uncertainty Estimator (AuxUE) scheme, considering both aleatoric and epistemic uncertainties.
result DIDO method provides robust uncertainty estimates in noisy inputs, scalable to image-level and pixel-wise tasks.
As deep learning applications are becoming more and more pervasive in robotics, the question of evaluating the reliability of inferences becomes a central question in the robotics community. This domain, known as predictive uncertainty, has come under the scrutiny of research groups developing Bayesian approaches adapt…
New method attributes feature uncertainty in ML models using cooperative game theory.
problem Lack of feature-level uncertainty attribution in explainable AI.
method Proposes a novel, model-agnostic uncertainty attribution method using cooperative game theory and conformal prediction.
result Demonstrates improved runtime efficiency and practical utility in real-world applications.
Clarifies challenges in machine learning uncertainty quantification.
problem Inconsistent terminology and diverse technical requirements for trustworthy uncertainties.
method Examines estimation targets, uncertainty constructs, and problematic mappings.
result Advocates for alignment between intent and implementation in UQ.
Recent model-free reinforcement learning algorithms have proposed incorporating learned dynamics models as a source of additional data with the intention of reducing sample complexity. Such methods hold the promise of incorporating imagined data coupled with a notion of model uncertainty to accelerate the learning of c…
New method improves deep learning models in noisy label classification.
problem Improving deep learning models in noisy label classification.
method Analyzes loss and uncertainty changes during training, designs a new robust training method.
result Significantly outperforms other state-of-the-art methods in various deep learning models.
Develops a method to disaggregate aerosol optical depth into vertical extinction profiles.
problem Uncertainty in measuring aerosol vertical distributions due to limited observations.
method Bayesian nonparametric Gaussian process modeling using meteorological predictors.
result Model reconstructs realistic extinction profiles with well-calibrated uncertainty, outperforming idealized baselines.
Machine learning combines high- and low-fidelity models for efficient uncertainty quantification and optimization.
problem Efficiently combining high- and low-fidelity models for uncertainty quantification and optimization.
method Machine learning-based multi-fidelity methods for uncertainty quantification and optimization.
result Unified perspective on multi-fidelity priors for optimization.
This report provides an in-depth overview over the implications and novelty Generalized Variational Inference (GVI) (Knoblauch et al., 2019) brings to Deep Gaussian Processes (DGPs) (Damianou & Lawrence, 2013). Specifically, robustness to model misspecification as well as principled alternatives for uncertainty quantif…
Prob-GNN quantifies travel demand uncertainty with deep learning.
problem Uncertainty in travel demand prediction.
method Probabilistic Graph Neural Networks (Prob-GNN) framework.
result Probabilistic assumptions significantly impact uncertainty prediction.
Wasserstein active regression improves estimation precision.
problem Improving regression model accuracy through active learning.
method Combines Wasserstein distance and GroupSort Neural Networks for uncertainty quantification.
result Wasserstein active regression often provides more precise estimations.
Even though probabilistic treatments of neural networks have a long history, they have not found widespread use in practice. Sampling approaches are often too slow already for simple networks. The size of the inputs and the depth of typical CNN architectures in computer vision only compound this problem. Uncertainty in…
Single neuron learns predictive uncertainty in deep learning models.
problem Uncertainty estimation in deep learning models, especially with respect to model specification and training procedure.
method Introduces a non-parametric quantile estimation method using a single neuron.
result The method achieves competitive predictive uncertainty quantification quality and coverage compared to state-of-the-art solutions.
Study introduces a probabilistic framework for air-sea fluxes using neural networks.
problem Accurately quantifying air-sea fluxes for understanding interactions and improving weather/climate models.
method Gaussian distributions conditioned on input variables, artificial neural networks, eddy-covariance data, minimizing negative log-likelihood loss.
result Trained neural networks provide alternative mean flux estimates and quantify uncertainty.
Probabilistic programming aids in automatically dating ice cores, reducing manual error and uncertainty.
problem Automatically dating ice cores with high accuracy and capturing uncertainty.
method Probabilistic models and probabilistic programming for automatic inference.
result Demonstrated the use of probabilistic programming for ice core dating, showcasing its benefits and limitations.
Improved sampling for Bayesian neural networks reduces vanishing acceptance rates and increases predictive accuracy.
problem Sampling inefficiency in Bayesian neural networks, especially with deep architectures and large datasets.
method Approximate blocked Gibbs sampling to partition and sample subgroups of parameters.
result Increased predictive accuracy and quantification of predictive uncertainty in classification tasks.
A new depth measure for non-convex data supports, faster than halfspace depth.
problem Non-convex data supports in multivariate statistics.
method Extending halfspace depth to Reproducing Kernel Hilbert Space (RKHS).
result The new depth measure is consistent and can be computed faster.
The paper studies randomized approximations of Tukey's depth for log-concave isotropic data.
problem The challenge of approximating Tukey's depth in high dimensions.
method The study examines randomized algorithms for approximating Tukey's depth for log-concave isotropic data.
result Randomized algorithms correctly approximate maximal depth and close to zero depths but not intermediate depths.
This paper connects functional data analysis with machine learning techniques.
problem Lack of theoretical analysis for functional depths.
method Viewing functional depths as kernel mean embeddings in machine learning.
result Facilitates answers to open questions about functional depths.
Self-attention models benefit equally from width and depth, but beyond a certain point, depth becomes less efficient.
problem Understanding the optimal balance between depth and width in self-attention models.
method Theoretical predictions and empirical ablations on networks of varying depths and widths.
result An optimal width of 30K is recommended for a 1-Trillion parameter network, marking a significant width for self-attention models.
A new depth measure based on optimal control theory captures multi-modal data.
problem Statistical depths for high-dimensional data.
method Eikonal equations and optimal control theory.
result The new depth measure is robust under adversarial models.
A new depth measure and median defined on Hadamard manifolds.
problem Statistical depth and median on Hadamard manifolds.
method Horospherical depth and Busemann median defined using renormalized distance functions.
result The Busemann median exists for every Borel probability measure on Hadamard manifolds.
New approach uses loss functions to extend data depth for anomaly detection.
problem Anomaly detection in high-dimensional data.
method Introducing loss depths to generalize halfspace depth.
result New loss depths improve anomaly detection efficiency and interpretability.
Proves depth 2 neural networks can't approximate certain functions as well as depth 3 networks.
problem Approximating functions with depth 2 networks in high dimensions.
method Lower bound proof using worst-to-average-case random self-reducibility.
result Proves depth 2 networks can't approximate certain functions as well as depth 3 networks, resolving an open problem.
New findings on depth vs. width in neural networks, showing depth can improve learnability.
problem Understanding the role of depth in neural networks, especially when width is unbounded.
method Analyzing sample complexity for learnability in norm-controlled depth-2 and depth-3 ReLU networks.
result Depth can improve learnability of functions that are otherwise unlearnable with depth-2 networks.
Learning based methods have shown very promising results for the task of depth estimation in single images. However, most existing approaches treat depth prediction as a supervised regression problem and as a result, require vast quantities of corresponding ground truth depth data for training. Just recording quality d…
Introduces Polar Depth for analyzing multivariate heavy-tailed data extremes.
problem Analyzing the behavior of extremes from multivariate heavy-tailed distributions.
method Introduces Polar Depth, a novel statistical depth function expressed in polar coordinates.
result The polar depth of the largest observations converges to the polar depth of the limiting distribution as the threshold increases.