VarDeepPCA: A Sampling-Free Variational DNN Plugin for OOD Segmentation with Uncertainty Estimation
problem Deep neural networks (DNNs) fail to generalize to out-of-distribution (OOD) medical images due to variations in scanners and acquisition protocols.
method VarDeepPCA is a lightweight variational DNN framework that learns a distribution of valid anatomical geometries using small in-distribution datasets.
result VarDeepPCA restores segmentation maps produced by existing methods on OOD data to improve anatomical plausibility and reduce errors.
We present a sampling-free approach for computing the epistemic uncertainty of a neural network. Epistemic uncertainty is an important quantity for the deployment of deep neural networks in safety-critical applications, since it represents how much one can trust predictions on new data. Recently promising works were pr…
We propose a new Bayesian Neural Net formulation that affords variational inference for which the evidence lower bound is analytically tractable subject to a tight approximation. We achieve this tractability by (i) decomposing ReLU nonlinearities into the product of an identity and a Heaviside step function, (ii) intro…
There has recently been a concerted effort to derive mechanisms in vision and machine learning systems to offer uncertainty estimates of the predictions they make. Clearly, there are enormous benefits to a system that is not only accurate but also has a sense for when it is not sure. Existing proposals center around Ba…
Density-Softmax improves uncertainty estimation and robustness without sampling, reducing model size and latency.
problem Sampling-based uncertainty estimation methods suffer from large model size and high latency.
method Combines a Lipschitz-constrained feature extractor with the softmax layer to create a sampling-free deterministic framework.
result Density-Softmax reduces over-confidence under distribution shifts and achieves competitive results in uncertainty and robustness.
New method for privacy amplification without sampling for matrix factorization.
problem Privacy amplification for differentially private model training with matrix factorization.
method Sampling-free bounds based on Rényi divergence and conditional composition.
result Stronger privacy guarantees for small ε, applicable to various matrices.
VarDeepPCA refines medical image segmentation from small datasets, improving anatomical plausibility and reducing errors.
problem Medical image segmentation fails on out-of-distribution data due to variations in scanners and protocols.
method VarDeepPCA learns valid anatomical geometries using only small in-distribution datasets, providing uncertainty estimates.
result VarDeepPCA restores segmentation maps to OOD data, improving anatomical plausibility and reducing errors.
Efficient inference for multimodal Gaussian mixture models of interacting dynamical systems.
problem Efficient inference for multimodal distributions in stochastic dynamical systems.
method Graph neural networks with moment matching for sample-free inference and structured covariance approximations.
result Sample-free inference with improved efficiency and stability compared to Monte Carlo alternatives.
Improved uncertainty estimation in neural networks with VBLL.
problem Improving uncertainty estimation in neural networks.
method Deterministic variational formulation for training Bayesian last layer neural networks.
result Improves predictive accuracy, calibration, and out-of-distribution detection.
A new framework for lightweight BNNs learns heteroscedastic uncertainties efficiently.
problem Learning heteroscedastic uncertainties from BNNs for lightweight networks.
method Embedding heteroscedastic variances into BNN parameters and using moment propagation for inference.
result Improves predictive performance for lightweight BNNs without increasing parameter count.
Bayesian learning of model parameters in neural networks is important in scenarios where estimates with well-calibrated uncertainty are important. In this paper, we propose Bayesian quantized networks (BQNs), quantized neural networks (QNNs) for which we learn a posterior distribution over their discrete parameters. We…
New method for efficient probabilistic deep state-space models.
problem Efficient inference for probabilistic deep state-space models.
method Deterministic inference algorithm for ProDSSM with neural network weights.
result Superior balance between predictive performance and computational budget.
Paper presents a method to accurately quantify neural network uncertainty without sampling.
problem Uncertainty quantification in neural networks for reliability and robustness.
method Sample-free moment propagation technique for mean vectors and covariance matrices.
result Analytic solution for covariance of nonlinear activation functions.
Labeling training data is one of the most costly bottlenecks in developing machine learning-based applications. We present a first-of-its-kind study showing how existing knowledge resources from across an organization can be used as weak supervision in order to bring development time and cost down by an order of magnit…
Unified framework for self-supervised learning via latent distribution matching.
problem Lack of a unifying theoretical framework for diverse SSL methods.
method Casting SSL as latent distribution matching (LDM): maximizing alignment and uniformity.
result Derives a Bayesian filtering model and proves identifiable latent representations.
Real-time uncertainty estimation for computer vision tasks.
problem Real-time inference of uncertainty in deep learning models.
method Uncertainty-Aware Distribution Distillation method for fast inference.
result Significantly reduced inference time with improved uncertainty and predictive performance.
Proposes a new method to approximate Gaussian inference in classification tasks.
problem Uncertainty quantification in classification tasks using softmax functions.
method Develops a new formalism to approximate Gaussian distributions over logit space and proposes using element-wise normCDF or sigmoid instead of softmax.
result Improves uncertainty quantification compared to softmax Monte Carlo sampling.
Neural backdoor attack is emerging as a severe security threat to deep learning, while the capability of existing defense methods is limited, especially for complex backdoor triggers. In the work, we explore the space formed by the pixel values of all possible backdoor triggers. An original trigger used by an attacker …
New method uses TT approximations to solve HJB equations for efficient sampling.
problem Efficiently sampling from complex probability densities.
method Direct time integration of HJB equations using Tensor Train compression.
result Sample-free, dimensionality-avoiding integration method.
Proposes a new method to approximate Bayesian predictive uncertainty.
problem Bayesian uncertainty quantification in model predictions.
method Self-supervised learning approach to approximate posterior predictive distribution.
result SSLA and ASSLA outperform classical Laplace approximations in predictive calibration.
Proposes a new tensor decomposition method for functional temporal data with adaptive complexity.
problem Challenges in temporal tensor decomposition for general tensor data with continuous indexes.
method Encodes continuous spatial indexes as learnable Fourier features and uses neural ODEs for temporal trajectories. Introduces a sparsity-inducing prior for complexity adaptation.
result Significantly outperforms existing methods in prediction performance and robustness against noise.