The paper calibrates uncertainty in dropout variational inference models.
problem Miscalibration of model uncertainty in dropout variational inference.
method Logit scaling methods are extended to recalibrate model uncertainty.
result Logit scaling reduces miscalibration, improving reliability of predictions.
The calibration of a measurement device is crucial for every scientific experiment, where a signal has to be inferred from data. We present CURE, the calibration uncertainty renormalized estimator, to reconstruct a signal and simultaneously the instrument's calibration from the same data without knowing the exact calib…
RoPE framework calibrates misspecified simulators for reliable inference.
problem Misspecification compromises reliability of simulation-based inference.
method Data-driven calibration using optimal transport and a small calibration set.
result RoPE framework improves inference accuracy and uncertainty calibration.
CP4SBI improves the calibration of credible sets in SBI models.
problem Inaccurate credible sets in SBI models lead to underestimation of true parameters.
method Develops a local conformal calibration framework for SBI models.
result Improves the quality of uncertainty quantification for neural posterior estimators.
New method calibrates neural SBI to avoid overconfident posteriors.
problem Overconfident posteriors in SBI due to inaccurate uncertainty quantification.
method Introduces a calibration term into neural model training objective, enabling end-to-end backpropagation.
result Achieves competitive or better coverage and posterior density than existing methods.
SVI and GP surrogates improve calibration of ABMs in epidemiology.
problem Calibrating stochastic ABMs in epidemiology is computationally expensive.
method Stein Variational Inference (SVI) with Gaussian process (GP) surrogates.
result SVI maintains comparable predictive accuracy and calibration effectiveness to MCMC.
Calibrated Prediction-Powered Inference improves semisupervised mean estimation by calibrating prediction scores.
problem Semisupervised mean estimation with a small labeled sample and a large unlabeled sample, and miscalibrated prediction models.
method Calibrated Prediction-Powered Inference (Calibeating) post-hoc calibrates the prediction score on the labeled sample before using it for semisupervised estimation.
result Calibrated Prediction-Powered Inference can improve the original score both as a predictor of the outcome and as a regression adjustment for semisupervised inference.
Proposes stabilized weights for causal inference using isotonic calibration.
problem Stability and bias issues in inverse propensity weighting.
method Post-hoc isotonic calibration of inverse propensity weights.
result Improves performance of doubly robust estimators of average treatment effect.
Bayesian calibration of black-box computer models offers an established framework to obtain a posterior distribution over model parameters. Traditional Bayesian calibration involves the emulation of the computer model and an additive model discrepancy term using Gaussian processes; inference is then carried out using M…
AECF improves multimodal inference robustness and calibration.
problem Robustness and calibration issues in multimodal systems with missing inputs.
method Adaptive Entropy-Gated Contrastive Fusion (AECF) layer.
result Improves masked-input mAP by +18 pp at a 50% drop rate.
Model uncertainty obtained by variational Bayesian inference with Monte Carlo dropout is prone to miscalibration. The uncertainty does not represent the model error well. In this paper, temperature scaling is extended to dropout variational inference to calibrate model uncertainty. Expected uncertainty calibration erro…
The paper addresses errors in online selective conformal prediction and proposes new strategies to ensure valid inference.
problem Online selective conformal prediction's exchangeability issues and false coverage rate control problems.
method Evaluation and correction of existing calibration selection strategies, proposing new ones that preserve exchangeability.
result Novel calibration selection strategies ensure both selection-conditional coverage and FCR control.
Calibrated PRMs improve inference efficiency for LLMs by dynamically adjusting compute budgets.
problem Poor calibration of PRMs leads to overestimation of success probabilities in partial reasoning steps.
method Quantile regression for calibration, instance-adaptive scaling (IAS) framework.
result Calibrated PRMs reduce inference costs while maintaining accuracy, especially on confident problems.
CRUDE calibrates regression uncertainty without assuming specific error distributions.
problem Uncalibrated uncertainty estimates in regression models, especially for modern predictive tasks.
method CRUDE assumes error distributions have a constant shape, shifted and scaled by predicted mean and standard deviation.
result CRUDE produces sharper, better calibrated, and more accurate uncertainty estimates than existing methods.
Bayesian method calibrates local volatility with Gaussian processes.
problem Calibrating local volatility models is challenging.
method Bayesian inference with Gaussian process priors.
result Rich probabilistic model of local volatility with uncertainty.
Post-calibration improves the accuracy of causal effect estimation.
problem Improperly calibrated propensity scores lead to inaccurate causal effect estimation.
method Performed a simulation study to assess the impact of post-calibration on causal effect estimation.
result Post-calibration reduces the error in estimating the average treatment effect, especially for expressive uncalibrated statistical estimators.
New method calibrates neural network uncertainty for medical images.
problem Uncalibrated probabilistic outputs from deep neural networks in medical diagnosis.
method Functional space variational inference for Bayesian neural networks.
result Better calibrated uncertainty estimates at lower computational cost.
Efficiently calibrates computationally expensive models using vine copulas.
problem Computational models are expensive and hard to calibrate with real data.
method Variational Bayes inference with vine copulas for dependent data.
result Computational scalability and efficiency of the proposed algorithm.
PBI inference may not be calibrated if predictive model is inaccurate.
problem Uncertainty quantification in PBI may be unreliable if the predictive model is not accurate.
method Predictive Bayesian inference with a forward predictive model.
result Posterior concentration depends on the predictive model, leading to potential calibration issues.
Posterior SBC validates inference conditionally on observed data.
problem Validating inference for specific observed data.
method Simulation-based calibration checking (SBC) adapted to use posterior parameters.
result Validates inference conditionally on observed data.
Researchers develop methods to calibrate ABMs using Bayesian techniques.
problem Challenges in calibrating ABMs due to likelihood intractability and non-differentiability.
method Generalised variational inference for differentiable ABMs.
result Accurate Bayesian parameter inferences for differentiable ABMs demonstrated.
SGPA calibrates transformer uncertainty for safety-critical tasks.
problem Uncertainty estimation in transformer models for safety-critical domains.
method Bayesian inference in transformer's output space using sparse Gaussian processes.
result SGPA-based Transformers improve in-distribution calibration and out-of-distribution robustness.
VMoER improves uncertainty quantification in MoE layers for scalable foundation models.
problem Uncertainty quantification in large-scale models like MoE layers.
method Structured Bayesian approach with amortized variational inference over routing logits and temperature parameter inference.
result Improves routing stability, reduces calibration error, and increases AUROC by 12%.
We propose a generic framework to calibrate accuracy and confidence of a prediction in deep neural networks through stochastic inferences. We interpret stochastic regularization using a Bayesian model, and analyze the relation between predictive uncertainty of networks and variance of the prediction scores obtained by …
New methods improve confidence set calibration in complex models.
problem Challenges in maintaining confidence set coverage in complex models.
method TRUST and TRUST++ methods using simulated data for calibration.
result Methods achieve distribution-free conditional coverage and robust inference.
A new method calibrates scientific models by adding randomness to their predictions.
problem Current scientific foundation models lack calibrated uncertainty.
method Stochastic Attention, which randomizes attention weights using multinomial samples.
result Stochastic Attention achieves the strongest native calibration and sharpest prediction intervals.
Improves robustness of propensity score estimators in challenging settings.
problem Limited overlap, small sample sizes, or unbalanced data.
method Extends calibration techniques for propensity score models, focusing on sample-splitting schemes.
result Calibration reduces variance and bias in inverse probability weighting and double/debiased machine learning frameworks.
This paper provides a method for noise-calibrated inference from DP synthetic data.
problem Inference from DP synthetic data is often miscalibrated and lacks principled uncertainty quantification.
method Release DP sufficient statistics, perform noise-calibrated likelihood-based inference, and optional synthetic data generation.
result Asymptotic normality and valid confidence intervals for the plug-in DP MLE.
Leaky ReLU activations improve the calibration of Bayesian neural networks.
problem Bayesian neural networks struggle with mean-field variational inference for ReLU activations.
method Investigated the effect of activation functions on the calibration of Bayesian neural networks.
result Leaky ReLU activations lead to more Gaussian-like weight posteriors and lower expected calibration error.
MEC improves efficiency and robustness in semi-supervised inference.
problem Efficient inference with limited labeled data and robust uncertainty quantification.
method Machine-Learning-Assisted Generalized Entropy Calibration (MEC) using cross-fitted, calibration-weighted PPI.
result MEC achieves semiparametric efficiency bounds under weaker assumptions and provides near-nominal coverage.
Loss-calibrated EP improves Bayesian decision-making by focusing on utility-sensitive posterior approximations.
problem Bayesian decision-making under asymmetric utility functions.
method Loss-calibrated expectation propagation (Loss-EP) that tilts the posterior towards higher utility decisions.
result Loss-EP can capture useful information for decision-making under asymmetric penalties.
Simplified matrix generator resolves credit migration model calibration issues.
problem Fundamental difficulties in calibrating Markovian credit migration models.
method Simplified matrix generator and elementary ideas from differential geometry.
result Risk-neutral calibration requires volatility information and is unstable.
Corrects mismatch in consistency of nuisance estimators for doubly robust methods.
problem Mismatch in consistency of nuisance estimators in doubly robust methods.
method Calibrated debiased machine learning (calibrated DML) with isotonic regression adjustment.
result Calibrated DML yields doubly robust asymptotic normality with slower convergence of nuisance estimators.
FMCPE improves SBI accuracy by correcting posterior estimators with flow matching.
problem Model misspecification in SBI leads to biased or overconfident posteriors.
method Flow Matching Corrected Posterior Estimation (FMCPE) trains a posterior approximator and corrects it using calibration samples.
result FMCPE consistently mitigates misspecification effects, improving inference accuracy and uncertainty quantification.
Inflationary flows use DBMs for accurate Bayesian inference.
problem Calibrated uncertainty quantification in Bayesian inference.
method Inflationary flows leverage DBMs to map data to a Gaussian latent space.
result Inflationary flows produce accurate, identifiable posterior distributions.
Framework disentangles deep feature uncertainty for efficient inference.
problem Inference-time uncertainty estimation for reliable decision-making.
method Uncertainty-Guided Inference-Time Selection framework.
result Significantly tighter prediction intervals and 60% compute reduction.
We propose a new scalable multi-class Gaussian process classification approach building on a novel modified softmax likelihood function. The new likelihood has two benefits: it leads to well-calibrated uncertainty estimates and allows for an efficient latent variable augmentation. The augmented model has the advantage …
The paper presents methods to improve uncertainty calibration in Bayesian Neural Networks.
problem Uncalibrated Bayesian Neural Networks often lead to overconfidence.
method The paper uses alpha-divergences from Information Geometry for calibration.
result Calibration using alpha-divergences provides better uncertainty estimates and is more efficient.
BCI provides calibrated prediction intervals for time series forecasts.
problem Calibration of prediction intervals for time series forecasts.
method BCI wraps around any time series forecasting models and optimizes interval lengths using dynamic programming.
result BCI achieves long-term coverage under arbitrary distribution shifts and temporal dependence.
New method calibrates ABMs using graph neural networks for microdata.
problem Calibrating ABMs to granular microdata with high-dimensional learning tasks.
method Temporal graph neural networks for learning parameter posteriors.
result Graph neural networks offer inductive biases for Bayesian inference with ABM microstates.
TCP provides well-calibrated prediction intervals for nonstationary time series.
problem Nonstationary time series forecasting with well-calibrated prediction intervals.
method Temporal Conformal Prediction (TCP) couples a modern quantile forecaster with a rolling split-conformal calibration layer.
result TCP achieves near-nominal coverage, providing slightly wider intervals than Historical Simulation.
Self-calibrating conformal prediction improves interval efficiency and offers a practical alternative.
problem Improving the reliability and uncertainty quantification of machine learning predictions.
method Combines Venn-Abers calibration and conformal prediction for binary and regression problems.
result Improves interval efficiency through model calibration and offers practical alternatives.
Bayesian decision theory outlines a rigorous framework for making optimal decisions based on maximizing expected utility over a model posterior. However, practitioners often do not have access to the full posterior and resort to approximate inference strategies. In such cases, taking the eventual decision-making task i…
Improved Bayesian FL method calibrates predictions for federated learning.
problem Overconfident predictions in Bayesian FL methods for federated learning.
method β-Predictive Bayes algorithm interpolates between mixture and product of local predictive posteriors, tuning parameter β for better calibration.
result Demonstrated superior calibration compared to other baselines, even with increased data heterogeneity.
Bayesian inference was once a gold standard for learning with neural networks, providing accurate full predictive distributions and well calibrated uncertainty. However, scaling Bayesian inference techniques to deep neural networks is challenging due to the high dimensionality of the parameter space. In this paper, we …
Misspecification-Aware Simulation-Based Inference via Side-Channel Guidance
problem Simulation-based inference (SBI) of latent parameters is hindered by simulator misspecification.
method Misspecification-Aware Simulation-Based Inference (MA-SBI) turns side-channel text into a posterior correction.
result MA-SBI matches the oracle posterior across 10 seeds and two backbones.
Deep neural networks (DNNs) are poorly calibrated when trained in conventional ways. To improve confidence calibration of DNNs, we propose a novel training method, distance-based learning from errors (DBLE). DBLE bases its confidence estimation on distances in the representation space. In DBLE, we first adapt prototypi…
New methods for scoring function decomposition improve forecast evaluation.
problem Improving forecast evaluation and understanding forecast components.
method Linear recalibration of forecasts for miscalibration, discrimination, and uncertainty.
result Enhanced statistical power and deeper insights into forecast components.