Deep ensembles don't necessarily improve calibration in low data regimes.
problem Calibration issues in deep learning models, especially in low data regimes.
method Examination of data-augmentation, ensembling, and post-processing calibration methods.
result Standard ensembling techniques can lead to less calibrated models in low data regimes.
Proposes a new method to calibrate uncertainty estimates in deep learning models.
problem Poorly calibrated uncertainty estimates in deep learning models.
method Repurposes heteroscedastic regression as a surrogate for calibration.
result Eliminates the need for recalibration and regularizes the training process.
New method calibrates deep networks by preserving top-k predictions.
problem Calibrated confidence scores for multi-class deep networks to avoid rare mistakes.
method Intra order-preserving functions combined with neural network architecture.
result Outperforms state-of-the-art methods in evaluation metrics.
KCal calibrates deep networks by embedding logits in a metric space.
problem Overconfident predictions from DNNs, especially in high-risk applications.
method KCal learns a metric space on the penultimate-layer latent embedding and generates predictions using kernel density estimates.
result KCal provides a provable full calibration guarantee and consistently outperforms baselines.
This paper examines how to calibrate ensemble members for better prediction accuracy.
problem Improper calibration of deep neural networks leads to unreliable probability estimates.
method Theoretical analysis and empirical evaluation on CIFAR-100 dataset.
result Well-calibrated ensemble members do not guarantee a well-calibrated ensemble prediction, but a well-calibrated ensemble prediction cannot exceed the average performance of its members.
Improves model calibration for deep neural networks using proper scores.
problem Calibration errors in deep neural networks are often biased and inconsistent.
method Introduces proper calibration errors related to proper scores.
result Demonstrates the superiority of proper scores over common estimators.
Deep neural network improves Heston model calibration accuracy and speed.
problem Calibrating the Heston model with numerical stability issues.
method Gradient-based deep learning framework (DDN) to learn Heston model and its derivatives.
result DDN significantly outperforms non-differential neural networks in calibration accuracy and speed.
Efficiently calibrates volatility models using Chebyshev Tensors.
problem Calibrating pricing models efficiently.
method Used Chebyshev Tensors to speed up calibration of the rough Bergomi volatility model.
result Chebyshev Tensors can calibrate the rough Bergomi volatility model 40,000 times more efficiently than brute-force methods.
Recently, Deep Neural Networks (DNNs) have been achieving impressive results on wide range of tasks. However, they suffer from being well-calibrated. In decision-making applications, such as autonomous driving or medical diagnosing, the confidence of deep networks plays an important role to bring the trust and reliabil…
The paper calibrates the G2++ model using deep learning for interest rates.
problem Calibrating interest rate models with deep learning.
method Calibrated G2++ model using Neural Networks trained on covariances and correlations of Zero-Coupon and Forward rates.
result Deep learning calibration outperforms classic methods.
New method calibrates deep models for both in-distribution and out-of-distribution samples.
problem Ensuring calibration for deep models in safety-critical applications, especially in OOD regions.
method Geodesic distance and Gaussian kernel to calibrate deep models.
result Proposed KDF and KDN methods achieve well-calibrated posteriors for both in-distribution and out-of-distribution samples.
This work evaluates uncertainty in deep Gaussian processes.
problem Uncertainty quantification in deep Gaussian processes.
method Hierarchical deep Gaussian processes (DGPs) and Deep Sigma Point Processes (DSPPs) evaluated on regression and classification tasks.
result DSPPs provide strong in-distribution calibration but are less robust under distribution shift compared to ensembles.
New method calibrates neural network predictions for better reliability.
problem Improper probability estimates from deep networks leading to unreliable predictions.
method Proposes a constrained optimization approach for a monotonic calibration map.
result Achieves state-of-the-art performance across various datasets and models.
Proposes isotonic regression for calibrating Deep Cox models' survival probabilities.
problem Poor calibration of Deep Cox models' survival probabilities.
method Isotonic regression for post hoc calibration of Deep Cox models.
result Establishes favorable theoretical guarantees and demonstrates empirical effectiveness.
Post-hoc calibration improves uncertainty under domain shift.
problem Improving uncertainty calibration under domain shift.
method Apply perturbations to validation set before post-hoc calibration.
result Perturbation step results in better calibration under domain shift.
Annealing Double-Head calibrates deep neural networks during training.
problem Overestimation or underestimation of predictive confidence in deep neural networks.
method An additional calibration head and Annealing technique to dynamically scale logits.
result State-of-the-art model calibration performance achieved without post-processing.
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
Proposes a new calibration error estimator for deep neural networks.
problem Improves calibration of deep neural networks, especially for canonical calibration.
method Uses a Dirichlet kernel density estimate to create a low-bias, trainable calibration error estimator.
result Asymptotically converges to true Lp calibration error, enabling efficient estimation and mini-batch updates. Focal loss improves deep neural networks' accuracy and calibration.
problem Miscalibration in deep neural networks.
method Using focal loss and temperature scaling to improve model calibration.
result Focal loss leads to state-of-the-art calibrated models without sacrificing accuracy.
Mix-n-Match improves uncertainty calibration in deep learning.
problem Post-hoc calibration of machine learning classifiers.
method Ensemble and composition strategies to improve accuracy, efficiency, and expressive power.
result Mix-n-Match strategies achieve better data-efficiency and expressive power while maintaining classification accuracy.
This research improves deep neural network calibration using a new loss function.
problem Improving probability calibration in deep neural networks.
method Introduces Focal Calibration Loss (FCL) to minimize Euclidean norm and penalize calibration error.
result FCL achieves state-of-the-art performance in both calibration and accuracy metrics.
Deep learning calibrates a rough Heston model to match implied volatilities.
problem Calibrating the quadratic rough Heston model to match market implied volatilities.
method Multi-factor approximation and deep learning for efficient calibration.
result The model accurately reproduces SPX and VIX implied volatilities.
Excellent ranking power along with well calibrated probability estimates are needed in many classification tasks. In this paper, we introduce a technique, Calibrated Boosting-Forest that captures both. This novel technique is an ensemble of gradient boosting machines that can support both continuous and binary labels. …
CCAC calibrates DNN classifiers on OOD datasets by separating mis-classified samples.
problem Calibrating DNN classifiers on out-of-distribution datasets is challenging.
method CCAC introduces an auxiliary class to map DNN output to calibrated confidence, separating mis-classified from correctly classified samples.
result CCAC consistently outperforms prior methods on various DNN models, datasets, and applications.
Deep learning accelerates Heston model calibration.
problem Calibrating stochastic volatility models is computationally expensive.
method Differential Machine Learning (DML) technique to train neural networks on differentials of features and labels.
result DML reduces Heston model calibration time significantly.
New loss improves DNN calibration without sacrificing accuracy.
problem Overfitting and overconfident predictions in DNNs.
method Proposes a new loss function inspired by Bayes decision theory.
result Improves DNN calibration without compromising accuracy.
Proposes top-label calibration and M2B framework for multiclass to binary calibration.
problem Multiclass calibration and interpretation issues.
method Top-label calibration and M2B reduction framework.
result M2B + HB achieves lower calibration error than other methods.
A fast deep learning method for parallel MRI without calibration.
problem Calibration issues in parallel MRI reconstruction.
method Model-based deep learning, self-learning non-linear annihilation filters, Fourier domain pre-learning.
result Significantly faster than SLR methods (3 orders of magnitude), improved performance with spatial domain prior.
Algorithm constructs confidence sets for deep neural networks with PAC guarantees.
problem Ensuring reliable predictions for deep neural networks with high confidence.
method Combines calibrated prediction and learning theory bounds.
result Constructs PAC confidence sets for various deep models.
Time series foundation models are well-calibrated, improving over baseline models.
problem Calibration of time series foundation models for practical applications.
method Systematic evaluations of five time series foundation models and two baselines, assessing calibration, prediction heads, and long-term forecasting.
result Time series foundation models are consistently better calibrated than baseline models and do not show over- or under-confidence.
Unified detector calibration and simulation using MLE from generative models.
problem Combining detector calibration and simulation using traditional methods.
method Maximum likelihood estimation from conditional generative models.
result Prior-independent and non-Gaussian resolutions possible.
Deep learning calibrates HJM forward curves for commodity options pricing.
problem Calibrating HJM forward curves for accurate option pricing in commodity markets.
method Introduced a neural network to approximate true option prices from model parameters, calibrated using observed option prices.
result Neural network calibration yields high accuracy in recovering option prices, even with model parameter approximation loss.
The paper proposes a method to improve model calibration without explicitly measuring uncertainties.
problem Improving the quality of uncertainty estimators in deep neural networks.
method A novel algorithm that performs simultaneous interval estimation for different calibration levels, effectively refining mean estimates.
result The proposed approach consistently outperforms existing regularization strategies in deep regression models.
Proposes an accuracy-preserving calibration method for DNNs.
problem Calibration of deep neural networks (DNNs) to measure prediction reliability.
method Uses Concrete distribution on the probability simplex to calibrate DNNs without accuracy loss.
result The proposed method outperforms previous methods in accuracy-preserving calibration tasks.
Improves calibration of regression models without requiring additional data.
problem Poor calibration of regression models leading to unreliable predictions.
method Quantile regularizer based on cumulative KL divergence.
result Significantly improves calibration for regression models trained with Dropout VI and Deep Ensembles.
Proposes a new method for deep ensembles that improves accuracy and calibration.
problem Improving accuracy and calibration of deep ensembles.
method Estimates confusion matrices of ensemble members and weighs them according to their inferred performance.
result Empirically shows superiority of soft Dawid Skene over ensemble averaging.
The paper extends calibration to sets of probabilistic classifiers, finding many ensembles are poorly calibrated.
problem Evaluating the validity of epistemic uncertainty in sets of probabilistic classifiers.
method Proposed a novel nonparametric calibration test for sets of probabilistic classifiers.
result Ensembles of deep neural networks are often not well calibrated.
We construct a deep portfolio theory. By building on Markowitz's classic risk-return trade-off, we develop a self-contained four-step routine of encode, calibrate, validate and verify to formulate an automated and general portfolio selection process. At the heart of our algorithm are deep hierarchical compositions of p…
Techniques from deep learning play a more and more important role for the important task of calibration of financial models. The pioneering paper by Hernandez [Risk, 2017] was a catalyst for resurfacing interest in research in this area. In this paper we advocate an alternative (two-step) approach using deep learning t…
Study explores calibration properties in neural architectures.
problem Calibration issues in deep neural networks despite improved accuracy.
method Leverages Neural Architecture Search (NAS) to evaluate 117,702 neural networks.
result Identifies key architectural designs beneficial for calibration.
Efficient approach improves prediction calibration for domain shifts.
problem Improving uncertainty-aware predictions for domain shifts.
method Combining entropy-encouraging and adversarial calibration losses.
result Substantially outperforms existing approaches in domain drift calibration.
Improved DNN calibration without sacrificing accuracy.
problem Poor calibration of over-parametrized DNNs in safety-critical applications.
method Decoupling feature extraction and classification layers, and applying Gaussian priors.
result Significant improvement in model calibration with minimal training cost.
New analysis shows how to balance privacy and accuracy in deep learning.
problem Balancing privacy and accuracy in deep learning models.
method Continuous time analysis through neural tangent kernel (NTK) for arbitrary architectures.
result Large clipping norm improves calibration without sacrificing accuracy.
Deep Neural Networks (DNNs) have achieved state-of-the-art accuracy performance in many tasks. However, recent works have pointed out that the outputs provided by these models are not well-calibrated, seriously limiting their use in critical decision scenarios. In this work, we propose to use a decoupled Bayesian stage…
New framework calibrates computer models using deep learning and quantile regression.
problem Uncertainty in computer model input parameters due to high-dimensional time series data.
method Deep neural network with long-short term memory layers for inverse modeling, quantile regression for interval predictions.
result Accurate point and interval estimates for input parameters in WRF-hydro model.
Bayesian calibration improves ABMs for predicting travel patterns.
problem Calibrating ABMs for accurate travel pattern predictions.
method Gaussian Process emulator with deep learning dimensionality reduction for high-dimensional, non-stationary data.
result Improved accuracy in predicting travel patterns using traffic flow data.
OMADA improves uncertainty calibration for deep networks.
problem Challenges in reliably quantifying uncertainty for modern deep networks.
method On-Manifold Adversarial Data Augmentation (OMADA) using latent space adversarial attacks.
result OMADA consistently yields more accurate and better calibrated classifiers than baseline models.
New method calibrates neural network uncertainty for medical images.
problem Uncalibrated probabilistic outputs from deep neural networks in medical diagnosis.
method Functional space variational inference for Bayesian neural networks.
result Better calibrated uncertainty estimates at lower computational cost.