Combining ensembles and data augmentation harms model calibration.
problem Improving model calibration and robustness with ensembles and data augmentation leads to a trade-off.
method Combining ensemble averaging and data augmentation techniques.
result Combining ensembles and data augmentation can harm model calibration.
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
The article compares neural networks and logistic regression for credit scoring and introduces a new probability calibration technique.
problem Improving credit scoring accuracy using machine learning techniques.
method Comparison of logistic regression and neural networks, feature importance assessment, temporal feature inclusion, and SURE probability calibration.
result Neural networks can slightly improve credit scoring performance, and SURE calibration technique enhances probability calibration.
Mixup technique improved, reducing manifold mismatch for better calibration.
problem Improving calibration of models using Mixup.
method Dynamic adjustment of interpolation coefficients based on sample similarity.
result Improved predictive performance and calibration with reduced manifold mismatch.
Techniques from deep learning play a more and more important role for the important task of calibration of financial models. The pioneering paper by Hernandez [Risk, 2017] was a catalyst for resurfacing interest in research in this area. In this paper we advocate an alternative (two-step) approach using deep learning t…
We provide an introduction to the theory of calibrated submanifolds through the key examples related with special holonomy. We focus on calibrated geometry in Calabi-Yau, G2 and Spin(7) manifolds, and describe fundamental results and techniques in the field.
Deep learning accelerates Heston model calibration.
problem Calibrating stochastic volatility models is computationally expensive.
method Differential Machine Learning (DML) technique to train neural networks on differentials of features and labels.
result DML reduces Heston model calibration time significantly.
Deep ensembles don't necessarily improve calibration in low data regimes.
problem Calibration issues in deep learning models, especially in low data regimes.
method Examination of data-augmentation, ensembling, and post-processing calibration methods.
result Standard ensembling techniques can lead to less calibrated models in low data regimes.
Inertial confinement fusion (ICF) experiments are designed using computer simulations that are approximations of reality, and therefore must be calibrated to accurately predict experimental observations. In this work, we propose a novel nonlinear technique for calibrating from simulations to experiments, or from low fi…
Excellent ranking power along with well calibrated probability estimates are needed in many classification tasks. In this paper, we introduce a technique, Calibrated Boosting-Forest that captures both. This novel technique is an ensemble of gradient boosting machines that can support both continuous and binary labels. …
A new method for CT using graph-based regularization.
problem Transfer calibrations between instruments without suitable transfer standards.
method Employing manifold regularization of PLS objective to enforce invariant projections in latent variable space.
result Implicit removal of inter-device variation in predictive directions.
In order to cope with the increased data volumes generated by modern radio interferometers such as LOFAR (Low Frequency Array) or SKA (Square Kilometre Array), fast and efficient calibration algorithms are essential. Traditional radio interferometric calibration is performed using nonlinear optimization techniques such…
Survey of methods to calibrate neural network predictions.
problem Ensuring neural networks provide accurate confidence levels.
method Empirical comparison of calibration methods.
result Various techniques for calibrating neural networks.
Annealing Double-Head calibrates deep neural networks during training.
problem Overestimation or underestimation of predictive confidence in deep neural networks.
method An additional calibration head and Annealing technique to dynamically scale logits.
result State-of-the-art model calibration performance achieved without post-processing.
New method calibrates models under covariate shifts.
problem Calibration of models can be lost under covariate shifts.
method Importance sampling based approach.
result Efficacy demonstrated on real-world and synthetic datasets.
Bayesian calibration speeds up ABM for pandemic modeling.
problem Calibrating stochastic ABMs for accurate pandemic predictions is computationally intensive.
method Random forest surrogate modeling for accelerated ABM evaluation.
result Improved predictive performance with random forest calibration compared to previous methods.
We propose a novel and generic calibration technique for four-factor foreign-exchange hybrid local-stochastic volatility models with stochastic short rates. We build upon the particle method introduced by Guyon and Labordère [Nonlinear Option Pricing, Chapter 11, Chapman and Hall, 2013] and combine it with new variance…
A post-hoc framework improves model performance by calibrating different feature spaces.
problem Improving AUC performance on binary classification tasks for overconfident models.
method Identifies heterogeneous partitions of the feature space and applies post-hoc calibration techniques to each partition.
result Theoretical optimality of the framework for any model, demonstrated on deep neural networks.
Proposes a neural network for calibrating stochastic volatility models.
problem Calibrating stochastic volatility models with robustness and efficiency.
method Combines grid approach with pointwise two-stage calibration, using random grids for training.
result Validates the approach with empirical and Monte Carlo experiments for rough Bergomi and Heston models.
This work addresses building fair and calibrated models.
problem Building models that are both fair and calibrated.
method Developed a new definition of fairness and showed that group-wise calibration results in fairness. Proposed post-processing techniques and modifications of calibration losses.
result Demonstrated that ensuring group-wise calibration results in a fair model under the new definition of fairness.
Meta-Cal improves post-hoc calibration of neural networks.
problem Improving the accuracy of uncalibrated neural network predictions.
method Meta-Cal uses a base calibrator and a ranking model with constraints to provide high-probability bounds.
result Meta-Cal significantly outperforms existing methods in post-hoc multi-class classification calibration.
The paper proposes a method to calibrate healthcare AI models for reliability and interpretability.
problem Characterizing model reliability and enabling introspection of model behavior in clinical decision making.
method A calibration-driven learning method combined with interpretability techniques based on counterfactual reasoning.
result Demonstrates the effectiveness of the proposed approach using a lesion classification problem with dermoscopy images.
This paper enhances uplift modeling for multi-treatment marketing campaigns.
problem Optimizing marketing strategies by selecting individuals likely to respond to different treatments.
method Leveraging score ranking and calibration techniques.
result Improves overall performance of marketing campaigns.
The paper addresses probability calibration for incomplete sequences.
problem Improving probability estimates from incomplete sequences.
method Adapting traditional calibration techniques to sequences of varying lengths.
result Proposed methods improve probability calibration for modern sequential models.
In classification applications, we often want probabilistic predictions to reflect confidence or uncertainty. Dropout, a commonly used training technique, has recently been linked to Bayesian inference, yielding an efficient way to quantify uncertainty in neural network models. However, as previously demonstrated, conf…
We prove that tangent cones to 2-dimensional calibrated cycles are unique. Using this result we prove a rate of convergence for the mass of the blow-up of a calibrated integral 2-cycle towards the limiting density. With the same techniques, we can also prove such a rate for J-holomorphic maps between almost complex man…
Regime switching volatility models provide a tractable method of modelling stochastic volatility. Currently the most popular method of regime switching calibration is the Hamilton filter. We propose using the Baum-Welch algorithm, an established technique from Engineering, to calibrate regime switching models instead. …
Bayesian calibration of black-box computer models offers an established framework to obtain a posterior distribution over model parameters. Traditional Bayesian calibration involves the emulation of the computer model and an additive model discrepancy term using Gaussian processes; inference is then carried out using M…
The paper explores using machine learning for yield curve calibration in multiple markets.
problem Calibration challenges in multiple yield curve markets.
method Gaussian process regression and Adam optimizer.
result Good results for single curve markets, but many challenges for multi curve markets.
`Gluing' is a technique of constructing solutions to non-linear (elliptic) partial differential equations such as Yang--Mills equations, minimal surface equations and Einstein equations. Calibrated submanifolds are a certain class of minimal surfaces, and there are various examples of them constructed by the gluing tec…
Computer model calibration typically operates by choosing parameter values in a computer model so that the model output faithfully predicts reality. By using performance targets in place of observed data, we show that calibration techniques can be repurposed to wed engineering and material design, two processes that ar…
The calibration of a measurement device is crucial for every scientific experiment, where a signal has to be inferred from data. We present CURE, the calibration uncertainty renormalized estimator, to reconstruct a signal and simultaneously the instrument's calibration from the same data without knowing the exact calib…
A new method improves SNPE for intractable likelihood models.
problem Simulation-based models with intractable likelihoods.
method Adaptive calibration kernel and variance reduction techniques.
result The proposed method provides a better approximation of the posterior.
This paper is devoted to the application of B-splines to volatility modeling, specifically the calibration of the leverage function in stochastic local volatility models and the parameterization of an arbitrage-free implied volatility surface calibrated to sparse option data. We use an extension of classical B-splines …
We use AD to compute gradients for complex functionals in stochastic model calibration.
problem Computing gradients for functions involving expectations in stochastic models.
method Automatic Adjoint Differentiation and parallelization.
result Faster and easier to implement approaches for gradient computation.
CRUDE calibrates regression uncertainty without assuming specific error distributions.
problem Uncalibrated uncertainty estimates in regression models, especially for modern predictive tasks.
method CRUDE assumes error distributions have a constant shape, shifted and scaled by predicted mean and standard deviation.
result CRUDE produces sharper, better calibrated, and more accurate uncertainty estimates than existing methods.
New method calibrates deep networks by preserving top-k predictions.
problem Calibrated confidence scores for multi-class deep networks to avoid rare mistakes.
method Intra order-preserving functions combined with neural network architecture.
result Outperforms state-of-the-art methods in evaluation metrics.
SVI and GP surrogates improve calibration of ABMs in epidemiology.
problem Calibrating stochastic ABMs in epidemiology is computationally expensive.
method Stein Variational Inference (SVI) with Gaussian process (GP) surrogates.
result SVI maintains comparable predictive accuracy and calibration effectiveness to MCMC.
We study statistical calibration, i.e., adjusting features of a computational model that are not observable or controllable in its associated physical system. We focus on functional calibration, which arises in many manufacturing processes where the unobservable features, called calibration variables, are a function of…
Investigates how multivariate Lévy models affect calibration and pricing.
problem How multivariate Lévy models affect calibration and pricing.
method Calibration methods of Luciano and Semeraro (2010) and Ballotta and Bonfiglioli (2016) are studied.
result Models can fit market data and price exotic derivatives with rich dependence structures.
This work improves neural network calibration using explicit regularization.
problem Improving predictive uncertainty in neural networks.
method Introducing a probabilistic calibration measure and exploring explicit regularization techniques.
result Explicit regularization improves log-likelihood and predictive uncertainty.
SGPA calibrates transformer uncertainty for safety-critical tasks.
problem Uncertainty estimation in transformer models for safety-critical domains.
method Bayesian inference in transformer's output space using sparse Gaussian processes.
result SGPA-based Transformers improve in-distribution calibration and out-of-distribution robustness.
Online Platt Scaling adapts to varying data distributions.
problem Adapting Platt scaling to non-i.i.d. settings with distribution drift.
method Combines Platt scaling with online logistic regression and calibeating.
result OPS+calibeating method is guaranteed to be calibrated for adversarial outcomes.
Applications such as weather forecasting and personalized medicine demand models that output calibrated probability estimates---those representative of the true likelihood of a prediction. Most models are not calibrated out of the box but are recalibrated by post-processing model outputs. We find in this work that popu…
Calibrates historical and implied correlations in energy markets.
problem Challenges in aligning historical correlations of futures contracts with implied volatility smiles.
method Multiplicative multi-factor Heath-Jarrow-Morton model combined with stochastic volatility from lifted Heston model, using Kemna-Vorst approximation and Fourier-based techniques.
result Remarkable joint historical and implied calibration fits on the German power market.
Unified detector calibration and simulation using MLE from generative models.
problem Combining detector calibration and simulation using traditional methods.
method Maximum likelihood estimation from conditional generative models.
result Prior-independent and non-Gaussian resolutions possible.
Improves robustness of propensity score estimators in challenging settings.
problem Limited overlap, small sample sizes, or unbalanced data.
method Extends calibration techniques for propensity score models, focusing on sample-splitting schemes.
result Calibration reduces variance and bias in inverse probability weighting and double/debiased machine learning frameworks.
Framework calibrates ML models for risk control in various tasks.
problem Achieving statistical guarantees for model predictions.
method Reframing risk control as multiple hypothesis testing, applying statistical techniques.
result New calibration methods for multi-label classification, instance segmentation, outlier detection, and confidence set coverage.