Proposes isotonic recalibration for insurance pricing to ensure auto-calibration under low signal-to-noise ratio.
problem Ensuring auto-calibration in insurance pricing models to prevent cross-financing.
method Applies isotonic recalibration to regression models to achieve auto-calibration.
result Isotonically recalibrated regression functions have low complexity under low signal-to-noise ratio.
This paper introduces minimum-risk recalibration for probabilistic classifiers, improving their reliability and accuracy.
problem Improving the reliability and accuracy of probabilistic classifiers.
method Minimum-risk recalibration within the MSE decomposition framework, analyzing UMB method and label shift adaptation.
result The optimal number of bins for UMB scales with n1/3, resulting in a risk bound of approximately O(n−2/3). Framework improves classifier calibration under differential privacy for domain shift.
problem Improving classifier calibration under domain shift with privacy constraints.
method Differential privacy framework for adapting recalibration algorithms.
result Novel accuracy temperature scaling algorithm outperforms existing methods on private datasets.
The paper analyzes recalibration methods for binary classifiers under distribution shift.
problem Recalibrating binary classifiers to match a target prior probability.
method Analysis of distribution shift assumptions and proposal of new recalibration methods.
result QMM methods provide conservative results for risk weights functions.
Optimizes predictions by recalibrating online forecasts with minimal error.
problem Tackles the challenge of recalibrating online predictions to be more accurate.
method Uses an imbalanced extension of the Blackwell approachability reduction framework to achieve (ε,ε2)-recalibration. result Achieves (ε,ε2)-recalibration for Lipschitz proper losses in T≈ε−3 rounds. A new framework improves VaR recalibration by balancing reliance on imperfect volatility proxies.
problem How to balance reliance on imperfect volatility proxies in one-sided VaR recalibration.
method Proxy-reliance control framework that interpolates between constant-shift and proxy-scaled corrections.
result Lower or intermediate proxy reliance can outperform fully proxy-scaled recalibration in stressed left-tail VaR control.
New method calibrates machine learning models with theoretical guarantees.
problem Lack of theoretical guarantees for recalibration in multiclass classification.
method PAC-Bayes analysis for generalization error in calibration.
result First optimizable upper bound for generalization error in calibration.
New method recalibrates VaR for option books, reducing forecast errors.
problem Inaccurate VaR forecasts due to missing operational choices.
method Marking-aware sequential VaR recalibration targeting normalized book-level loss.
result Sequential VaR recalibration improves VaR performance across different markets and options.
This paper introduces a novel recalibration method for multivariate forecasts.
problem Multivariate calibration for potentially misspecified models.
method Local mappings between marginal probability integral transform values and observed space, using K-nearest neighbors or normalizing flows.
result Demonstrated effectiveness on currency exchange rate and childhood malnutrition data.
Proposes MCLLO for assessing and recalibrating multiclass probability predictions.
problem Limited multicategory recalibration methods for assessing and comparing model calibration.
method MCLLO recalibration method that assesses calibration without model access and is easy to interpret.
result MCLLO outperforms other methods in simulations and real-world case studies.
3D ConvNets improved with Project & Excite for medical imaging segmentation.
problem Improving segmentation performance in 3D medical imaging.
method Proposed Project & Excite (PE) modules for 3D F-CNNs, extending 2D recalibration methods.
result Project & Excite modules boost segmentation performance up to 0.3 in Dice Score.
This paper addresses recalibration issues in hedging callable assets, proposing a new risk-adjusted approach.
problem The mismatch between dynamic hedging theory and practice due to daily recalibration.
method Extends HVA model risk approach to callable assets, focusing on recalibration and model risks.
result Model risk reserves adjusted for exercise decisions may significantly exceed basic valuation differences.
New models capture dynamic derivatives pricing with efficient simulations.
problem Capturing dynamic features of derivatives' term structures.
method Machine learning techniques to store and efficiently simulate complex drift terms.
result First efficient dynamic term structure models.
We focus on two particular aspects of model risk: the inability of a chosen model to fit observed market prices at a given point in time (calibration error) and the model risk due to recalibration of model parameters (in contradiction to the model assumptions). In this context, we follow the approach of Glasserman and …
Study evaluates uncertainty in BP estimation from PPG signals under domain shift.
problem Uncertainty quantification in healthcare, especially for cuffless BP estimation.
method Compared deep ensembles, Monte Carlo dropout, and various recalibration techniques.
result Deep ensembles provide stronger robustness under domain shift.
The paper enhances representations to show left-orderability of certain 3-manifold groups.
problem Left-orderability of 3-manifold groups using enhanced representations.
method Recalibration of Calegari and Dunfield's flipping construction for $\mbox{Homeo}_+(S^1)$-representations.
result Branched covers of links are left-orderable, generalizing known results.
Proposes a method to generate multivariate prediction intervals for random forests.
problem Uncertainty estimates for iterative design of experiments with multiple correlated model outputs.
method Recalibrated bootstrap method for bagged models.
result Significantly decreases the number of iterations required for satisfactory candidate in sequential learning problems.
We study option pricing and hedging with uncertainty about a Black-Scholes reference model which is dynamically recalibrated to the market price of a liquidly traded vanilla option. For dynamic trading in the underlying asset and this vanilla option, delta-vega hedging is asymptotically optimal in the limit for small u…
Unified calibration metrics improve forecast sharpness and accuracy.
problem Improving the sharpness of probabilistic forecasts while maintaining calibration.
method Kernel-based calibration metrics that unify and generalize existing methods for classification and regression.
result Enhanced calibration, sharpness, and decision-making across various tasks.
This work evaluates and benchmarks calibration metrics for data-driven regression models.
problem Conflicting results from different calibration metrics make it hard to compare and interpret model performance.
method Systematically extracted and benchmarked 14 regression calibration metrics across various data types and recalibration methods.
result Many metrics disagree on the same recalibration result, highlighting the need for careful metric selection.
The paper introduces a spline-based method for calibrating neural networks.
problem Ensuring neural network outputs are reliable for safety-critical applications.
method Approximating the empirical cumulative distribution function using splines to map network outputs to calibrated probabilities.
result The spline-based recalibration consistently outperforms existing methods on calibration measures.
New method turns any regression model into a calibrated probabilistic model.
problem Calibration and sharpness of uncertainty estimates in regression models.
method Modular Conformal Calibration (MCC) framework.
result MCC algorithms achieve near-perfect calibration and improved sharpness.
The paper calibrates uncertainty in dropout variational inference models.
problem Miscalibration of model uncertainty in dropout variational inference.
method Logit scaling methods are extended to recalibrate model uncertainty.
result Logit scaling reduces miscalibration, improving reliability of predictions.
The role of uncertainty quantification (UQ) in deep learning has become crucial with growing use of predictive models in high-risk applications. Though a large class of methods exists for measuring deep uncertainties, in practice, the resulting estimates are found to be poorly calibrated, thus making it challenging to …
New method uses AI predictions as cheaper alternatives to expensive outcomes.
problem Using expensive outcomes for statistical inference.
method Recalibrated prediction-powered inference using machine learning techniques.
result Significant gains in effective sample size over existing PPI proposals.
Isotonic regression binning affects calibration statistics of machine learning models.
problem Isotonic regression binning introduces aleatoric uncertainty in calibration statistics.
method Calibration error statistics are recalibrated using isotonic regression, which produces stratified uncertainties.
result Stratified uncertainties lead to significant differences in bin-based calibration statistics.
Bayesian logistic regression improves clinical risk prediction models over time.
problem Improving clinical risk prediction models after deployment to adapt to temporal shifts.
method Bayesian logistic regression (BLR) and Markov variant (MarBLR) for online recalibration and revision of prediction models.
result BLR and MarBLR consistently outperform static models and other online revision methods, improving average AUC and calibration index.
Unsupervised recalibration (URC) is a general way to improve the accuracy of an already trained probabilistic classification or regression model upon encountering new data while deployed in the field. URC does not require any ground truth associated with the new field data. URC merely observes the model's predictions a…
Proposes h-calibration for improving miscalibrated probability outputs of neural networks.
problem Improving reliability of probability outputs from neural networks.
method Probabilistic learning framework for calibration, including a simple yet effective post-hoc algorithm.
result Significantly better performance than traditional methods, validated by experiments.
The analytical tractability of affine (short rate) models, such as the Vasicek and the Cox-Ingersoll-Ross models, has made them a popular choice for modelling the dynamics of interest rates. However, in order to account properly for the dynamics of real data, these models need to exhibit time-dependent or even stochast…
Framework monitors insurance pricing models for drift and recalibration.
problem Maintaining predictive performance of pricing models in evolving insurance portfolios.
method Formalizes deviance loss and Murphy's score, studies Gini score, develops monitoring framework.
result Framework guides decisions on refitting or recalibrating pricing models.
PPI uses proxy data to improve inference from limited labels across related tasks.
problem Statistical inference with limited labels across multiple related tasks.
method Prediction-powered inference framework that uses cross-task recalibration to improve power and accuracy.
result Cross-task recalibration can substantially reduce confidence interval widths when labels are scarce.
A new method for generating SPX and VIX risk scenarios using perturbed optimal transport.
problem Generating accurate risk estimates for SPX and VIX without full recalibration.
method A joint optimal transport calibration with perturbation methodology for sensitivities, combined with Skew Stickiness Ratio dynamics.
result The proposed method produces accurate risk estimates relative to full recalibration and is computationally faster.
New method calibrates uncertainty estimates for image classifiers without labeled data.
problem Uncertainty estimates for modern classifiers are unreliable without labeled calibration data.
method Calibrates uncertainty estimates using unlabeled examples for distribution shifts.
result Proposes a method that provides excellent uncertainty estimates under natural distribution shifts.
Paper proposes faster adaptation to distribution shifts in online settings.
problem Violation of exchangeability assumption in evolving data environments.
method Online conformal inference with retrospective adjustment.
result Faster adaptation to distributional shifts demonstrated through numerical studies.
Develops a method to make predictions more informative without sacrificing accuracy.
problem Tension between well-calibrated and bold predictions.
method Bayesian model selection-based approach for assessing calibration and maximizing boldness.
result Maximally emboldening predictions while maintaining desired level of calibration.
New approach calibrates predictions for better decision-making.
problem Achieving reliable predictions for multi-class problems is hard.
method Introduces decision calibration, a new approach to calibrate predictions.
result Designs a recalibration algorithm that makes predictions reliable for decision-making.
A framework assesses the trustworthiness of probabilistic classifiers using local calibration error.
problem Assessing the trustworthiness of probabilistic classifiers beyond traditional metrics.
method I-trustworthy framework linking local calibration to trustworthiness; Kernel Local Calibration Error (KLCE) method for hypothesis testing.
result The effectiveness of the proposed test statistic demonstrated through simulated and real-world datasets.
The study quantifies decision-making risks from suboptimal classifiers and proposes methods to reduce these risks.
problem Excess risk in decision-making from suboptimal probabilistic classifiers.
method Analytical expressions and upper/lower bounds for excess risk, calibration curve estimation, grouping loss estimator.
result Identifies regimes where recalibration alone or post-training is more effective.
New methods reduce bias in estimating calibration error.
problem Reducing bias in estimating calibration error.
method Synthesizing model outputs and using equal-mass bins.
result Two reliable calibration-error estimators found: debiased estimator and ECE_sweep.
ICP improves prediction intervals for continuous outcomes at lower computational cost.
problem Systematic bias in point predictions that undermines their use in decision-making.
method Develops Isotonic Conformal Prediction (ICP) framework to decouple calibration from prediction-set construction.
result SICP and TICP procedures match SC-CP coverage at lower computational cost.
Overconfidence and underconfidence in machine learning classifiers is measured by calibration: the degree to which the probabilities predicted for each class match the accuracy of the classifier on that prediction. How one measures calibration remains a challenge: expected calibration error, the most popular metric, ha…
Two methods improve calibration of probabilistic classifiers, especially for multi-class problems.
problem Improving calibration of probabilistic classifiers, especially for multi-class problems.
method Two techniques: reduced calibration and class-wise calibration.
result Class-wise reduced calibration algorithms reduce prediction and per-class calibration errors.
Applications such as weather forecasting and personalized medicine demand models that output calibrated probability estimates---those representative of the true likelihood of a prediction. Most models are not calibrated out of the box but are recalibrated by post-processing model outputs. We find in this work that popu…
Within the field of electromyography-based (EMG) gesture recognition, disparities exist between the offline accuracy reported in the literature and the real-time usability of a classifier. This gap mainly stems from two factors: 1) The absence of a controller, making the data collected dissimilar to actual control. 2) …
Optimal hedging strategies for exotic options using vanilla options.
problem Hedging exotic options with illiquid vanilla options.
method Simple approximations and variational techniques in a market model and stochastic volatility model framework.
result Optimal Delta and Vega hedging strategies can be computed easily.
We study the calibration of several state of the art neural machine translation(NMT) systems built on attention-based encoder-decoder models. For structured outputs like in NMT, calibration is important not just for reliable confidence with predictions, but also for proper functioning of beam-search inference. We show …
Improved probabilistic forecasts using behavioral transformations.
problem Improving accuracy and consistency of probabilistic asset price forecasts.
method Behavioral transformation of fundamental expectations to disentangle sentiment-induced biases.
result Substantial forecast gains across various models and risk-preferences.