New method calibrates deep networks by preserving top-k predictions.
problem Calibrated confidence scores for multi-class deep networks to avoid rare mistakes.
method Intra order-preserving functions combined with neural network architecture.
result Outperforms state-of-the-art methods in evaluation metrics.
In this paper we refine the process of computing calibration functions for a number of multiclass classification surrogate losses. Calibration functions are a powerful tool for easily converting bounds for the surrogate risk (which can be computed through well-known methods) into bounds for the true risk, the probabili…
New method improves calibration in multi-output probabilistic models.
problem Challenges in achieving multivariate calibration in multi-output regression.
method General regularization framework to enforce multivariate calibration during training for arbitrary pre-rank functions.
result Significant improvement in calibration across all pre-rank functions without sacrificing predictive accuracy.
New algorithms achieve decision calibration without sample complexity dependent on feature dimension.
problem Achieving decision calibration for nonlinear loss functions with polynomial sample complexity.
method Developed smooth relaxation of decision calibration, enabling dimension-free algorithms.
result Efficient algorithms post-process predictors to satisfy decision calibration without worsening accuracy.
In this paper we introduce and study the notion of plurisubharmonic functions in calibrated geometry. These functions generalize the classical plurisubharmonic functions from complex geometry and enjoy many of their important properties. Moreover, they exist in abundance whereas the corresponding pluriharmonics are gen…
The paper introduces a spline-based method for calibrating neural networks.
problem Ensuring neural network outputs are reliable for safety-critical applications.
method Approximating the empirical cumulative distribution function using splines to map network outputs to calibrated probabilities.
result The spline-based recalibration consistently outperforms existing methods on calibration measures.
Improves forecast calibration for extreme events using modified loss functions.
problem Improperly specified models do not issue calibrated forecasts for extreme events.
method Adapting loss functions based on weighted scoring rules and tail miscalibration regularization.
result Calibrated forecasts for extreme wind speeds can be improved by suitable adaptations to the loss function during model training.
We introduce and study the notion of plurisubharmonic functions in calibrated geometry. These functions generalize the classical plurisubharmonic functions from complex geometry and enjoy their important properties. Moreover, they exist in abundance whereas the corresponding pluriharmonics are generally quite scarce. A…
The paper provides a uniform convergence bound for smooth calibration error and its relationship with functional gradient.
problem Limited theoretical understanding of learning algorithms achieving high accuracy and good calibration.
method Focuses on smooth calibration error, providing a uniform convergence bound and proving the relationship with functional gradient.
result Derives conditions for simultaneous classification and calibration guarantees in gradient boosting trees, kernel boosting, and neural networks.
We study statistical calibration, i.e., adjusting features of a computational model that are not observable or controllable in its associated physical system. We focus on functional calibration, which arises in many manufacturing processes where the unobservable features, called calibration variables, are a function of…
Optimizing proper loss yields calibrated models under specific conditions.
problem Understanding when optimizing proper loss functions leads to calibrated predictions.
method Local optimality condition and Lipschitz functions.
result Predictors with local optimality are nearly calibrated and nearly locally optimal.
Unified framework for generalized Venn and Venn-Abers calibration for reliable prediction.
problem Asymptotic guarantees of popular distribution-free methods in model calibration.
method Unified framework extending Vovk's approach to generic loss functions, transforming predictors into set-valued predictions.
result Finite-sample set predictions shrink to a single conditionally calibrated prediction, capturing epistemic uncertainty.
Corrects mismatch in consistency of nuisance estimators for doubly robust methods.
problem Mismatch in consistency of nuisance estimators in doubly robust methods.
method Calibrated debiased machine learning (calibrated DML) with isotonic regression adjustment.
result Calibrated DML yields doubly robust asymptotic normality with slower convergence of nuisance estimators.
We use AD to compute gradients for complex functionals in stochastic model calibration.
problem Computing gradients for functions involving expectations in stochastic models.
method Automatic Adjoint Differentiation and parallelization.
result Faster and easier to implement approaches for gradient computation.
Extends DML for parametric problems, improving accuracy and efficiency in pricing and calibration.
problem Improving precision and efficiency in pricing and calibration for parametric problems.
method Exploits derivative information, uses adaptive parameter sampling, constructs pricing surrogates, and optimizes globally.
result Demonstrates improved accuracy and efficiency in pricing and calibration for complex models.
New method calibrates neural network uncertainty for medical images.
problem Uncalibrated probabilistic outputs from deep neural networks in medical diagnosis.
method Functional space variational inference for Bayesian neural networks.
result Better calibrated uncertainty estimates at lower computational cost.
DNAMite creates interpretable, calibrated survival analysis models.
problem Limited interpretability in survival analysis models, especially for healthcare applications.
method Feature discretization and kernel smoothing in embedding module for flexible shape functions.
result DNAMite produces calibrated shape functions interpretable as contributions to cumulative incidence function.
SVI and GP surrogates improve calibration of ABMs in epidemiology.
problem Calibrating stochastic ABMs in epidemiology is computationally expensive.
method Stein Variational Inference (SVI) with Gaussian process (GP) surrogates.
result SVI maintains comparable predictive accuracy and calibration effectiveness to MCMC.
The article compares neural networks and logistic regression for credit scoring and introduces a new probability calibration technique.
problem Improving credit scoring accuracy using machine learning techniques.
method Comparison of logistic regression and neural networks, feature importance assessment, temporal feature inclusion, and SURE probability calibration.
result Neural networks can slightly improve credit scoring performance, and SURE calibration technique enhances probability calibration.
Post-hoc calibration of neural networks using g-Layers proves theoretical justification.
problem Ensuring the confidence of neural network decisions in real-world applications.
method Proves theoretical justification for post-hoc calibration methods by adding g-Layers and minimizing NLL.
result Proves that adding g-Layers and minimizing NLL can lead to a calibrated network.
Recently the authors showed that there is a robust potential theory attached to any calibrated manifold (X,φ). In particular, on X there exist φ-plurisubharmonic functions, φ-convex domains, φ-convex boundaries, etc., all inter-related and having a number of good properties. In this paper we show that, in a strong sens…
This research improves deep neural network calibration using a new loss function.
problem Improving probability calibration in deep neural networks.
method Introduces Focal Calibration Loss (FCL) to minimize Euclidean norm and penalize calibration error.
result FCL achieves state-of-the-art performance in both calibration and accuracy metrics.
Variational characterization of calibrated submanifolds in different contexts.
problem Characterize calibrated submanifolds using variational principles.
method Variational approach with special variations of ambient metrics and calibrations.
result Critical points of volume functional correspond to calibrated submanifolds.
Applications such as weather forecasting and personalized medicine demand models that output calibrated probability estimates---those representative of the true likelihood of a prediction. Most models are not calibrated out of the box but are recalibrated by post-processing model outputs. We find in this work that popu…
Efficiently calibrates epidemiological models using Bayesian optimization.
problem Calibration of complex epidemiological models is computationally expensive and challenging.
method Graybox Bayesian optimization scheme leveraging Gaussian processes and functional structure of compartmental models.
result Proposed methods achieve efficient calibration and improved performance compared to existing schemes.
New framework for evaluating multiclass classifier calibration.
problem Ensuring classifiers are well-calibrated for trustworthy predictions.
method Utility Calibration framework that measures calibration error relative to a utility function.
result Unified and robust interpretation of existing calibration metrics.
New method calibrates asynchronous, error-prone covariates for longitudinal data.
problem Estimation biases and slow convergence in analyzing time-varying covariates with measurement error.
method Functional calibration approach based on functional principal component analysis.
result Asymptotically unbiased and consistent estimators for time-invariant coefficients; optimal convergence rate for time-varying coefficients.
This paper presents an algorithm for a complete and efficient calibration of the Heston stochastic volatility model. We express the calibration as a nonlinear least squares problem. We exploit a suitable representation of the Heston characteristic function and modify it to avoid discontinuities caused by branch switchi…
The study extends calibrated geometry to smooth maps and finds energy bounds.
problem Finding energy bounds for smooth maps between Riemannian manifolds.
method Generalizing calibrated submanifolds to smooth maps and applying to energy functional.
result Lower bounds to the energy of smooth maps in homotopy classes.
Leaky ReLU activations improve the calibration of Bayesian neural networks.
problem Bayesian neural networks struggle with mean-field variational inference for ReLU activations.
method Investigated the effect of activation functions on the calibration of Bayesian neural networks.
result Leaky ReLU activations lead to more Gaussian-like weight posteriors and lower expected calibration error.
Efficiently calibrates volatility models using Chebyshev Tensors.
problem Calibrating pricing models efficiently.
method Used Chebyshev Tensors to speed up calibration of the rough Bergomi volatility model.
result Chebyshev Tensors can calibrate the rough Bergomi volatility model 40,000 times more efficiently than brute-force methods.
New method calibrates local volatility models to marginal distributions.
problem Calibrating local volatility models to specific marginal distributions.
method Inspired by volatility interpolation, constructs time-homogeneous or continuous local volatility functions.
result Efficient numerical algorithms for constructing local volatility functions.
The paper investigates how calibrating propensity scores improves DML estimates of average treatment effects.
problem Improving the accuracy of DML estimates in finite samples.
method Propensity score calibration within the Double/debiased machine learning framework.
result Calibrating propensity scores reduces the root mean squared error of DML estimates of average treatment effects in finite samples.
Proposes efficient calibration method for LIBOR Market Model with stochastic volatility.
problem Calibrating LIBOR Market Model with stochastic volatility.
method Derives analytical gradient of swaptions prices for DDSVLMM and uses it for gradient-based optimization.
result Analytical gradient-based calibration is highly competitive and efficient for DDSVLMM.
There has been much recent interest in application of the pool-adjacent-violators (PAV) algorithm for the purpose of calibrating the probabilistic outputs of automatic pattern recognition and machine learning algorithms. Special cost functions, known as proper scoring rules form natural objective functions to judge the…
Study explores calibration properties in neural architectures.
problem Calibration issues in deep neural networks despite improved accuracy.
method Leverages Neural Architecture Search (NAS) to evaluate 117,702 neural networks.
result Identifies key architectural designs beneficial for calibration.
For Hitchin's generalised geometries we introduce and analyse the concept of a structured submanifold which encapsulates the classical notion of a calibrated submanifold. Under a suitable integrability condition on the ambient geometry, these generalised calibrated cycles minimise a functional occurring as D-brane ener…
SWIFT method speeds up Heston model calibration for European options.
problem Calibrating the Heston model for European options efficiently.
method Extends SWIFT method to Heston model, simplifying gradient computation.
result Extremely fast calibration, outperforming state-of-the-art methods.
MCP extends conformal prediction to vector-valued score functions without data splitting.
problem Fixed prediction set shapes in scalar score functions limit coverage guarantees.
method MCP uses a single optimization problem for prediction set design and calibration, eliminating data splitting.
result RemMCP and RelMCP achieve target coverage with smaller or comparable prediction set sizes, reducing variance.
We are concerned with obtaining well-calibrated output distributions from regression models. Such distributions allow us to quantify the uncertainty that the model has regarding the predicted target value. We introduce the novel concept of distribution calibration, and demonstrate its advantages over the existing defin…
Boosting trees can test necessary conditions for regression model calibration.
problem Testing calibration and auto-calibration in regression models.
method Using boosting trees to test calibration and auto-calibration.
result Boosting trees prove to be very powerful in testing calibration and auto-calibration in large insurance datasets.
In many classification problems it is desirable to output well-calibrated probabilities on the different classes. We propose a robust, non-parametric method of calibrating probabilities called SplineCalib that utilizes smoothing splines to determine a calibration function. We demonstrate how applying certain transforma…
Loss-calibrated EP improves Bayesian decision-making by focusing on utility-sensitive posterior approximations.
problem Bayesian decision-making under asymmetric utility functions.
method Loss-calibrated expectation propagation (Loss-EP) that tilts the posterior towards higher utility decisions.
result Loss-EP can capture useful information for decision-making under asymmetric penalties.
New method improves calibration of neural networks by targeting robust margins and local smoothness.
problem Poor calibration of neural networks, leading to unreliable confidence estimates.
method Intervene on training procedure by targeting robust margins and local smoothness.
result Improved out-of-sample calibration without sacrificing accuracy.
The paper calibrates shrinkage covariance estimators for spectral functionals in high dimensions.
problem Calibrating shrinkage covariance estimators for spectral functionals in high dimensions.
method Derives first-order null laws, distribution-free Davis-Kahan bands, and calibrated tests for spectral functionals under shrinkage.
result Calibrated tests and intervals for spectral functionals are provided, addressing the issue of estimation noise and shrinkage bias.
We extend a variational framework to estimate calibration errors for Lp divergences.
problem Ensuring predicted probabilities match observed class frequencies in machine learning.
method Extend variational framework to Lp divergences, separating over- and under-confidence. result Avoids overestimation and separates over- and under-confidence.
A new method calibrates value predictions in offline RL to improve reliability.
problem Difficulty in long-horizon value prediction in offline reinforcement learning.
method Bellman calibration, a weak reliability criterion, and Iterated Bellman Calibration.
result Finite-sample guarantees show that Bellman calibration error is controlled at nonparametric rates.
New calibration energy measures deviation from calibrated geometry, enabling mean curvature flow in infinite volumes.
problem Mean curvature flow in infinite volumes with finite energy.
method Introducing calibration energy and proving its dissipation identity for mean curvature flows.
result Every proper self-expander with finite calibration energy is a plane in all dimensions and codimensions.