New algorithm samples Bayesian neural networks for improved calibration.
problem Improving calibration of Bayesian neural networks.
method Symmetric Minibatch Splitting-UBU (SMS-UBU) algorithm.
result SMS-UBU provides better calibration performance than standard methods.
The paper solves Bernstein problems for specific submanifolds in high-dimensional spaces.
problem Bernstein problem for smooth maps to lower dimensions forming calibrated submanifolds.
method Established conditions for maps to be affine based on the slope's second elementary symmetric polynomial.
result Conditions ensuring maps are affine for coassociative and Cayley submanifolds in R7 and R8. This paper aims to provide a better understanding of a symmetric loss. First, we emphasize that using a symmetric loss is advantageous in the balanced error rate (BER) minimization and area under the receiver operating characteristic curve (AUC) maximization from corrupted labels. Second, we prove general theoretical p…
CP4SBI improves the calibration of credible sets in SBI models.
problem Inaccurate credible sets in SBI models lead to underestimation of true parameters.
method Develops a local conformal calibration framework for SBI models.
result Improves the quality of uncertainty quantification for neural posterior estimators.
SGPA calibrates transformer uncertainty for safety-critical tasks.
problem Uncertainty estimation in transformer models for safety-critical domains.
method Bayesian inference in transformer's output space using sparse Gaussian processes.
result SGPA-based Transformers improve in-distribution calibration and out-of-distribution robustness.
Paper develops new conformal prediction methods for sum or average of unknown labels.
problem Uncertainty quantification in joint distributions of random variables.
method Introduces novel conformal prediction methods for sum or average of unknown labels.
result Validates the proposed method for sum or average of unknown labels under permutation invariant assumptions.
We reconsider the multivariate Kyle model in a risk-neutral setting with a single, perfectly informed rational insider and a rational competitive market maker, setting the price of n correlated securities. We prove the unicity of a symmetric, positive definite solution for the impact matrix and provide insights on its …
Noise titration benchmarks time series forecasting models rigorously.
problem Evaluation of time series forecasting models is often flawed due to lack of interventionist methods.
method Interventionist benchmarking using Gaussian noise titration of dynamical systems.
result Fern model outperforms state-of-the-art models in non-stationary conditions.
A new method for designing accurate emulators using deep learning with interval calibration.
problem Designing accurate emulators for scientific processes with modern machine learning methods.
method Learn-by-Calibrating (LbC) approach based on interval calibration.
result Significant improvements in generalization error over widely-used loss functions.
Study energy-minimizing maps in projective spaces, proving sharp bounds.
problem Finding optimal mappings in projective spaces.
method Proving lower bounds and characterizing energy-minimizing maps.
result Sharp lower bounds and characterization of energy-minimizing maps.
Let D be a closed disk centered at the origin in the horizontal hyperplane {t=0} of the sub-Riemannian Heisenberg group $\hh^n$, and C the vertical cylinder over D. We prove that any finite perimeter set E such that D⊂E⊂C has perimeter larger than or equal to the one of the rotationally symm…
Label smoothing improves model performance even with noisy labels.
problem Mitigating label noise in deep learning models.
method Examined label smoothing as a technique to cope with label noise and compared it to loss-correction methods.
result Label smoothing is competitive with loss-correction techniques under label noise and beneficial for distillation from noisy data.
New model accounts for scale variation and noise in pairwise comparisons.
problem Nonreciprocal pairwise comparisons in decision analysis.
method Additive model with structured matrix and random perturbation.
result Explicit estimators and probability assessments of admissible ranking regions.
Optimizes data splitting for shorter conformal prediction intervals.
problem Minimizing prediction interval length while maintaining coverage.
method Theoretical framework for optimal data splitting in split conformal prediction.
result Analytical characterizations of length-optimal split ratios in various settings.
The paper shows over-confidence in models isn't just due to over-parametrization.
problem Over-confidence in machine learning models, especially in binary classification.
method Theoretical analysis of logistic regression and other binary classification problems.
result Logistic regression is inherently over-confident in certain settings, but over-confidence is not always the case.
Proposes new method for calibrating treatment effect predictors.
problem Calibrating predictors of heterogeneous treatment effects.
method Causal isotonic calibration and cross-calibration.
result Achieves fast calibration rates under weak conditions.
Proposes top-label calibration and M2B framework for multiclass to binary calibration.
problem Multiclass calibration and interpretation issues.
method Top-label calibration and M2B reduction framework.
result M2B + HB achieves lower calibration error than other methods.
Study explores calibration properties in neural architectures.
problem Calibration issues in deep neural networks despite improved accuracy.
method Leverages Neural Architecture Search (NAS) to evaluate 117,702 neural networks.
result Identifies key architectural designs beneficial for calibration.
The paper analyzes the joint dynamics of prices and order flow in electronic order books.
problem Understanding the micro-dynamics of asset prices in high-frequency trading environments.
method Double coarse-graining procedure and Principal Component Analysis to extract meaningful information.
result The VAR model captures the stability of liquidity modes and their dynamical evolution.
A new perfectly truthful calibration measure improves prediction reliability.
problem Improving the reliability of predictions by ensuring they are conditionally unbiased.
method Designing a simple, perfectly truthful calibration measure called ATB.
result ATB is the first perfectly truthful calibration measure in the batch setting.
New truthful calibration errors improve model ranking in multiclass prediction.
problem Non-truthful calibration errors can mislead model comparisons.
method Introduced perfectly truthful calibration errors for multiclass predictions.
result Truthful calibration errors preserve decision-theoretic dominance and stabilize model rankings.
New framework for evaluating multiclass classifier calibration.
problem Ensuring classifiers are well-calibrated for trustworthy predictions.
method Utility Calibration framework that measures calibration error relative to a utility function.
result Unified and robust interpretation of existing calibration metrics.
We propose a new framework to improve the calibration of neural networks.
problem Improving the accuracy of model confidence predictions.
method Introducing a differentiable surrogate for expected calibration error (DECE) and a meta-learning framework to optimise model hyper-parameters for validation set calibration.
result Achieved competitive performance with existing calibration approaches.
Certified calibration methods protect model confidence from adversarial attacks.
problem Adversarial attacks degrade model calibration, reducing confidence in predictions.
method Developed certified calibration methods to provide worst-case bounds on calibration under adversarial perturbations.
result Certified calibration methods produce analytic and approximate bounds for the Brier score and expected calibration error.
Meta-Cal improves post-hoc calibration of neural networks.
problem Improving the accuracy of uncalibrated neural network predictions.
method Meta-Cal uses a base calibrator and a ranking model with constraints to provide high-probability bounds.
result Meta-Cal significantly outperforms existing methods in post-hoc multi-class classification calibration.
Post-processing predictors reduces calibration errors for decision-making.
problem Predictors with low calibration error for machine learning may have high error for decision-making.
method Post-processing with ε distance to calibration adds noise to make predictions differentially private.
result Post-processing achieves O(√ε) ECE and CDL, asymptotically optimal.
A new calibration metric bridges testability and actionability.
problem Combining testability and actionable insights for forecast probabilities.
method Cutoff Calibration Error (CCE) that assesses calibration over intervals of forecasted probabilities.
result Cutoff Calibration Error is both testable and actionable.
New study on neural network calibration, linking it to generalization gap.
problem Neural networks lack strong guarantees on calibration.
method Decomposed calibration error into train set and generalization gap.
result Models with small generalization gap are well-calibrated.
A new method for multiclass calibration using vector quantization.
problem Challenges in multiclass calibration, especially in high-stakes settings.
method Compositional approach via Vector Quantization (VQ) to learn region-specific calibration maps.
result Significant improvements in local calibration with competitive global calibration and predictive performance.
This post introduces model calibration and evaluation measures, highlighting issues with a common measure.
problem Ensuring model confidence accurately reflects true outcomes.
method Explains common calibration definition, ECE, and its drawbacks.
result New evaluation measures needed for comprehensive model calibration.
Unified calibration metrics improve forecast sharpness and accuracy.
problem Improving the sharpness of probabilistic forecasts while maintaining calibration.
method Kernel-based calibration metrics that unify and generalize existing methods for classification and regression.
result Enhanced calibration, sharpness, and decision-making across various tasks.
New decision-theoretic calibration error metric improves prediction reliability.
problem Improving the reliability of predictions for decision-making.
method Proposed Calibration Decision Loss (CDL) and an efficient algorithm to achieve near-optimal CDL.
result Near-optimal CDL guarantees vanishing payoff loss from miscalibration.
Cone structures over minimal products can't be calibrated smoothly.
problem Calibrating cones over minimal products with smooth calibrations.
method Extending a key result from [Zha26], showing obstruction.
result Cone structures over minimal products cannot be calibrated by smooth calibrations.
Post-hoc calibration improves uncertainty under domain shift.
problem Improving uncertainty calibration under domain shift.
method Apply perturbations to validation set before post-hoc calibration.
result Perturbation step results in better calibration under domain shift.
Smooth calibration improves forecast reliability even with leaked information.
problem Improving forecast reliability with leaked information.
method Combining nearby forecasts to ensure smooth calibration, which can be guaranteed by deterministic procedures.
result Smooth calibration can be guaranteed by deterministic procedures even with leaked forecasts, and it yields uncoupled finite-memory dynamics in games.
Planes are the only calibrated submanifolds with flat normal bundles.
problem Characterizing submanifolds with specific geometric properties.
method Using constant-coefficient differential forms and parallel calibrations.
result Calibrated submanifolds with flat normal bundles are planes.
We describe a family of calibrations arising naturally on a hyperkähler manifold M. These calibrations calibrate the holomorphic Lagrangian, holomorphic isotropic and holomorphic coisotropic subvarieties. When M is an HKT (hyperkaehler with torsion) manifold with holonomy SL(n,H), we construct another fam…
Simple proof shows forecasts can be calibrated in a few periods.
problem Ensuring forecasts are calibrated over multiple periods.
method Uses minimax theorem to prove existence and guarantees calibration error.
result Calibration can be achieved in N3 periods with error at most 1/N. Survey of methods to calibrate neural network predictions.
problem Ensuring neural networks provide accurate confidence levels.
method Empirical comparison of calibration methods.
result Various techniques for calibrating neural networks.
This paper rethinks confidence calibration under covariate shifts.
problem Calibration methods struggle with covariate shifts and unstable importance weighting.
method Derives Expectation consistency condition and proposes Expectation consistency loss (ECL).
result ECL loss is compatible with various types of calibration and has the same sample complexity as ECE.
New method calibrates neural network predictions for better reliability.
problem Improper probability estimates from deep networks leading to unreliable predictions.
method Proposes a constrained optimization approach for a monotonic calibration map.
result Achieves state-of-the-art performance across various datasets and models.
New conditions for calibrated submanifolds in Riemannian geometry.
problem Characterizing calibrated submanifolds with extrinsic geometry.
method Introducing compliancy condition and analyzing extrinsic geometry.
result Conditions for extrinsic geometry of calibrated submanifolds.
Survey of Optimal Transport for model calibration.
problem Model calibration using Optimal Transport.
method General framework and numerical algorithms for various models.
result Calibration of volatility models and path-dependent options.
Develops confidence intervals for ECE, a measure of model calibration.
problem Ensuring the calibration of probabilistic predictions in machine learning models.
method Develops confidence intervals for the ℓ2 Expected Calibration Error (ECE), considering top-1-to-k calibration. result Shows asymptotic normality and different convergence rates for calibrated and miscalibrated models, developing methods to construct valid confidence intervals.
The paper addresses poor calibration in fine-tuned LLMs after preference alignment.
problem Poor calibration in fine-tuned Large Language Models (LLMs) after preference alignment.
method Proposes a calibration-aware fine-tuning approach to restore calibration without compromising model performance.
result Demonstrates the effectiveness of the proposed methods through extensive experiments.
Calibration in 16D disproves Federer's product question.
problem Proving Federer's product question in 16D.
method Adapted Cayley calibration in R^16, constructing a specific calibration Φ.
result Product of two orthogonally supported calibrations is not always a calibration.
New calibration bands for various distributions improve testing for auto-calibration.
problem Testing for auto-calibration in finite samples is challenging.
method Construct calibration bands for the exponential dispersion family using finite sample properties.
result Calibration bands allow for various tests for calibration and auto-calibration.
This paper improves multi-class calibration methods using mutual information maximization-based binning.
problem Calibration of deep neural network predictions, especially for small prior classes.
method I-Max concept for binning, shared class-wise calibration strategy.
result Improves multi-class ranking and calibration performance using a small calibration set.