The paper studies the distance from calibration in sequential prediction, proving upper and lower bounds.
problem The challenge is to measure and minimize the deviation from perfect calibration in sequential binary prediction.
method The approach involves proving an O ( T ) O(\sqrt{T}) O ( T ) upper bound and an Ω ( T 1 / 3 ) Ω(T^{1/3}) Ω ( T 1/3 ) lower bound, using structural results and minimax arguments. result An O ( T ) O(\sqrt{T}) O ( T ) upper bound on the calibration distance is achieved, with an Ω ( T 1 / 3 ) Ω(T^{1/3}) Ω ( T 1/3 ) lower bound showing the inherent difficulty. Simple algorithm achieves distance to calibration error of at most 2√T+1.
problem Achieving distance to calibration error of O(√T) in adversarial setting.
method An extremely simple, efficient, deterministic algorithm.
result Obtains distance to calibration error at most 2√T+1.
Study on computing and estimating calibration distance, showing hardness and efficiency.
problem Computing and estimating calibration distance under different assumptions.
method Efficient algorithm for exact computation, polynomial-time approximation scheme; sample-based estimation for upper bounds.
result The problem becomes NP-hard when assumptions are removed, but efficient algorithms exist under certain conditions.
Deep neural networks (DNNs) are poorly calibrated when trained in conventional ways. To improve confidence calibration of DNNs, we propose a novel training method, distance-based learning from errors (DBLE). DBLE bases its confidence estimation on distances in the representation space. In DBLE, we first adapt prototypi…
Bayesian neural networks improve uncertainty calibration with DAP priors.
problem Improving predictive uncertainty in deep learning models outside training data.
method Distance-Aware Prior (DAP) calibration method to correct overconfidence.
result Demonstrated effectiveness in various classification and regression tasks.
Post-processing predictors reduces calibration errors for decision-making.
problem Predictors with low calibration error for machine learning may have high error for decision-making.
method Post-processing with ε distance to calibration adds noise to make predictions differentially private.
result Post-processing achieves O(√ε) ECE and CDL, asymptotically optimal.
New method calibrates deep models for both in-distribution and out-of-distribution samples.
problem Ensuring calibration for deep models in safety-critical applications, especially in OOD regions.
method Geodesic distance and Gaussian kernel to calibrate deep models.
result Proposed KDF and KDN methods achieve well-calibrated posteriors for both in-distribution and out-of-distribution samples.
Efficiently simulates and calibrates the rough Bergomi model using Wasserstein distance.
problem High computational complexity in pricing and calibration of the rough Bergomi model.
method Developed a modified-sum-of-exponentials Monte Carlo scheme and a calibration approach based on Wasserstein-1 distance.
result The method achieves high pricing accuracy and improved parameter recovery, optimization stability, and out-of-sample performance.
This research improves deep neural network calibration using a new loss function.
problem Improving probability calibration in deep neural networks.
method Introduces Focal Calibration Loss (FCL) to minimize Euclidean norm and penalize calibration error.
result FCL achieves state-of-the-art performance in both calibration and accuracy metrics.
LADaR framework calibrates machine learning models for instance-wise predictions.
problem Challenges in assessing and calibrating predictive distributions for complex inputs.
method Local Amortized Diagnostics and Reshaping of Conditional Densities (LADaR) framework and e x t t t C a l − P I T exttt{Cal-PIT} e x ttt C a l − P I T algorithm. result Achieves better instance-wise calibration than existing methods in galaxy distance estimation.
Geometric method improves uncertainty estimation in real-time.
problem Improving uncertainty estimation in machine learning models.
method Geometric distance from training inputs for uncertainty estimation, post-hoc calibration.
result Method yields better uncertainty estimations than existing approaches.
A new calibration metric bridges testability and actionability.
problem Combining testability and actionable insights for forecast probabilities.
method Cutoff Calibration Error (CCE) that assesses calibration over intervals of forecasted probabilities.
result Cutoff Calibration Error is both testable and actionable.
The paper improves conformal prediction by analyzing the beta law of conditional coverage.
problem Improving finite-sample marginal coverage guarantees for non-i.i.d. data.
method The method uses Wasserstein distances to quantify deviations from the beta law of conditional coverage.
result The framework provides direct bounds on marginal coverage gaps and bad-calibration probabilities.
A new metric CKCE improves model calibration comparison.
problem Comparing the calibration of probabilistic models is challenging.
method CKCE based on Hilbert-Schmidt norm of conditional mean operators.
result CKCE provides more consistent and robust model calibration comparisons.
The paper proposes a framework to calibrate multi-agent simulation models from output series using Bayesian optimization.
problem Calibrating multi-agent simulation models from observable output series.
method Novel eligibility set concept, two-sample Kolmogorov-Smirnov test with Bonferroni correction, Bayesian optimization (BO), and trust-region BO (TuRBO).
result Demonstrated the efficiency of the proposed framework using numerical experiments.
A new perfectly truthful calibration measure improves prediction reliability.
problem Improving the reliability of predictions by ensuring they are conditionally unbiased.
method Designing a simple, perfectly truthful calibration measure called ATB.
result ATB is the first perfectly truthful calibration measure in the batch setting.
This paper proposes a new differential privacy definition using Rao distance.
problem Improving differential privacy definitions for better sequential composition.
method Using Rao distance instead of divergences of densities to define privacy.
result Proposed definition shares interpretation with previous definitions but improves sequential composition.
Mixup technique improved, reducing manifold mismatch for better calibration.
problem Improving calibration of models using Mixup.
method Dynamic adjustment of interpolation coefficients based on sample similarity.
result Improved predictive performance and calibration with reduced manifold mismatch.
PRISM-FCP improves federated prediction robustness against Byzantine attacks.
problem Byzantine attacks in federated learning.
method Partial model sharing and distance-based maliciousness scores.
result Maintains nominal coverage guarantees under Byzantine attacks.
The paper connects hyperbolicity in calibrated geometry to properties of Smith immersions.
problem Hyperbolicity in calibrated manifolds and its relation to Smith immersions.
method Establishes a theorem relating hyperbolicity to the equicontinuity of Smith immersions, proving a new Schwarz lemma.
result Calibrated hyperbolicity of compact φ φ φ -replete manifolds is equivalent to the equicontinuity of Smith immersions. Improves uncertainty estimation and OOD detection in neural networks.
problem Accurate uncertainty estimation and OOD detection in neural networks.
method Investigates one-vs-all and distance-based logit representations for probabilities.
result One-vs-all formulations improve calibration without additional complexity.
Proposes a new scoring function for linear classifiers to improve object positioning in feature space.
problem Lack of information about relative positions of recognized objects in feature space.
method Calculates a scoring function based on object distance from decision boundary and class centroid.
result Demonstrates effectiveness of the proposed method compared to other ensemble algorithms on multiple datasets.
New algorithm tests model calibration in nearly-linear time.
problem Testing model calibration from samples efficiently.
method Reformulated as minimum-cost flow, solved with dynamic programming.
result Optimal testing problem solved in nearly-linear time.
Proposes a new method for multi-class classification with well-calibrated predictions.
problem Improving the accuracy and reliability of multi-class classification models.
method Trains data in a latent space induced by an ( n − 1 ) (n-1) ( n − 1 ) -dimensional simplex, then extends and fits a regression model. result Demonstrates a well-calibrated classifier with improved prediction and calibration properties.
Meta learns low-rank covariance factors for better uncertainty estimation.
problem Sub-optimal covariance matrices in multi-task settings.
method Meta learns diagonal or diagonal plus low-rank factors using an attentive set encoder.
result Efficiently constructed task-specific covariance matrices improve uncertainty estimation.
New method calibrates photometric redshift PDFs more accurately.
problem Inaccurate photometric redshift uncertainties lead to systematic errors.
method Local re-calibration using feature-space regression of Probability Integral Transform (PIT) distributions.
result Calibrated PDFs are more accurate at all locations in feature space.
The paper proposes a neural network method to calibrate LSV models without interpolation.
problem Calibrating LSV models with market option prices using neural networks.
method Parametrizing leverage function with neural networks and learning parameters from market prices; using deep hedging for variance reduction.
result The method accurately calibrates LSV models and outperforms interpolation methods.
New method for selective prediction under interventions learns causal structure from data.
problem Tight uncertainty sets in selective conformal prediction under unknown interventional settings.
method Partial causal structure learning for descendant indicators, contamination-robust coverage theorem, algorithms for descendant discovery and distance estimation.
result Valid selective conformal prediction under contamination up to 30% with controlled coverage.
Calibrations help estimate volumes on odd spheres without gaps.
problem Estimating the minimum volume of tangent vector fields on odd spheres.
method Using a specific calibration and analyzing stable mass in the section class.
result No smooth unit field on S n S^n S n has a graph that is ω ω ω -calibrated everywhere. Given ( M ˉ , Ω ) (\bar{M},Ω) ( M ˉ , Ω ) a calibrated Riemannian manifold with a parallel calibration of rank m m m , and M m M^m M m an immersed orientable submanifold with parallel mean curvature H H H we prove that if cos θ \cos θ cos θ is bounded away from zero, where θ θ θ is the Ω Ω Ω -angle of M M M , and if M M M has zero Cheeger constant, then M M M is minimal. I…
Normalizing Flows improve prediction interval efficiency in CP.
problem Inefficient prediction intervals in CP due to non-uniform error distribution.
method Train a Normalizing Flow to optimize the distance metric between errors and inputs.
result Optimized prediction intervals are more efficient and valid.
Paper improves CDO calibration using Magnus Expansion and Deep Learning.
problem Calibrating CDO to iTraxx market data.
method Large basket approximation, SPDE, Magnus expansion, Deep Learning.
result Highly accurate calibration to market data.
Proposes CCE to assess point-wise reliability of neural network predictions.
problem Overconfidence and misaligned predictive distributions in neural networks.
method Introduces Conditional Congruence (CCE) metric using conditional kernel mean embeddings.
result CCE exhibits correctness, monotonicity, reliability, and robustness in high-dimensional regression tasks.
PAIR-CI calibrates CI tests for causal discovery with incomplete data.
problem Miscalibration of CI tests when imputing incomplete data.
method Integrates multiple imputation directly into the inferential procedure via a paired permutation design.
result PAIR-CI reduces false positive rates to below 5% in simulations.
Morse neural networks improve uncertainty quantification and detection.
problem Uncertainty quantification and out-of-distribution detection.
method Generalizes unnormalized Gaussian densities to high-dimensional submanifolds using KL-divergence loss.
result Unified approach for OOD detection, anomaly detection, and continuous learning.
This paper improves confidence measurement in deep metric learning models.
problem Measuring confidence in deep metric learning models is challenging.
method Approximates class distributions using Gaussian kernel smoothing and calibrates the confidence metric.
result Improves generalization and robustness of deep metric learning models.
The paper improves uncertainty quantification for node classification using distance-based regularization.
problem Uncertainty in deep learning models, especially for node classification tasks.
method Graph posterior networks (GPNs) with UCE loss function, followed by a distance-based regularization.
result The proposed distance-based regularization outperforms state-of-the-art methods in OOD detection and misclassification detection.
This paper introduces a new method to compare collections of distributions on manifolds and graphs.
problem Comparing collections of probability distributions over diverse domains.
method Intrinsic slicing construction for Wasserstein distances, Hilbert embedding, resampling, p-value combination.
result Powerful and well-calibrated p-values for comparing distributions on manifolds and graphs.
New method learns SIMs with arbitrary monotone activations without strong distributional assumptions.
problem Learning Single-Index Models with arbitrary monotone activations.
method Based on omniprediction with calibrated multiaccuracy and Bregman divergences.
result First agnostic learning result for SIMs with arbitrary monotone activations.
Model tracks structural changes in Brownian particle configurations on a sphere.
problem Tracking structural changes in Brownian particle configurations on a sphere.
method Introduces Frustrated Distance Matrix (FDM) model for dynamic distance matrices on S^2.
result Preserves static BBS template with dynamics as redistributed spectral mass.
New insights show embedding lengths correlate with semantic properties.
problem Contrastive embedding norms ignore embedding magnitudes but correlate with semantic properties.
method Formal theoretical framework and analysis of optimization dynamics.
result Embedding lengths encode semantic information as a byproduct of training.
Proposes HetSNGP method for joint model and data uncertainty modeling.
problem Uncertainty estimation in deep learning for safety-critical applications.
method Jointly models model and data uncertainty with HetSNGP method.
result Outperforms baseline methods on challenging out-of-distribution datasets.
SNGP improves DNNs' uncertainty estimation with minimal changes.
problem Uncertainty estimation in deep learning models for real-time applications.
method Formalizing uncertainty as a minimax problem, SNGP adds weight normalization and replaces the output layer with a Gaussian process.
result SNGP outperforms other single-model approaches in uncertainty estimation across vision and language tasks.
XGB-Chiarella model generates realistic intra-day financial price data using agent-based models.
problem Generating accurate intra-day financial price data for research and risk management.
method Agent-based financial market simulation with XGBoost machine learning calibration.
result XGB-Chiarella model accurately reflects real market behaviours and generates realistic price time series.
Recent advances in deep learning have achieved impressive gains in classification accuracy on a variety of types of data, including images and text. Despite these gains, however, concerns have been raised about the calibration, robustness, and interpretability of these models. In this paper we propose a simple way to m…
A new method for unlearning trained models without needing the original data.
problem Lack of access to original training data for privacy-preserving unlearning.
method Uses a surrogate dataset to approximate statistical properties and calibrates noise based on statistical distance.
result Effective unlearning of trained models with strong privacy guarantees, even without access to the original data.
Develops a method for stress testing correlations of financial portfolios.
problem Stress testing correlations in financial asset portfolios.
method Parametric representation of correlations, Bayesian variable selection, joint distribution of stress scenarios.
result Inference of worst-case correlation scenarios using stress tests.
Hidden Markov Model helps track asymptomatic carriers in pandemic.
problem Tracking spread of asymptomatic carriers (super-spreaders) during pandemic.
method Applied Hidden Markov Model to analyze COVID-19 data.
result Better assessment of spread extent for calibrated interventions.