arXiv research

Calibrating classifiers reduces grouping loss using sufficiency criteria.

In this paper we refine the process of computing calibration functions for a number of multiclass classification surrogate losses. Calibration functions are a powerful tool for easily converting bounds for the surrogate risk (which can be computed through well-known methods) into bounds for the true risk, the probabili…

Optimizing proper loss yields calibrated models under specific conditions.

We study consistency properties of surrogate loss functions for general multiclass learning problems, defined by a general multiclass loss matrix. We extend the notion of classification calibration, which has been studied for binary and multiclass 0-1 classification problems (and for certain other specific learning pro…

This paper rethinks confidence calibration under covariate shifts.

Study on calibration and consistency of adversarial surrogate losses.

This research improves deep neural network calibration using a new loss function.

Focal loss improves deep neural networks' accuracy and calibration.

Optimal multiclass U-calibration error found to be Θ(√KT).

New method improves model calibration by adjusting confidence based on prediction correctness.

Establishes a condition for multiclass classification-calibration of Gamma-Phi losses.

This research analyzes the consistency of convex and nonconvex surrogate losses for adversarially robust classification.

New decision-theoretic calibration error metric improves prediction reliability.

Loss-calibrated EP improves Bayesian decision-making by focusing on utility-sensitive posterior approximations.

New loss function calibrates WW-hinge loss for multiclass SVM.

We present surrogate regret bounds for arbitrary surrogate losses in the context of binary classification with label-dependent costs. Such bounds relate a classifier's risk, assessed with respect to a surrogate loss, to its cost-sensitive classification risk. Two approaches to surrogate regret bounds are developed. The…

Improves forecast calibration for extreme events using modified loss functions.

New method estimates grouping loss in neural networks to improve confidence scores.

New method simplifies checking consistency of differentiable loss functions.

Focal loss reduces model curvature for better calibration.

New methods calibrate causal estimates using standard predictive models.

EnsLoss combines multiple loss functions to prevent overfitting in classification.

The top-$k$ error is often employed to evaluate performance for challenging classification tasks in computer vision as it is designed to compensate for ambiguity in ground truth labels. This practical success motivates our theoretical analysis of consistent top-$k$ classification. Surprisingly, it is not rigorously und…

New method improves decision-making accuracy without complex calculations.

Unified framework for generalized Venn and Venn-Abers calibration for reliable prediction.

New loss improves DNN calibration without sacrificing accuracy.

New algorithms achieve decision calibration without sample complexity dependent on feature dimension.

This paper characterizes and designs loss functions for robust classification with abstention.

Post-processing predictors reduces calibration errors for decision-making.

This paper establishes the asymptotic consistency of the {\it loss-calibrated variational Bayes} (LCVB) method. LCVB was proposed in~\cite{LaSiGh2011} as a method for approximately computing Bayesian posteriors in a `loss aware' manner. This methodology is also highly relevant in general data-driven decision-making con…

This work investigates square loss in overparametrized neural networks, revealing its advantages in robustness and calibration.

Unified binary and multiclass margin-based classification methods.

In many classification problems it is desirable to output well-calibrated probabilities on the different classes. We propose a robust, non-parametric method of calibrating probabilities called SplineCalib that utilizes smoothing splines to determine a calibration function. We demonstrate how applying certain transforma…

Study on learning to defer to multiple experts with consistent surrogates and confidence calibration.

New algorithm reduces prediction errors across various loss functions.

Proposes squentropy loss for improved classification accuracy and model calibration.

Efficient algorithms find optimal monotone transforms for calibration under strictly convex losses.

Generative models often misrepresent class frequencies; this paper calibrates them.

The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.

We investigate the problem of multiclass classification with rejection, where a classifier can choose not to make a prediction to avoid critical misclassification. First, we consider an approach based on simultaneous training of a classifier and a rejector, which achieves the state-of-the-art performance in the binary …

Boosted decision trees typically yield good accuracy, precision, and ROC area. However, because the outputs from boosting are not well calibrated posterior probabilities, boosting yields poor squared error and cross-entropy. We empirically demonstrate why AdaBoost predicts distorted probabilities and examine three cali…

This paper argues against using calibration metrics for assessing posterior probabilities and proposes expected proper scoring rules instead.

Study compares various calibration methods for binary classification tasks.

Loss estimator improves model calibration and generalization.

The paper optimizes forecasting for risk-adjusted decisions under trading frictions.

Study non-monotonic loss functions in CRC, achieving valid risk control with large calibration samples.

New method improves calibration in multi-output probabilistic models.

This paper improves risk bounds and calibration for smart predict-then-optimize method.