arXiv research

Paper develops proper, lower-bounded losses for weakly supervised classification.

Paper analyzes proper losses and their performance in machine learning tasks.

We study proper losses for discrete generative models without knowing the target distribution.

We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determ…

This work broadens calibeating to various proper losses using Bregman divergence.

This work generalizes calibeating for a broader range of proper losses using Bregman divergence.

Optimizing proper loss yields calibrated models under specific conditions.

Study on proper learning under relaxed worst-case robust loss for VC classes.

A new method learns proper multiclass losses and probabilities.

Optimal multiclass U-calibration error found to be Θ(√KT).

New algorithm reduces prediction errors across various loss functions.

The goal of online prediction with expert advice is to find a decision strategy which will perform almost as well as the best expert in a given pool of experts, on any sequence of outcomes. This problem has been widely studied and $O(\sqrt{T})$ and $O(\log{T})$ regret bounds can be achieved for convex losses (\cite{zin…

The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.

The problem of bipartite ranking, where instances are labeled positive or negative and the goal is to learn a scoring function that minimizes the probability of mis-ranking a pair of positive and negative instances (or equivalently, that maximizes the area under the ROC curve), has been widely studied in recent years. …

Proposes measures for uncertainty quantification using proper scoring rules.

Paper proposes fitting loss functions to data using source functions from information geometry.

New algorithms minimize dynamic regret for strongly convex losses.

Improves generative models for cost-sensitive decisions.

New method improves decision-making accuracy without complex calculations.

Generative Cross-Entropy improves classification with fewer labels.

We consider composite loss functions for multiclass prediction comprising a proper (i.e., Fisher-consistent) loss over probability distributions and an inverse link function. We establish conditions for their (strong) convexity and explore the implications. We also show how the separation of concerns afforded by using …

This work examines uncertainty sampling in binary classification using equivalent loss.

The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules via the proper-composite representation, in which predictions are mapped to probab…

A new method decomposes subjective risk into epistemic and aleatoric uncertainties.

Estimates proper calibration errors and refinement terms in probabilistic predictions.

We study strictly proper scoring rules in the Reproducing Kernel Hilbert Space. We propose a general Kernel Scoring rule and associated Kernel Divergence. We consider conditions under which the Kernel Score is strictly proper. We then demonstrate that the Kernel Score includes the Maximum Mean Discrepancy as a special …

We analyze impermanent loss in AMMs and show G3Ms are simplest.

The paper argues that uncertainty quantification in ML is application-specific and proposes a flexible family of measures.

A novel framework quantifies uncertainty using proper scores for various tasks.

New algorithms achieve near-optimal cumulative loss in nonparametric online learning and games.

SurvivalBoost improves prediction of event times in competing risks scenarios.

The concept of refinement from probability elicitation is considered for proper scoring rules. Taking directions from the axioms of probability, refinement is further clarified using a Hilbert space interpretation and reformulated into the underlying data distribution setting where connections to maximal marginal diver…

The last few years have seen a staggering number of empirical studies of the robustness of neural networks in a model of adversarial perturbations of their inputs. Most rely on an adversary which carries out local modifications within prescribed balls. None however has so far questioned the broader picture: how to fram…

Critiques binary classification evaluation methods, advocating for proper scoring rules.

Deep models can fit noisy labels, but robustness and reliability are still issues.

This paper argues against using calibration metrics for assessing posterior probabilities and proposes expected proper scoring rules instead.

Transformer models outperform LSTM in financial forecasting with MADL loss.

Proper learning is possible with labeled data, but unlabeled data can improve performance.

Boosted ensemble of decision tree (DT) classifiers are extremely popular in international competitions, yet to our knowledge nothing is formally known on how to make them \textit{also} differential private (DP), up to the point that random forests currently reign supreme in the DP stage. Our paper starts with the proof…

New findings show second-order scoring rules can't accurately represent epistemic uncertainty.

The paper introduces new measures for quantifying uncertainty in machine learning.

This work explores alternative learning criteria beyond traditional risk.

New scoring rules improve probabilistic classification model evaluation.

New method estimates grouping loss in neural networks to improve confidence scores.

We develop efficient algorithms to train $\ell_1$-regularized linear classifiers with large dimensionality $d$ of the feature space, number of classes $k$, and sample size $n$. Our focus is on a special class of losses that includes, in particular, the multiclass hinge and logistic losses. Our approach combines several…

This work improves diffusion models by estimating the optimal loss value for better training diagnostics.

Focal loss improves classification but not class-posterior probability estimation.

We unify f-divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically studying integral and variational representations of these objects and in so doing identify their primitives which all are rela…