arXiv research

Establishes a condition for multiclass classification-calibration of Gamma-Phi losses.

This paper aims to provide a better understanding of a symmetric loss. First, we emphasize that using a symmetric loss is advantageous in the balanced error rate (BER) minimization and area under the receiver operating characteristic curve (AUC) maximization from corrupted labels. Second, we prove general theoretical p…

We present new excess risk bounds for general unbounded loss functions including log loss and squared loss, where the distribution of the losses may be heavy-tailed. The bounds hold for general estimators, but they are optimized when applied to $η$-generalized Bayesian, MDL, and empirical risk minimization estimators. …

The impact of a stress scenario of default events on the loss distribution of a credit portfolio can be assessed by determining the loss distribution conditional on these events. While it is conceptually easy to estimate loss distributions conditional on default events by means of Monte Carlo simulation, it becomes imp…

This work introduces a new loss function to improve the efficiency of optimization-based PDE solvers.

Gradient descent with logistic loss can interpolate deep networks with smoothed ReLU activations under certain conditions.

CcGAN tackles conditional image generation for continuous labels.

Zero loss is achievable in overparametrized DL networks under specific conditions.

New bounds on neural network test loss derived from conditional information measures.

Enhanced $H$-consistency bounds derived under relaxed conditions.

Paper explores challenges in training PINNs and loss landscape effects.

In this work, we study data preconditioning, a well-known and long-existing technique, for boosting the convergence of first-order methods for regularized loss minimization. It is well understood that the condition number of the problem, i.e., the ratio of the Lipschitz constant to the strong convexity modulus, has a h…

Improves GANs by incorporating class information with a multi-hinge loss.

The paper explores conditions for predicting optimization performance.

We consider the problem of training probabilistic conditional random fields (CRFs) in the context of a task where performance is measured using a specific loss function. While maximum likelihood is the most common approach to training CRFs, it ignores the inherent structure of the task's loss function. We describe alte…

New method simplifies checking consistency of differentiable loss functions.

Recent advances in conditional image generation tasks, such as image-to-image translation and image inpainting, are largely accounted to the success of conditional GAN models, which are often optimized by the joint use of the GAN loss with the reconstruction loss. However, we reveal that this training recipe shared by …

Study tackles criterion collapse in learning criteria, showing conditions for loss minimization.

A new convex loss function optimizes set predictions with balanced size and coverage.

This paper rethinks confidence calibration under covariate shifts.

We study the error landscape of deep linear and nonlinear neural networks with the squared error loss. Minimizing the loss of a deep linear neural network is a nonconvex problem, and despite recent progress, our understanding of this loss surface is still incomplete. For deep linear networks, we present necessary and s…

Symmetric losses improve classifier robustness from corrupted labels.

Enhances Transformers for better risk assessment in finance.

It is widely conjectured that the reason that training algorithms for neural networks are successful because all local minima lead to similar performance, for example, see (LeCun et al., 2015, Choromanska et al., 2015, Dauphin et al., 2014). Performance is typically measured in terms of two metrics: training performanc…

Paper proposes a new loss function for conditional models using soft targets.

Paper develops proper, lower-bounded losses for weakly supervised classification.

Symmetrizes loss functions to improve neural network robustness against noisy labels.

New method for risk allocation under multimodality of loss distribution.

Investigates conditions for risk or utility functionals to be sensitive to large losses.

Paper corrects Max-Margin loss for multi-label tasks.

This research analyzes the consistency of convex and nonconvex surrogate losses for adversarially robust classification.

We propose a general approach for supervised learning with structured output spaces, such as combinatorial and polyhedral sets, that is based on minimizing estimated conditional risk functions. Given a loss function defined over pairs of output labels, we first estimate the conditional risk function by solving a (possi…

Paper establishes a universal growth rate for smooth surrogate losses in classification.

The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.

We analyze systems of agents sharing light-tailed risky claims issued by different financial objects. Assuming exponentially distributed claims, we obtain that both agents' and system's losses follow generalized exponential mixture distributions. We show that this leads to qualitatively different results on individual …

We consider the following conditional linear regression problem: the task is to identify both (i) a $k$-DNF condition $c$ and (ii) a linear rule $f$ such that the probability of $c$ is (approximately) at least some given bound $μ$, and $f$ minimizes the $\ell_p$ loss of predicting the target $z$ in the distribution of …

Modern machine learning focuses on highly expressive models that are able to fit or interpolate the data completely, resulting in zero training loss. For such models, we show that the stochastic gradients of common loss functions satisfy a strong growth condition. Under this condition, we prove that constant step-size …

We consider composite loss functions for multiclass prediction comprising a proper (i.e., Fisher-consistent) loss over probability distributions and an inverse link function. We establish conditions for their (strong) convexity and explore the implications. We also show how the separation of concerns afforded by using …

Derives derivatives of risk measures for various types of portfolio losses.

In domains like bioinformatics, information retrieval and social network analysis, one can find learning tasks where the goal consists of inferring a ranking of objects, conditioned on a particular target object. We present a general kernel framework for learning conditional rankings from various types of relational da…

This paper studies Fenchel-Young losses, a generic way to construct convex loss functions from a regularization function. We analyze their properties in depth, showing that they unify many well-known loss functions and allow to create useful new ones easily. Fenchel-Young losses constructed from a generalized entropy, …

Gradient descent with logistic loss can make two-layer networks interpolate binary classification data.

Framework improves ETF volatility forecasting by adapting to market conditions.

Proposes risk-averse learning framework using CVaR for better performance evaluation.

Optimizing proper loss yields calibrated models under specific conditions.

This paper examines the Histogram Loss for regression, revealing its effectiveness without needing complex tuning.

The goal of online prediction with expert advice is to find a decision strategy which will perform almost as well as the best expert in a given pool of experts, on any sequence of outcomes. This problem has been widely studied and $O(\sqrt{T})$ and $O(\log{T})$ regret bounds can be achieved for convex losses (\cite{zin…

The past decade has witnessed a successful application of deep learning to solving many challenging problems in machine learning and artificial intelligence. However, the loss functions of deep neural networks (especially nonlinear networks) are still far from being well understood from a theoretical aspect. In this pa…