arXiv research

Improved online learning for hidden-convex losses achieves optimal regret.

Paper reveals hidden convexities in deep learning models using sparse signal processing.

We found hidden convexity in FPCA and developed a faster algorithm.

The paper explains how simple methods can converge to optimal solutions in complex neural games.

In recent years, spectral clustering has become a standard method for data analysis used in a broad range of applications. In this paper we propose a new class of algorithms for multiway spectral clustering based on optimization of a certain "contrast function" over the unit sphere. These algorithms, partly inspired by…

New method trains quantized neural networks to global optimality.

Wasserstein GANs are shown to have hidden convexity, enabling exact solutions with convex optimization.

We solve the optimization of two-layer ReLU networks using convex math.

Path regularization reveals convex optimization in deep ReLU networks.

A new method reduces variance in PG methods for RL, improving efficiency and convergence.

Gradient Descent Ascent converges to von-Neumann solution in hidden zero-sum games.

New method for RL with general utilities using variational policy gradient.

Introduces Fitzpatrick losses, tighter than Fenchel-Young losses.

We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determ…

We present the Tamed Cross Entropy (TCE) loss function, a robust derivative of the standard Cross Entropy (CE) loss used in deep learning for classification tasks. However, unlike other robust losses, the TCE loss is designed to exhibit the same training properties than the CE loss in noiseless scenarios. Therefore, th…

Unified surrogate loss framework for multi-label learning with strong consistency guarantees.

This paper introduces new loss functions for balanced multi-class classification.

We present $α$-loss, $α\in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($α=1$) and $0$-$1$ loss ($α= \infty$). We prove that $α$-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal y…

This work broadens calibeating to various proper losses using Bregman divergence.

This work generalizes calibeating for a broader range of proper losses using Bregman divergence.

Proposes squentropy loss for improved classification accuracy and model calibration.

New loss function calibrates WW-hinge loss for multiclass SVM.

The study analyzes a model for aggregate losses with dependent and overdispersed inter-losses times.

Symmetric losses improve classifier robustness from corrupted labels.

Two new algorithms improve performance in adversarial bandits with unbounded losses.

Paper explores connections between loss functions and consistency in binary classification and regression.

Paper introduces a new topological loss for better convergence.

Novel loss functions improve decision tree learning from noisy data.

Theoretical analysis of cross-entropy loss functions and their robustness.

This paper improves operational risk modeling by selecting better loss severity distributions.

This paper improves loss functions for deep learning with noisy labels.

Classification is the most important process in data analysis. However, due to the inherent non-convex and non-smooth structure of the zero-one loss function of the classification model, various convex surrogate loss functions such as hinge loss, squared hinge loss, logistic loss, and exponential loss are introduced. T…

We study cross-country GDP losses due to financial crises in terms of frequency (number of loss events per period) and severity (loss per occurrence). We perform the Loss Distribution Approach (LDA) to estimate a multi-country aggregate GDP loss probability density function and the percentiles associated to extreme eve…

The paper proves deep learning can be robust with certain loss functions.

The paper explores transferability of adversarial examples between convex and 01 loss models, finding non-transferability due to different decision boundaries caused by outliers.

Symmetrizes loss functions to improve neural network robustness against noisy labels.

EnsLoss combines multiple loss functions to prevent overfitting in classification.

Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural network loss surfaces at large. This work proposes a novel method to visualise basins of attraction together with the associated stationary points…

Study compares metric learning loss functions for speaker verification.

SoftAdapt dynamically adjusts loss weights for multi-part functions.

In this work, we introduce the {\em average top-$k$} (\atk) loss as a new aggregate loss for supervised learning, which is the average over the $k$ largest individual losses over a training dataset. We show that the \atk loss is a natural generalization of the two widely used aggregate losses, namely the average loss a…

Novel approach embeds loss tunnels in neural networks, revealing insights into their structure.

We establish linear regret bounds for convex smooth losses using Fenchel-Young losses.

Establishes a condition for multiclass classification-calibration of Gamma-Phi losses.

Unhinged loss minimization fails to improve classifier accuracy for simple data.

A hybrid loss framework improves time series forecasting by balancing global and component errors.

New loss functions based on f-divergences improve language model performance.

Proposes a new loss function for robust learning.