arXiv research

It is widely conjectured that the reason that training algorithms for neural networks are successful because all local minima lead to similar performance, for example, see (LeCun et al., 2015, Choromanska et al., 2015, Dauphin et al., 2014). Performance is typically measured in terms of two metrics: training performanc…

MRCs minimize worst-case expected 0-1 loss and provide performance guarantees.

We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…

A hybrid loss framework improves time series forecasting by balancing global and component errors.

Boosted CVaR Classification improves tail performance in classification tasks.

Paper establishes a formula linking model performance to insurance loss ratio.

DLM for BNNs fails to improve over ELBO optimization.

Equations track profits and losses in trading algorithms.

Two new algorithms improve performance in adversarial bandits with unbounded losses.

Paper introduces MRCs that minimize worst-case 0-1 loss, providing tight performance guarantees.

This paper improves multi-label ranking by reweighting univariate losses, enhancing consistency and performance.

Study proposes a differentiable surrogate loss function for optimizing $F_β$ score in binary classification with imbalanced data.

The paper explores conditions for predicting optimization performance.

Given two networks with the same training loss on a dataset, when would they have drastically different test losses and errors? Better understanding of this question of generalization may improve practical applications of deep networks. In this paper we show that with cross-entropy loss it is surprisingly simple to ind…

Study on optimal fees in hedge funds with first-loss compensation.

Multi-task learning is a method for improving the generalizability of multiple tasks. In order to perform multiple classification tasks with one neural network model, the losses of each task should be combined. Previous studies have mostly focused on multiple prediction tasks using joint loss with static weights for tr…

Enhances trading metrics with financially grounded loss functions.

Out-of-distribution (OOD) detection approaches usually present special requirements (e.g., hyperparameter validation, collection of outlier data) and produce side effects (e.g., classification accuracy drop, slower energy-inefficient inferences). We argue that these issues are a consequence of the SoftMax loss anisotro…

This work explores alternative learning criteria beyond traditional risk.

New expressive losses improve adversarial robustness without sacrificing accuracy.

This paper examines how different loss functions affect neural network features and performance.

The one-bit quantization is implemented by one single comparator that operates at low power and a high rate. Hence one-bit compressive sensing (1bit-CS) becomes attractive in signal processing. When measurements are corrupted by noise during signal acquisition and transmission, 1bit-CS is usually modeled as minimizing …

Improves continual learning with theoretical guarantees and a new algorithm.

In few-shot learning, typically, the loss function which is applied at test time is the one we are ultimately interested in minimising, such as the mean-squared-error loss for a regression problem. However, given that we have few samples at test time, we argue that the loss function that we are interested in minimising…

Square loss performs comparably or better than cross-entropy in neural architectures for various tasks.

We present a generalization of the Cauchy/Lorentzian, Geman-McClure, Welsch/Leclerc, generalized Charbonnier, Charbonnier/pseudo-Huber/L1-L2, and L2 loss functions. By introducing robustness as a continuous parameter, our loss function allows algorithms built around robust loss minimization to be generalized, which imp…

The gain-loss ratio is known to enjoy very good properties from a normative point of view. As a confirmation, we show that the best market gain-loss ratio in the presence of a random endowment is an acceptability index and we provide its dual representation for general probability spaces. However, the gain-loss ratio w…

Tilting loss functions improves machine learning performance.

Adma proposes a flexible loss function for neural networks.

The overarching goal of this paper is to derive excess risk bounds for learning from exp-concave loss functions in passive and sequential learning settings. Exp-concave loss functions encompass several fundamental problems in machine learning such as squared loss in linear regression, logistic loss in classification, a…

New loss functions based on f-divergences improve language model performance.

We present $α$-loss, $α\in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($α=1$) and $0$-$1$ loss ($α= \infty$). We prove that $α$-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal y…

Paper introduces a new loss function for deep imbalanced classification.

In order to push the performance on realistic computer vision tasks, the number of classes in modern benchmark datasets has significantly increased in recent years. This increase in the number of classes comes along with increased ambiguity between the class labels, raising the question if top-1 error is the right perf…

Super learner with Huber loss improves cost prediction and causal effect estimation in healthcare expenditure data.

We study cross-country GDP losses due to financial crises in terms of frequency (number of loss events per period) and severity (loss per occurrence). We perform the Loss Distribution Approach (LDA) to estimate a multi-country aggregate GDP loss probability density function and the percentiles associated to extreme eve…

Study robust linear regression with outliers, providing exact asymptotics for ERM performance.

Convolution Neural Networks (CNN) have recently achieved state-of-the art performance on handwritten Chinese character recognition (HCCR). However, most of CNN models employ the SoftMax activation function and minimize cross entropy loss, which may cause loss of inter-class information. To cope with this problem, we pr…

Local update methods' performance depends on learning rates, affecting convergence rates and alignment with true loss.

Overparameterized deep networks have the capacity to memorize training data with zero \emph{training error}. Even after memorization, the \emph{training loss} continues to approach zero, making the model overconfident and the test performance degraded. Since existing regularizers do not directly aim to avoid zero train…

Develops gradient boosting for multi-label classification.

Flexible framework for bounding high-loss predictions using quantiles.

Cross-entropy loss together with softmax is arguably one of the most common used supervision components in convolutional neural networks (CNNs). Despite its simplicity, popularity and excellent performance, the component does not explicitly encourage discriminative learning of features. In this paper, we propose a gene…

Improves deep learning models by blending gradients from training loss and auxiliary objective.

Closed-form flow matching yields similar performance to stochastic version, improving model performance.

Paper analyzes proper losses and their performance in machine learning tasks.

Improved time series forecasting with expert loss integration.

New method improves deep learning models in noisy label classification.