arXiv research

New loss function optimization improves training speed and accuracy.

Study proposes a differentiable surrogate loss function for optimizing $F_β$ score in binary classification with imbalanced data.

This work classifies SOC loss functions based on their gradient properties.

Efficiently learns loss functions for better model performance.

A new convex loss function optimizes set predictions with balanced size and coverage.

We analyze the optimization landscape of α-loss in logistic models.

Paper proposes a method to minimize non-differentiable loss functions.

Optimal unimodal fitting for linear loss functions in a sequential, efficient manner.

We consider the problem of estimating a low-rank matrix from a noisy observed matrix. Previous work has shown that the optimal method depends crucially on the choice of loss function. In this paper, we use a family of weighted loss functions, which arise naturally for problems such as submatrix denoising, denoising wit…

Near-optimal algorithms for predicting across multiple loss functions efficiently.

Study visualizes actor-critic loss landscapes for inventory optimization.

New $\alpha$-divergence loss function improves neural density ratio estimation.

New method finds better loss functions for neural nets.

Optimizing proper loss yields calibrated models under specific conditions.

Boosting can efficiently optimize any loss function without requiring first-order information.

SoftAdapt dynamically adjusts loss weights for multi-part functions.

Simplified screening tests for data points in optimization.

The paper explores conditions for predicting optimization performance.

Paper introduces MADL loss function for better AIS model optimization.

Private algorithms minimize population loss with optimal rate.

Paper proposes new loss functions for training energy networks.

Multi-output is essential in machine learning that it might suffer from nonconforming residual distributions, i.e., the multi-output residual distributions are not conforming to the expected distribution. In this paper, we propose "Wrapped Loss Function" to wrap the original loss function to alleviate the problem. This…

This work introduces a new loss function to improve the efficiency of optimization-based PDE solvers.

The simplicity of gradient descent (GD) made it the default method for training ever-deeper and complex neural networks. Both loss functions and architectures are often explicitly tuned to be amenable to this basic local optimization. In the context of weakly-supervised CNN segmentation, we demonstrate a well-motivated…

Study uses property elicitation to understand how fairness regularizers affect optimal decisions.

Efficient algorithms find optimal monotone transforms for calibration under strictly convex losses.

Many real-world analytics problems involve two significant challenges: prediction and optimization. Due to the typically complex nature of each challenge, the standard paradigm is predict-then-optimize. By and large, machine learning tools are intended to minimize prediction error and do not account for how the predict…

The paper proves neural networks' consistency and optimal convergence rates for various function classes.

In the hypothesis of rare loss events, the general expression of the policy value has been determined as a functional of the "expected frequency / loss severity" function and of the retention function. Exponential disutility has been chosen after mathematical characterization of some of its economical aspects, where fu…

We address the problem of aggregating an ensemble of predictors with known loss bounds in a semi-supervised binary classification setting, to minimize prediction loss incurred on the unlabeled data. We find the minimax optimal predictions for a very general class of loss functions including all convex and many non-conv…

A new loss function $\alpha$-loss improves classification robustness and calibration.

Paper introduces a new topological loss for better convergence.

Peer loss functions learn from noisy labels without noise rate specification.

We propose an online convex optimization algorithm (RescaledExp) that achieves optimal regret in the unconstrained setting without prior knowledge of any bounds on the loss functions. We prove a lower bound showing an exponential separation between the regret of existing algorithms that require a known bound on the los…

In most machine learning training paradigms a fixed, often handcrafted, loss function is assumed to be a good proxy for an underlying evaluation metric. In this work we assess this assumption by meta-learning an adaptive loss function to directly optimize the evaluation metric. We propose a sample efficient reinforceme…

Integrates uncertainty of loss landscape into stochastic optimization.

Paper introduces a new loss function for deep imbalanced classification.

We study the rates of convergence from empirical surrogate risk minimizers to the Bayes optimal classifier. Specifically, we introduce the notion of \emph{consistency intensity} to characterize a surrogate loss function and exploit this notion to obtain the rate of convergence from an empirical surrogate risk minimizer…

The scalability of submodular optimization methods is critical for their usability in practice. In this paper, we study the reducibility of submodular functions, a property that enables us to reduce the solution space of submodular optimization problems without performance loss. We introduce the concept of reducibility…

New solver for MKL-SVM with 0/1 loss function.

Improved privacy-preserving methods for convex optimization with heavy-tailed data.

Much of the focus in machine learning research is placed in creating new architectures and optimization methods, but the overall loss function is seldom questioned. This paper interprets machine learning from a multi-objective optimization perspective, showing the limitations of the default linear combination of loss f…

New function class characterizes loss landscape of deep neural networks without over-parametrization.

Linear-Core Surrogates combine fast optimization and statistical efficiency in classification and structured prediction.

AUC (area under ROC curve) is an important evaluation criterion, which has been popularly used in many learning tasks such as class-imbalance learning, cost-sensitive learning, learning to rank, etc. Many learning approaches try to optimize AUC, while owing to the non-convexity and discontinuousness of AUC, almost all …

This work provides bounds on the performance of prediction models in the predict-then-optimize framework.

Ensemble techniques are powerful approaches that combine several weak learners to build a stronger one. As a meta-learning framework, ensemble techniques can easily be applied to many machine learning methods. Inspired by ensemble techniques, in this paper we propose an ensemble loss functions applied to a simple regre…

AdamL optimizes deep learning models by incorporating loss function information.