This paper improves operational risk modeling by selecting better loss severity distributions.
problem Inconsistent regulatory capital calculations due to changing loss severity distribution families.
method Presented truncation probability estimates and a consistent quantile scoring function for selection criteria. Also, recommended collecting loss frequencies below the minimum reporting threshold.
result More stable regulatory capital calculations through better selection of loss severity distributions.
The paper proves deep learning can be robust with certain loss functions.
problem The robustness of deep learning models under flawed data.
method Empirical-risk minimization with unbounded, Lipschitz-continuous loss functions.
result These loss functions provide efficient prediction under minimal data assumptions.
Study binary choice with asymmetric loss, offering simple solutions.
problem Binary choice with asymmetric loss in data-rich environments.
method Loss-based reweighting of logistic regression or machine learning techniques.
result Valid decisions on binary outcomes with general loss functions.
Unhinged loss minimization fails to improve classifier accuracy for simple data.
problem Accuracy of classifiers minimizing the unhinged loss.
method Minimizing the unhinged loss function.
result Minimizing the unhinged loss yields classifiers with accuracy no better than random guessing for simple data.
New methods evaluate data representations by complexity of low-loss predictor learning.
problem Evaluating quality of data representations for downstream tasks.
method Surplus Description Length (SDL) and ε Sample Complexity (εSC) methods.
result Methods measure the information needed to approximate optimal predictor up to specified tolerance.
Study proposes a differentiable surrogate loss function for optimizing Fβ score in binary classification with imbalanced data.
problem Non-differentiability of Fβ score makes it unsuitable for optimization by gradient-based learning. method Investigated relationship between Fβ score and loss functions, proposed a differentiable surrogate loss function. result Gradient paths of the proposed surrogate Fβ loss function approximate the gradient paths of the Fβ score. Generative adversarial nets (GANs) are widely used to learn the data sampling process and their performance may heavily depend on the loss functions, given a limited computational budget. This study revisits MMD-GAN that uses the maximum mean discrepancy (MMD) as the loss function for GAN and makes two contributions. F…
Gradient descent with logistic loss can make two-layer networks interpolate binary classification data.
problem Training two-layer networks for binary classification.
method Gradient descent with logistic loss applied to two-layer networks.
result Gradient descent can drive training loss to zero under certain conditions.
In this work, we study data preconditioning, a well-known and long-existing technique, for boosting the convergence of first-order methods for regularized loss minimization. It is well understood that the condition number of the problem, i.e., the ratio of the Lipschitz constant to the strong convexity modulus, has a h…
Informer model with GMADL loss outperforms benchmarks in high frequency Bitcoin trading.
problem Developing automated trading strategies for high frequency Bitcoin data.
method Informer architecture with RMSE, GMADL, and Quantile loss functions.
result Informer model with GMADL loss function outperforms benchmarks in trading outcomes.
The study finds a trade-off between model size, test loss, and training loss for linear predictors.
problem Finding the optimal balance between model size, test loss, and training loss for linear predictors.
method Established an algorithm and distribution-independent trade-off using non-asymptotic analysis.
result Models with low test loss are either classical (close to noise level training loss) or modern (large number of parameters).
Paper improves stability analysis of SGD for various loss functions and data distributions.
problem Improving stability analysis of SGD for non-convex loss functions and data distributions.
method Analyzes stability of SGD for convex and non-convex loss functions, and improves data-dependent bounds.
result Improved stability bounds for non-convex loss functions and convex regularized loss functions.
Novel loss functions improve decision tree learning from noisy data.
problem Training decision trees with noisy labels.
method Introducing distribution losses and a new negative exponential loss.
result The negative exponential loss leads to efficient and robust decision tree learning.
In this work, we introduce the {\em average top-k} (\atk) loss as a new aggregate loss for supervised learning, which is the average over the k largest individual losses over a training dataset. We show that the \atk loss is a natural generalization of the two widely used aggregate losses, namely the average loss a…
This paper quantifies privacy loss in exploratory data analysis.
problem Privacy loss in exploratory data analysis is often overlooked in privacy budgets.
method Quantitative analysis of privacy loss for statistical functions.
result Privacy loss must be considered in calculating machine learning privacy budgets.
This paper quantifies and mitigates a bias in the Hayashi-Yoshida estimator causing data loss.
problem Formulaic bias in the Hayashi-Yoshida estimator leading to data loss.
method Formalizes and quantifies the data loss, introduces (a,b)-asynchronous adversary, and provides algorithms.
result Proves that for equal rates, the minimal average cumulative data loss is 25%.
Study improves PM concentration forecasting using MCCR loss.
problem Forecasting particulate matter concentration in South Korea.
method Used MCCR loss for regression analysis of air pollution and weather data.
result MCCR loss is more effective for extreme value forecasting.
Loss Data Analytics is an interactive, online, freely available text. The idea behind the name Loss Data Analytics is to integrate classical loss data models from applied probability with modern analytic tools. In particular, we seek to recognize that big data (including social media and usage based insurance) are here…
Minimizing a convex risk function is the main step in many basic learning algorithms. We study protocols for convex optimization which provably leak very little about the individual data points that constitute the loss function. Specifically, we consider differentially private algorithms that operate in the local model…
Deep networks can memorize random labels; symmetric loss improves this.
problem Deep networks can memorize random labels, ignoring standard regularization.
method Empirical studies with MNIST and CIFAR-10 datasets, formal definition of robustness.
result Symmetric loss function improves network's ability to resist memorization.
A new method, VIF, calculates influence for non-decomposable losses efficiently.
problem Efficiently calculating influence for complex machine learning models with non-decomposable losses.
method Revisiting influence function from robust statistics, proposing Versatile Influence Function (VIF) for any non-decomposable loss.
result VIF method is up to 10^3 times faster than brute-force methods and closely matches influence results.
This paper examines the role and efficiency of the non-convex loss functions for binary classification problems. In particular, we investigate how to design a simple and effective boosting algorithm that is robust to the outliers in the data. The analysis of the role of a particular non-convex loss for prediction accur…
New method uses FY loss for better inverse optimization.
problem Estimating unknown parameters from noisy and suboptimal solutions.
method Fenchel-Young loss approach for efficient gradient-based optimization.
result Significant improvement in parameter estimation accuracy and computational speed.
Out-of-distribution (OOD) detection approaches usually present special requirements (e.g., hyperparameter validation, collection of outlier data) and produce side effects (e.g., classification accuracy drop, slower energy-inefficient inferences). We argue that these issues are a consequence of the SoftMax loss anisotro…
Gradient descent struggles to achieve zero loss in deep learning models due to non-generic data distributions.
problem Achieving zero loss minimizers in deep learning networks.
method Analysis of gradient descent algorithm in deep learning, focusing on underparametrized networks.
result Zero loss minimization cannot be achieved generically in deep learning networks.
A new framework learns differentiable structured losses from data.
problem Learning effective losses for complex structured prediction tasks.
method Contrastive learning to learn differentiable structured losses from output data.
result Achieves similar or better performance than kernel-based methods.
A new method optimizes anomaly scoring from score distribution to improve AD performance.
problem Vulnerability to anomaly contamination and lack of adaptability in existing AD methods.
method Optimizes anomaly scoring function from score distribution perspective, using Overlap loss.
result Overlap loss-based AD models significantly outperform state-of-the-art methods.
Enhances insurance loss models using InsurTech data and machine learning.
problem Traditional insurance loss models lack predictive accuracy due to limited data sources.
method Combining proprietary claims data with InsurTech data and applying machine learning techniques.
result Improved predictive accuracy of the loss model through machine learning.
Proposes a new loss function for learning with noisy labels.
problem Improving model learnability with noisy labels.
method Uses generalized Jensen-Shannon divergence as a noise-robust loss function.
result Shows state-of-the-art results on noisy data.
Optimizes hybrid insurance contracts for heavy-tailed losses.
problem Providing insurance against heavy-tailed losses with finite expected loss.
method Combines traditional and parametric insurance, using a Pareto-type criterion for optimization.
result The hybrid contract outperforms traditional contracts in simulations and real data.
Introduces CHL, a new loss function for continuous similarity learning.
problem Binary similarity learning limitations.
method CHL is a novel loss function that generalizes histogram loss to continuous similarities.
result CHL solves a wider range of tasks including similarity learning, representation learning, and data visualization.
Typically, operational risk losses are reported above a threshold. Fitting data reported above a constant threshold is a well known and studied problem. However, in practice, the losses are scaled for business and other factors before the fitting and thus the threshold is varying across the scaled data sample. A report…
The paper predicts loss scaling across different datasets and compute scales.
problem Predicting loss scaling across different datasets and compute scales.
method Derive shifted power law relationships between train and test losses.
result Shifted power law relationships hold for various datasets and tasks, improving prediction accuracy.
Gradient descent converges with arbitrary stepsize for separable data under Fenchel-Young losses.
problem Understanding the conditions under which gradient descent converges with arbitrary stepsize.
method Using Fenchel-Young losses and leveraging the classical perceptron argument to derive convergence rates.
result GD converges with arbitrary stepsize for a majority of Fenchel-Young losses, with better rates for specific loss functions.
Consider a classification problem where we have both labeled and unlabeled data available. We show that for linear classifiers defined by convex margin-based surrogate losses that are decreasing, it is impossible to construct any semi-supervised approach that is able to guarantee an improvement over the supervised clas…
Study how data structure impacts classification performance in overparametrized models.
problem Understanding the role of data structure in overparametrized models.
method Analytical and statistical physics methods applied to structured data.
result Data structure significantly affects the performance of overparametrized models, particularly for logistic loss.
Paper introduces MADL loss function for better AIS model optimization.
problem Optimizing machine learning models for AIS construction.
method Proposes Mean Absolute Directional Loss (MADL) function.
result MADL function improves hyperparameter selection and investment strategy efficiency.
Quantum models face barren plateaus, but specific losses can be trainable.
problem Barren plateaus and loss concentration in quantum generative models.
method Investigated explicit and implicit losses, and their interplay.
result Explicit losses lead to new barren plateaus, while implicit losses can be trainable.
Generative adversarial networks (GANs) can be interpreted as an adversarial game between two players, a discriminator D and a generator G, in which D learns to classify real from fake data and G learns to generate realistic data by "fooling" D into thinking that fake data is actually real data. Currently, a dominating …
Separable losses are inconsistent for structured prediction models.
problem Inconsistency of separable losses in structured prediction models.
method Analysis of separable negative log-likelihood losses for structured prediction.
result Separable losses are not Bayes consistent and may not predict the most probable structure.
ScoreStop uses gradient tests to stop gradient boosting early.
problem Overfitting in gradient boosted decision trees.
method ScoreStop uses a functional score test based on gradients to stop boosting.
result ScoreStop is competitive with loss-based early stopping methods.
A method to identify important features without solving the full problem.
problem Identifying important features in high-dimensional data.
method Persistent reduction using extreme ray identification on a polyhedral cone.
result A subset of features can be guaranteed to have zero coefficients in all optimal solutions.
Stochastic Gradient Descent (SGD) is a central tool in machine learning. We prove that SGD converges to zero loss, even with a fixed (non-vanishing) learning rate - in the special case of homogeneous linear classifiers with smooth monotone loss functions, optimized on linearly separable data. Previous works assumed eit…
Multi-output is essential in machine learning that it might suffer from nonconforming residual distributions, i.e., the multi-output residual distributions are not conforming to the expected distribution. In this paper, we propose "Wrapped Loss Function" to wrap the original loss function to alleviate the problem. This…
Paper explains learning property of logistic and softmax losses for balanced and imbalanced class data.
problem Understanding and optimizing loss functions for deep neural networks with class imbalances.
method Analyzing necessary conditions for convergence of logistic and softmax losses in CNNs.
result Proposes a novel reweighted logistic loss function that improves performance over softmax loss.
This work investigates square loss in overparametrized neural networks, revealing its advantages in robustness and calibration.
problem Theoretical understanding of square loss in overparametrized neural networks.
method Systematic investigation of square loss in the NTK regime for both separable and non-separable classes.
result Square loss shows fast convergence rates and robustness guarantees for overparametrized neural networks.
The paper develops loss functions for pricing models using observational data.
problem Evaluating pricing policies directly from observational data with historical biases.
method Adapting machine learning techniques for corrupted labels to derive unbiased loss functions.
result Identifies minimum variance and robust estimators for contextual pricing.
Proposes new loss functions for GANs to improve estimation accuracy and robustness.
problem Improving the training of GANs to achieve more accurate and robust models.
method Introduces Hellinger-type loss functions and analyzes their statistical properties.
result Demonstrates improved estimation accuracy and robustness of the proposed loss functions.