COREL learns latent representations that naturally cluster, outperforming CCE.
problem Training neural networks to learn useful latent representations.
method Attractive-Repulsive Loss Framework for Clustering-Oriented Representation Learning (COREL).
result COREL variants outperform CCE in various classification tasks.
Paper proposes ARB-Loss to improve classification precision in imbalanced datasets.
problem Improving classification precision on minor classes in imbalanced datasets.
method Introduces Attraction-Repulsion-Balanced Loss (ARB-Loss) to balance gradients across different classes.
result ARB-Loss achieves state-of-the-art performance with one-stage training.
DNNs with L2 regularization reveal feature learning dynamics and sparsity.
problem Understanding feature learning in DNNs with L2 regularization. method Reformulating loss in terms of layerwise activations and covariances.
result Proving sparsity of local minima in L2-regularized DNNs. New clustering method promotes diversity while maintaining fairness.
problem Developing clustering methods that enhance diversity with respect to protected attributes.
method Attraction-repulsion dissimilarities to penalize homogeneity with respect to protected attributes.
result Improves diversity in clusters without compromising fairness.
Neighbor embeddings balance attraction and repulsion to visualize data.
problem Visualizing high-dimensional datasets with trade-offs between continuous and discrete structures.
method Neighbor embeddings combine attractive and repulsive forces to visualize data.
result Changing the exaggeration parameter in t-SNE yields a spectrum of embeddings with a trade-off between continuous and discrete structures.
ARS visualization improves t-SNE dynamics with tunable attraction and repulsion.
problem Improve data visualization techniques for complex data sets.
method ARS framework based on t-SNE dynamics with normalized interactions and tunable kernels.
result ARS visualization provides better control over cluster tightness and spacing.
A new RL method uses flows to mix policies for better exploration.
problem Learning in high-dimensional state spaces with deceptive rewards.
method Uses normalizing flows to attract and repel policies in a population.
result Empirically outperforms Soft-Actor Critic (SAC) on MuJoCo tasks.
A new method, tree-SNE, solves the scale problem in t-SNE.
problem Clustering and visualizing high-dimensional data, especially MNIST digits.
method Revisits t-SNE idea to create a 2+1 dimensional embedding with a scale parameter.
result The optimal embedding depends continuously on the scale parameter for all initial conditions.
CAMEL embeds data into a manifold using curvature-augmented forces.
problem Data visualization and dimensionality reduction.
method Formulates DR as a physics model with curvature-augmented forces.
result CAMEL outperforms existing methods on benchmark datasets.
Method identifies IPS governing equations from particle data efficiently.
problem Identify governing equations of interacting particle systems efficiently.
method Combines mean-field theory and WSINDy for large N and M. result Converges with rate O(N−1/2) for N≥100. Establishes a link between heat diffusion and manifold distances in data.
problem No theoretical link between diffusion-based manifold learning and geodesic distances.
method Formulates heat geodesic embeddings based on Riemannian geometry.
result Method outperforms state-of-the-art in preserving manifold distances and cluster structure.
Study on capillarity minimizers with nonlocal repulsion and gravity, proving existence and nonexistence.
problem Minimizing an energy functional with capillarity, nonlocal repulsion, and gravity.
method Quantitative isoperimetric inequalities applied to capillarity problem in a half-space.
result Existence and nonexistence of minimizers for various nonlocal kernels and masses.
Graph convolutions can enhance high frequencies, leading to over-sharpening.
problem Graph convolutions suffer from over-smoothing and poor performance on heterophilic graphs.
method Rigorously prove that linear graph convolutions minimize a generalized Dirichlet energy, showing that weight matrices induce edge-wise attraction or repulsion.
result Graph convolutions can enhance high frequencies, leading to over-sharpening instead of over-smoothing.
A new loss function α-loss bridges log-loss and 0-1 loss for binary classification.
problem Improving binary classification performance using a tunable loss function.
method Introducing α-loss, proving its margin-based form and classification-calibration, and providing an upper bound on empirical risk. result Empirical and expected risk difference upper bound for logistic regression-based classification.
Introduces Fitzpatrick losses, tighter than Fenchel-Young losses.
problem Improving loss functions for machine learning.
method Introduces Fitzpatrick losses based on the Fitzpatrick function.
result Fitzpatrick losses are tighter than Fenchel-Young losses.
We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determ…
We present the Tamed Cross Entropy (TCE) loss function, a robust derivative of the standard Cross Entropy (CE) loss used in deep learning for classification tasks. However, unlike other robust losses, the TCE loss is designed to exhibit the same training properties than the CE loss in noiseless scenarios. Therefore, th…
Unified surrogate loss framework for multi-label learning with strong consistency guarantees.
problem Improving consistency and accounting for label correlations in multi-label learning.
method Introducing multi-label logistic loss and extending it to comprehensive multi-label comp-sum losses, proving strong consistency guarantees for any multi-label loss.
result Unified surrogate loss framework benefiting from strong consistency guarantees for any multi-label loss.
This paper introduces new loss functions for balanced multi-class classification.
problem Balancing class imbalance in multi-class classification.
method Introduces two new surrogate loss families: GLA and GCA.
result GCA losses offer stronger theoretical guarantees in imbalanced settings.
Visualizes basins of attraction for neural network loss functions.
problem Understanding the nature of neural network loss surfaces and basins of attraction.
method Gradient-based random sampling to visualize basins of attraction and stationary points.
result Entropic loss has a more searchable landscape with fewer stationary points than quadratic loss.
This work broadens calibeating to various proper losses using Bregman divergence.
problem Calibration for a wide range of proper losses.
method Regret minimization and Bregman divergence approach.
result U-calibration results for a family of Tsallis losses with logarithmic regret and dimension independence.
This work generalizes calibeating for a broader range of proper losses using Bregman divergence.
problem Calibration for a wide range of proper losses beyond Brier and log loss.
method Regret minimization based on Bregman divergence for a family of proper losses.
result U-calibration results for a family of Tsallis losses with logarithmic regret and dimension independence.
Proposes squentropy loss for improved classification accuracy and model calibration.
problem Theoretical and empirical evidence for cross-entropy loss is lacking.
method Introduces squentropy loss as the sum of cross-entropy and average square loss over incorrect classes.
result Squentropy loss outperforms cross-entropy and rescaled square losses in classification accuracy and model calibration.
New loss function calibrates WW-hinge loss for multiclass SVM.
problem WW-hinge loss not calibrated with 0-1 loss.
method Introduced ordered partition loss and proved WW-hinge loss is calibrated.
result WW-hinge loss is calibrated with ordered partition loss.
The study analyzes a model for aggregate losses with dependent and overdispersed inter-losses times.
problem Analyzing aggregate loss models with dependent and overdispersed inter-losses times.
method The study uses a two-state Markovian arrival process (MAP2) and a Markov renewal process to model the inter-losses times. Severities are modeled using a heavy-tailed, double-Pareto Lognormal distribution. The model is estimated via direct maximization of the likelihood function.
result The model with dependence and overdispersion in inter-losses times leads to higher capital charges compared to a Poisson process.
Logitron combines Perceptron and logistic loss for improved classification.
problem Non-convex and non-smooth zero-one loss function in classification models.
method Introduces a Perceptron-augmented convex classification framework with an extended logistic loss function.
result Hinge-Logitron outperforms logistic regression and SVM in classification accuracy.
Symmetric losses improve classifier robustness from corrupted labels.
problem Improving classifier performance from corrupted labels.
method Symmetric losses that satisfy a certain condition.
result Symmetric losses enhance robust classification from corrupted labels.
Two new algorithms improve performance in adversarial bandits with unbounded losses.
problem Adversarial Multi-Armed Bandits with unbounded losses.
method Developed UMAB-NN and UMAB-G for non-negative and general unbounded losses respectively.
result UMAB-NN achieves the first adaptive and scale-free regret bound for non-negative unbounded losses.
Paper explores connections between loss functions and consistency in binary classification and regression.
problem Consistency in binary classification and regression applications.
method Characterization of conformable loss functions and derivation of a new Huber-type loss function.
result Margin-based loss functions are equivalent to loss functions of squared standardized logistic regression residuals.
Paper introduces a new topological loss for better convergence.
problem Optimizing topological losses for model's desired topological behavior.
method Introduces a new regularized topology-aware loss function.
result Guarantees efficient optimization of the new loss function.
Novel loss functions improve decision tree learning from noisy data.
problem Training decision trees with noisy labels.
method Introducing distribution losses and a new negative exponential loss.
result The negative exponential loss leads to efficient and robust decision tree learning.
Theoretical analysis of cross-entropy loss functions and their robustness.
problem Guarantees for using cross-entropy as a surrogate loss function.
method Theoretical analysis of a broad family of loss functions, including cross-entropy.
result First H-consistency bounds for comp-sum losses and smooth adversarial comp-sum losses. A new loss function α-loss improves classification robustness and calibration.
problem Improving classification robustness and calibration in machine learning.
method Introduces a tunable loss function α-loss, parameterized by α, and analyzes its theoretical and practical properties. result The α-loss function can improve model robustness to label flips and sensitivity to imbalanced classes. This paper improves operational risk modeling by selecting better loss severity distributions.
problem Inconsistent regulatory capital calculations due to changing loss severity distribution families.
method Presented truncation probability estimates and a consistent quantile scoring function for selection criteria. Also, recommended collecting loss frequencies below the minimum reporting threshold.
result More stable regulatory capital calculations through better selection of loss severity distributions.
This paper improves loss functions for deep learning with noisy labels.
problem Training deep neural networks with noisy labels.
method The paper introduces a normalization technique to make any loss function robust to noisy labels and proposes a framework called Active Passive Loss (APL) to combine robust loss functions.
result The proposed APL framework consistently outperforms state-of-the-art methods, especially under high noise rates.
Introduces MWLD to measure loss inequality across groups.
problem Machine learning's focus on average loss can lead to large group loss discrepancies.
method Defines MWLD, relates it to fairness and robustness, and provides estimation methods.
result MWLD can be estimated efficiently under certain weighting functions and reduces loss variance without significant accuracy loss.
We study cross-country GDP losses due to financial crises in terms of frequency (number of loss events per period) and severity (loss per occurrence). We perform the Loss Distribution Approach (LDA) to estimate a multi-country aggregate GDP loss probability density function and the percentiles associated to extreme eve…
The paper proves deep learning can be robust with certain loss functions.
problem The robustness of deep learning models under flawed data.
method Empirical-risk minimization with unbounded, Lipschitz-continuous loss functions.
result These loss functions provide efficient prediction under minimal data assumptions.
Paper introduces Fenchel-Young losses for supervised learning tasks.
problem Choosing the right loss function for supervised learning tasks.
method Introduces Fenchel-Young losses as a generic way to construct convex loss functions.
result Fenchel-Young losses unify and create new loss functions.
The paper explores transferability of adversarial examples between convex and 01 loss models, finding non-transferability due to different decision boundaries caused by outliers.
problem Transferability of adversarial examples between convex and 01 loss models.
method Empirical study of transferability between linear 01 loss and convex (hinge) loss models, and between neural networks with different activation functions.
result Adversarial examples are non-transferable between convex and 01 loss models due to different decision boundaries caused by outliers.
Symmetrizes loss functions to improve neural network robustness against noisy labels.
problem Designing robust loss functions for noisy labels in neural networks.
method Symmetrization of multi-class loss functions, focusing on cross-entropy and unhinged loss.
result The multi-class unhinged loss is the unique convex symmetric loss under suitable assumptions.
The study explores loss functions for learning distributions, finding the log loss and others are sufficient under certain conditions.
problem Understanding loss functions for distribution learning and density estimation.
method An axiomatic approach to design loss functions, proposing criteria and showing that no single loss function satisfies all criteria.
result No loss function satisfies all criteria, but the log loss and others do under the condition of candidate distributions being calibrated.
EnsLoss combines multiple loss functions to prevent overfitting in classification.
problem Preventing overfitting in classification models.
method EnsLoss is an ensemble method that combines loss functions, ensuring calibration and consistency.
result EnsLoss improves classification accuracy compared to fixed loss methods.
Study compares metric learning loss functions for speaker verification.
problem Comparing metric learning loss functions for end-to-end speaker verification.
method Cross entropy loss, cosine loss, angular margin loss, center loss, contrastive loss, triplet loss.
result Additive angular margin loss outperforms other loss functions.
SoftAdapt dynamically adjusts loss weights for multi-part functions.
problem Slow convergence and poor weight selection for multi-part loss functions.
method SoftAdapt dynamically changes weights based on live performance statistics.
result Improved convergence and better weight selection for multi-part loss functions.
In this work, we introduce the {\em average top-k} (\atk) loss as a new aggregate loss for supervised learning, which is the average over the k largest individual losses over a training dataset. We show that the \atk loss is a natural generalization of the two widely used aggregate losses, namely the average loss a…
Novel approach embeds loss tunnels in neural networks, revealing insights into their structure.
problem Understanding the structure of neural network loss surfaces, especially low-loss tunnels.
method Directly embedding loss tunnels into the loss landscape of neural networks.
result Improved insights into the length and structure of loss tunnels, and better subspace inference in Bayesian neural networks.
We establish linear regret bounds for convex smooth losses using Fenchel-Young losses.
problem Establishing linear regret bounds for convex smooth losses.
method Constructing a convex smooth surrogate loss using Fenchel-Young losses generated by the convolutional negentropy.
result We derive a smooth loss with a linear surrogate regret bound.