We consider the problem of estimating a low-rank matrix from a noisy observed matrix. Previous work has shown that the optimal method depends crucially on the choice of loss function. In this paper, we use a family of weighted loss functions, which arise naturally for problems such as submatrix denoising, denoising wit…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Weighted Loss Functions”
SoftAdapt: Techniques for Adaptive Loss Weighting of Neural Networks with Multi-Part Loss Functionscs.LG
Adaptive loss function formulation is an active area of research and has gained a great deal of popularity in recent years, following the success of deep learning. However, existing frameworks of adaptive loss functions often suffer from slow convergence and poor choice of weights for the loss components. Traditionally…
Real-world large-scale datasets usually contain noisy labels and are imbalanced. Therefore, we propose derivative manipulation (DM), a novel and general example weighting approach for training robust deep models under these adverse conditions. DM has two main merits. First, loss function and example weighting are commo…
Though machine learning algorithms excel at minimizing the average loss over a population, this might lead to large discrepancies between the losses across groups within the population. To capture this inequality, we introduce and study a notion we call maximum weighted loss discrepancy (MWLD), the maximum (weighted) d…
New loss function connects learning rate and momentum.
problem Finding optimal learning rate and momentum empirically.
method Proposes a new information-theoretical loss function.
result Loss, learning rate, and momentum are closely connected.
New loss function restores importance weighting in overparameterized models.
problem Restoring importance weighting in overparameterized neural networks.
method Introduced polynomially-tailed losses to restore effects of importance weighting.
result Polynomially-tailed losses improve performance in correcting distribution shift.
Paper develops a framework to optimize neural networks using weighted metrics.
problem Discrepancy between maximizing weighted classification scores and minimizing loss function.
method Formalizes weighted classification metrics and constructs corresponding losses.
result Framework includes well-established approaches like cost-sensitive learning and weighted cross entropy.
Regularizers change the geometric properties of loss functions in neural networks.
problem Understanding how different regularizers affect the geometric properties of loss functions in neural networks.
method Examined several regularizers, including weight decay, to determine if the regularized loss function becomes Morse.
result For certain regularizers, the regularized loss function becomes Morse, indicating a change in geometric properties.
A new method for exponentially weighted moving models using approximations.
problem Efficiently updating moving averages for time series data.
method Approximates EWMM using a fixed window and quadratic term, solving non-growing problems.
result Approximation produces estimates similar to exact EWMM.
Proposes a method to generate prediction intervals using weighted asymmetric loss functions.
problem Generating reliable prediction intervals for neural network models.
method Uses a weighted asymmetric loss function to estimate prediction intervals.
result The method produces reliable prediction intervals in complex machine learning scenarios.
Much of the focus in machine learning research is placed in creating new architectures and optimization methods, but the overall loss function is seldom questioned. This paper interprets machine learning from a multi-objective optimization perspective, showing the limitations of the default linear combination of loss f…
AWP improves robustness by flattening weight loss landscape.
problem Improving robustness of deep neural networks against adversarial examples.
method Explicitly regularizes the flatness of weight loss landscape through adversarial weight perturbation.
result AWP forms a double-perturbation mechanism in adversarial training, leading to flatter weight loss landscape.
In this paper, we propose a new metric to measure goodness-of-fit for classifiers, the Real World Cost function. This metric factors in information about a real world problem, such as financial impact, that other measures like accuracy or F1 do not. This metric is also more directly interpretable for users. To optimize…
New aggregation strategy handles unbounded losses with regret bounds.
problem Online optimization with unbounded loss functions.
method Follow The Regularized Leader (FTRL) with φ-divergence.
result Worst regret bound for unbounded losses with alternative divergences.
This paper improves forecast stability without sacrificing accuracy using dynamic loss weighting.
problem Rolling origin forecast instability in time series forecasting.
method Dynamic loss weighting algorithms applied to the N-BEATS model.
result Dynamic loss weighting can further improve forecast stability without compromising accuracy.
Improved loss functions adapt to weight-space anisotropy, outperforming isotropic counterparts.
problem Adapting to the anisotropic nature of deep weight spaces for better performance.
method Refined local entropic loss functions restricted to a subset of weights, exploiting anisotropy.
result Partial local entropies outperform isotropic counterparts on image classification tasks.
Optimal weight windows are symmetric rectangles centered at peak.
problem Finding the best weight windows for weighted least squares.
method Investigated symmetric and tapered rectangle window weights, showing the best rectangle window is optimal.
result The best rectangle window is optimal for all tapered rectangle window definitions.
This paper introduces new loss functions for balanced multi-class classification.
problem Balancing class imbalance in multi-class classification.
method Introduces two new surrogate loss families: GLA and GCA.
result GCA losses offer stronger theoretical guarantees in imbalanced settings.
Deep convolutional neural networks (CNNs) can be applied to malware binary detection via image classification. The performance, however, is degraded due to the imbalance of malware families (classes). To mitigate this issue, we propose a simple yet effective weighted softmax loss which can be employed as the final laye…
HydaLearn dynamically adjusts task weights for better MTL performance.
problem Constant loss weights in MTL lead to poor results due to drifting relevance and varying mini-batch composition.
method HydaLearn uses mini-batch gradients to dynamically adjust task weights.
result HydaLearn improves performance on synthetic and real-world data.
Ranking is a key aspect of many applications, such as information retrieval, question answering, ad placement and recommender systems. Learning to rank has the goal of estimating a ranking model automatically from training data. In practical settings, the task often reduces to estimating a rank functional of an object …
New loss function helps learn unstable dynamical systems.
problem Gradient descent fails to learn unstable dynamical systems.
method Introduced a time-weighted logarithmic loss function.
result Time-weighted loss function effectively learns unstable systems.
Neural networks' weights don't converge to stationary points but training loss stabilizes.
problem The disconnect between theoretical analyses and neural network training practice.
method An invariant measure perspective inspired by ergodic theory of dynamical systems.
result The distribution of weights converges to an approximate invariant measure, explaining loss stabilization.
Heuristic weighting improves denoising score matching without requiring noise distribution assumptions.
problem Improving denoising score matching without assuming noise distribution.
method Demonstrated heteroskedasticity, derived optimal weighting functions, and provided theoretical and empirical comparisons.
result Heuristical weighting function can achieve lower variance than optimal weighting, facilitating more stable and efficient training.
Deep generative networks can simulate from a complex target distribution, by minimizing a loss with respect to samples from that distribution. However, often we do not have direct access to our target distribution - our data may be subject to sample selection bias, or may be from a different but related distribution. W…
Diffusion models accurately recover mixture weights from generated samples despite score function insensitivity.
problem Score-based generative models often fail to learn correct relative mode amplitudes (mixture weights) from generated samples.
method Relate diffusion score matching (DSM) loss to mixture weight estimation error, define diffusion score sensitivity index (DSSI), and prove its governing role in mixture weight recovery.
result Generated samples can accurately recover mixture weights from the DSM loss, even when the target score is insensitive to mixture weights.
WES improves neural network regression by stretching distribution error.
problem Improving prediction performance in neural-network-based regression.
method Proposed weighted empirical stretching (WES) loss function.
result WES outperforms existing loss functions, especially in extreme domains.
New loss functions improve extreme classification with missing labels.
problem Large number of infrequent labels and missing labels in XMC.
method Derive unbiased loss functions for XMC, incorporating them into existing algorithms.
result Significant improvement in extreme classification performance (up to 20%) over existing methods.
Ensemble techniques are powerful approaches that combine several weak learners to build a stronger one. As a meta-learning framework, ensemble techniques can easily be applied to many machine learning methods. Inspired by ensemble techniques, in this paper we propose an ensemble loss functions applied to a simple regre…
Real-world machine learning applications often have complex test metrics, and may have training and test data that are not identically distributed. Motivated by known connections between complex test metrics and cost-weighted learning, we propose addressing these issues by using a weighted loss function with a standard…
Paper explores connections between loss functions and consistency in binary classification and regression.
problem Consistency in binary classification and regression applications.
method Characterization of conformable loss functions and derivation of a new Huber-type loss function.
result Margin-based loss functions are equivalent to loss functions of squared standardized logistic regression residuals.
Paper introduces new loss functions for Siamese networks using FDA.
problem Training Siamese networks with improved loss functions.
method Proposes Fisher Discriminant Triplet (FDT) and Fisher Discriminant Contrastive (FDC) loss functions based on FDA.
result Shows effectiveness of FDT and FDC on MNIST and histopathology datasets.
Binary classification rules based on covariates typically depend on simple loss functions such as zero-one misclassification. Some cases may require more complex loss functions. For example, individual-level monitoring of HIV-infected individuals on antiretroviral therapy (ART) requires periodic assessment of treatment…
Adversarial training makes logistic regression weight loss landscapes sharper.
problem Understanding why adversarial training sharpens the weight loss landscape in logistic regression.
method Theoretical analysis of linear logistic regression model with L2 norm constraints, and experiments on ResNet18.
result Adversarial training sharpens the weight loss landscape in linear logistic regression models.
Framework for differentiating WFSTs for structured loss functions.
problem Training dynamic structured loss functions in neural networks.
method Automatic differentiation with WFSTs, combining pruning and back-off.
result Demonstrated learning over WFST latent phrase decomposition.
In this paper, we propose a Dual Focal Loss (DFL) function, as a replacement for the standard cross entropy (CE) function to achieve a better treatment of the unbalanced classes in a dataset. Our DFL method is an improvement on the recently reported Focal Loss (FL) cross-entropy function, which proposes a scaling metho…
A new loss function for VAEs improves image quality and efficiency.
problem Training VAEs to generate realistic images requires a loss function that reflects human perception.
method Based on Watson's perceptual model, the loss function computes a weighted distance in frequency space, accounts for luminance and contrast masking, and is extended to color images.
result VAEs trained with the new loss function generated high-quality, less blurry images with fewer artifacts and less computational resources.
Proposes neuron alignment to optimize mode connectivity in neural networks.
problem Understanding and optimizing mode connectivity in deep neural networks.
method Introduces neuron alignment to approximate optimal weight permutations and improve mode connectivity.
result Neuron alignment significantly alleviates robust loss barriers and improves model robustness and accuracy.
In multi-task learning, difficulty levels of different tasks are varying. There are many works to handle this situation and we classify them into five categories, including the direct sum approach, the weighted sum approach, the maximum approach, the curriculum learning approach, and the multi-objective optimization ap…
A new loss function boosts AI's stock trading performance.
problem Improving AI's ability to predict stock prices and make profitable trades.
method Introducing a return-weighted loss function for deep learning models.
result Best models achieve high annual returns and Sharpe Ratios.
The study examines correlations of logarithms of integers at different scalings.
problem Analyzing pair correlations of logarithms of integers at various scalings.
method Examined correlations of logarithms of positive integers at different scalings, proving the existence of pair correlation functions.
result Level repulsion at linear scaling, total loss of mass at superlinear scalings, and Poissonian behavior at sublinear scalings.
The multi-label classification framework, where each observation can be associated with a set of labels, has generated a tremendous amount of attention over recent years. The modern multi-label problems are typically large-scale in terms of number of observations, features and labels, and the amount of labels can even …
This research shows loss weighting remains effective in last layer retraining despite model overparameterization.
problem Overcoming biases in machine learning models at scale.
method Theoretical and practical exploration of last layer retraining in an overparameterized setting.
result Loss weighting is still effective in last layer retraining, but weights must account for model overparameterization.
We present a generalization of the adversarial linear bandits framework, where the underlying losses are kernel functions (with an associated reproducing kernel Hilbert space) rather than linear functions. We study a version of the exponential weights algorithm and bound its regret in this setting. Under conditions on …
Robust Variable Selection for High-dimensional Regression with Missing Data and Measurement Errorsstat.ME
Robust variable selection for high-dimensional data with missing and measurement errors.
problem Missing data and measurement errors confound data distribution.
method Exponential loss function with inverse probability weighting and additive error models.
result The Atan punishment method improves robust variable selection.
New learning process for neural classifiers simplifies decision weights.
problem Training decision layer weights in neural classifiers.
method Choosing appropriate loss functions and solving constrained optimization problems.
result A new learning process for pre-decision weights that is simple and effective.
Composite Binary Lossesstat.ML
We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determ…
Mitigates anomaly score imbalance in long-tailed distributions.
problem Class imbalance in normal data leads to skewed anomaly detection performance.
method Proposes an importance-weighted loss function to balance anomaly scores.
result Improves anomaly detection performance by 0.043 on real-world datasets.