In this manuscript we propose two objective terms for neural image compression: a compression objective and a cycle loss. These terms are applied on the encoder output of an autoencoder and are used in combination with reconstruction losses. The compression objective encourages sparsity and low entropy in the activatio…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “loss differences”
The paper explores transferability of adversarial examples between convex and 01 loss models, finding non-transferability due to different decision boundaries caused by outliers.
problem Transferability of adversarial examples between convex and 01 loss models.
method Empirical study of transferability between linear 01 loss and convex (hinge) loss models, and between neural networks with different activation functions.
result Adversarial examples are non-transferable between convex and 01 loss models due to different decision boundaries caused by outliers.
Given two networks with the same training loss on a dataset, when would they have drastically different test losses and errors? Better understanding of this question of generalization may improve practical applications of deep networks. In this paper we show that with cross-entropy loss it is surprisingly simple to ind…
We carefully study how well minimizing convex surrogate loss functions, corresponds to minimizing the misclassification error rate for the problem of binary classification with linear predictors. In particular, we show that amongst all convex surrogate losses, the hinge loss gives essentially the best possible bound, o…
The performance of standard learning procedures has been observed to differ widely across groups. Recent studies usually attribute this loss discrepancy to an information deficiency for one group (e.g., one group has less data). In this work, we point to a more subtle source of loss discrepancy---feature noise. Our mai…
Unified SVM algorithm for various losses with fast training.
problem Training SVM models with different convex or nonconvex losses.
method Introducing LS-DC loss, proposing DCA-based UniSVM algorithm.
result Unified algorithm solves SVM models with any convex or nonconvex LS-DC loss efficiently.
Regularizers change the geometric properties of loss functions in neural networks.
problem Understanding how different regularizers affect the geometric properties of loss functions in neural networks.
method Examined several regularizers, including weight decay, to determine if the regularized loss function becomes Morse.
result For certain regularizers, the regularized loss function becomes Morse, indicating a change in geometric properties.
Offline Signature Verification (OSV) is a challenging pattern recognition task, especially in presence of skilled forgeries that are not available during training. This study aims to tackle its challenges and meet the substantial need for generalization for OSV by examining different loss functions for Convolutional Ne…
We present and study a partial-information model of online learning, where a decision maker repeatedly chooses from a finite set of actions, and observes some subset of the associated losses. This naturally models several situations where the losses of different actions are related, and knowing the loss of one action p…
New loss function equivalence reveals PER's uniform sampling can be improved.
problem Improving Prioritized Experience Replay (PER) for better learning efficiency.
method Transforming non-uniformly sampled data loss functions into uniformly sampled ones.
result Some environments can replace PER with a new loss function without performance loss.
The paper predicts loss scaling across different datasets and compute scales.
problem Predicting loss scaling across different datasets and compute scales.
method Derive shifted power law relationships between train and test losses.
result Shifted power law relationships hold for various datasets and tasks, improving prediction accuracy.
Though machine learning algorithms excel at minimizing the average loss over a population, this might lead to large discrepancies between the losses across groups within the population. To capture this inequality, we introduce and study a notion we call maximum weighted loss discrepancy (MWLD), the maximum (weighted) d…
Paper proposes a loss extension for neural networks to improve OSR performance.
problem Open set recognition problem, distinguishing known and unknown classes.
method Introduces a loss function extension to find more discriminative polar representations.
result Significantly improves performance on datasets from different domains.
The paper explores how different loss functions impact reinforcement learning algorithms.
problem Improving reinforcement learning algorithms by optimizing loss functions.
method Comprehensive survey on loss functions in reinforcement learning, proving the benefits of specific loss functions.
result Binary cross-entropy loss leads to first-order bounds and is more efficient than squared loss.
Omnipredictorscs.LG
New machine learning paradigm ignores loss function until action time.
problem Learning with unknown loss functions.
method Introduces omnipredictors for any loss function.
result Extracts predictive power from any class, ignoring loss function.
01 loss neural network improves adversarial robustness on CIFAR10.
problem Improving machine learning models' resistance to adversarial attacks.
method Proposes a single hidden layer 01 loss neural network trained with stochastic coordinate descent.
result 01 loss network has the largest adversarial distortion compared to other models.
We study the asymptotic behavior of the difference between the values at risk VaR(L) and VaR(L+S) for heavy tailed random variables L and S for application in sensitivity analysis of quantitative operational risk management within the framework of the advanced measurement approach of Basel II (and III). Here L describe…
The paper presents a multi-power law for predicting loss curves across different learning rate schedules.
problem Understanding and optimizing the relationship between model performance and hyperparameters, especially learning rates.
method Proposes a multi-power law that combines power laws based on the sum of learning rates and additional laws for loss reduction due to decay.
result The multi-power law accurately predicts loss curves for unseen learning rate schedules and finds a schedule that outperforms cosine learning rate.
Classification and regression tasks in overparameterized models show different generalization properties.
problem Comparing classification and regression in overparameterized models.
method Comparison of least-squares minimum-norm interpolation and hard-margin SVM using different loss functions.
result Interpolating solutions generalize well with 0-1 loss but not with square loss.
This work classifies SOC loss functions based on their gradient properties.
problem Optimizing noisy systems in stochastic optimal control.
method Grouping loss functions into classes with the same gradient expectation.
result Different loss functions have the same optimization landscape but differ in gradient variance.
Extends LOSS invariant naturality to positive contact surgeries.
problem Naturality of LOSS invariant under positive contact surgeries.
method Proves naturality of LOSS invariant for Legendrian knots under positive integer contact surgeries.
result Contact invariant of +n surgery depends only on classical invariants of S.
Analyzes double descent in binary classification models with different losses.
problem Understanding the double descent phenomenon in binary classification models.
method Analytic study of gradient descent with logistic and square losses on binary linear classification models.
result The double descent phenomenon persists but with differences compared to logistic loss.
We analyze systems of agents sharing light-tailed risky claims issued by different financial objects. Assuming exponentially distributed claims, we obtain that both agents' and system's losses follow generalized exponential mixture distributions. We show that this leads to qualitatively different results on individual …
Under the Basel II standards, the Operational Risk (OpRisk) advanced measurement approach allows a provision for reduction of capital as a result of insurance mitigation of up to 20%. This paper studies the behaviour of different insurance policies in the context of capital reduction for a range of possible extreme los…
Proposes a hierarchical curriculum loss to improve model accuracy and interpretability.
problem Flat label spaces in classification algorithms fail to capture dependencies in real-world data.
method Introduces hierarchical curriculum loss with two properties: satisfying hierarchical constraints and providing non-uniform label weights.
result The proposed loss function significantly outperforms multiple baselines on real-world image datasets.
Study on using random subspaces for ERM with various loss functions.
problem Improving learning accuracy with computational savings from random subspaces.
method Random subspaces of a hypothesis space, considering data-dependent subspaces.
result Unified analysis showing computational efficiency can be improved without performance loss.
New loss function helps learn unstable dynamical systems.
problem Gradient descent fails to learn unstable dynamical systems.
method Introduced a time-weighted logarithmic loss function.
result Time-weighted loss function effectively learns unstable systems.
We analyze the semi-hard triplet loss using Edgeworth expansion for better understanding of its behavior.
problem Understanding the behavior of the semi-hard triplet loss function.
method Developed a higher-order asymptotic analysis using the Edgeworth expansion.
result Derived explicit Edgeworth expansions revealing first-order corrections in terms of the third cumulant.
We demonstrate that, in the classical non-stochastic regret minimization problem with decisions, gains and losses to be respectively maximized or minimized are fundamentally different. Indeed, by considering the additional sparsity assumption (at each stage, at most decisions incur a nonzero outcome), we derive…
Paper proposes a hybrid loss function for graph self-supervised learning.
problem Improving self-supervised representation learning for graphs.
method Hybrid loss function combining VICReg and HSIC.
result Hybrid loss function outperformed other methods in 4 out of 7 cases.
We present -loss, , a tunable loss function for binary classification that bridges log-loss () and - loss (). We prove that -loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal y…
A new framework combines multiple loss reserving models for better predictive performance.
problem Combining multiple loss reserving models to improve predictive performance.
method Systematic framework that considers full distributional properties and features of reserving data.
result Optimized ensemble outperforms traditional methods and captures relevant quantiles.
The critical locus of the loss function of a neural network is determined by the geometry of the functional space and by the parameterization of this space by the network's weights. We introduce a natural distinction between pure critical points, which only depend on the functional space, and spurious critical points, …
The L1 loss landscape of neural nets near local minima behaves differently, revealing exponential decay and increased vertex density.
problem Understanding the L1 loss landscape of neural nets near local minima.
method Iterative minimization of the loss function on adjacent vertices of the Deep ReLU Simplex algorithm.
result Exponential decay of loss levels and increased vertex density around local minima.
Teaching is critical to human society: it is with teaching that prospective students are educated and human civilization can be inherited and advanced. A good teacher not only provides his/her students with qualified teaching materials (e.g., textbooks), but also sets up appropriate learning objectives (e.g., course pr…
Paper proposes ARB-Loss to improve classification precision in imbalanced datasets.
problem Improving classification precision on minor classes in imbalanced datasets.
method Introduces Attraction-Repulsion-Balanced Loss (ARB-Loss) to balance gradients across different classes.
result ARB-Loss achieves state-of-the-art performance with one-stage training.
We analyze impermanent loss in AMMs and show G3Ms are simplest.
problem Understanding impermanent loss in automated market makers.
method Developed a general framework and analyzed Geometric Mean Market Makers (G3Ms).
result G3Ms have the simplest impermanent loss characteristics.
Paper proposes a principled method to learn loss functions for supervised learning tasks.
problem Choosing an appropriate loss function for supervised learning tasks.
method The paper revisits and generalizes the SLIsotron algorithm using Bregman divergences.
result The BregmanTron algorithm learns both the loss and classifier, with convergence guarantees.
Dependence bounds for the difference of stop-loss payoffs on the difference of two random variablesq-fin.PR
The paper examines bounds for stop-loss payoffs using transformed random variables.
problem Bounding stop-loss payoffs for a difference of two random variables.
method Analyzes crossing points of cdfs of original and transformed random variables.
result Unique pairwise crossing points for mortality-linked securities under symmetric copulas.
Introduces SoRR for aggregating losses in supervised learning.
problem Aggregating individual losses into a single output for machine learning models.
method Sum of ranked range (SoRR) minimization using DCA.
result Demonstrates effectiveness of AoRR and TKML in improving robustness of multi-label learning.
Informer model with GMADL loss outperforms benchmarks in high frequency Bitcoin trading.
problem Developing automated trading strategies for high frequency Bitcoin data.
method Informer architecture with RMSE, GMADL, and Quantile loss functions.
result Informer model with GMADL loss function outperforms benchmarks in trading outcomes.
Mixed precision training (MPT) is becoming a practical technique to improve the speed and energy efficiency of training deep neural networks by leveraging the fast hardware support for IEEE half-precision floating point that is available in existing GPUs. MPT is typically used in combination with a technique called los…
Adapting robust statistics to neural networks, researchers found neural networks can be more robust with certain loss functions.
problem The robustness of neural networks in complex learning tasks.
method Adapting the regression breakdown point from robust statistics to neural networks and comparing different configurations and contamination settings.
result Neural networks can benefit from robust loss functions, as demonstrated in extensive simulations.
Unified stability bounds for noisy SGD across convex and non-convex losses.
problem Deriving generalization bounds for noisy stochastic gradient descent.
method Unified approach using Lyapunov functions and applied probability.
result Time-uniform stability bounds for SGD on various loss functions.
Researchers improve visualization of neural network loss landscapes.
problem Understanding neural network generalization performance.
method Novel 'jump and retrain' procedure, non-linear dimensionality reduction (PHATE), computational homology.
result Improved visualization and quantification of neural network generalization performance.
Custom loss functions improve accuracy of wildfire rate of spread forecasts.
problem Improving accuracy of wildfire rate of spread forecasts.
method Examined custom loss functions in machine learning models of fuel moisture content.
result Custom loss functions improved accuracy of ROS forecasts by a small amount.
Neural network training relies on our ability to find "good" minimizers of highly non-convex loss functions. It is well-known that certain network architecture designs (e.g., skip connections) produce loss functions that train easier, and well-chosen training parameters (batch size, learning rate, optimizer) produce mi…
Proposes Group Loss for deep metric learning to improve clustering and image retrieval.
problem Improving deep metric learning for better clustering and image retrieval.
method Group Loss based on label-propagation method enforcing embedding similarity across all samples of a group.
result Shows state-of-the-art results on clustering and image retrieval on several datasets.